Repository navigation
fix: treat symlinks as symlinks on direct binds in test -L and rm - #377
ferdingler wants to merge 4 commits into
Conversation
|
@strandly-the-agent Please review this PR |
strandly-the-agent
left a comment
There was a problem hiding this comment.
Changes requested. The fix does what it claims on Linux and containment holds, but it introduces one regression: inside a symlinked directory on a bind, find . and ls -l now see the link instead of the directory's contents. A small fix is suggested inline.
- 🔴 After
cd /b/ld(a link to a directory),find . -name keepprints nothing with exit 0. It works onmainand in bash.ls -l /b/ld/andfind /b/ld/regress the same way. - 🟡 The new tests don't cover read-only binds or links that leave the bind. Two deliberate breakages of the new code pass the whole suite.
- 🟡
find,ls -lRandrm -ron a large bind are about 50% slower, because each path is now resolved twice.
Three bugs that exist on main with or without this PR are filed separately: #385, #386, #387.
What was verified, reading order, appendix
Verified at head 8428bd7 against base dbb6189 (Debian 12 aarch64, Rust 1.100.0)
cargo test --workspace --all-targetson head: exit 0, 2735 passed, 0 failed (2064 inshell_integration).- Base plus only this PR's test changes:
bind_direct_test_h_sees_symlinksfails at:9021andbind_direct_rm_removes_symlinks_not_their_targetsat:9046. Both pass on head. Reverting either theremove_fileor theremove_dirhunk alone makes the rm test fail, so both hunks are needed and pinned. test_symlink_flags.expectedmatches bash 5.2 output. As the description says, it runs on the in-memory filesystem only and passes on base too.- About 360 black-box cases run on base, head and bash 5.2 / coreutils 9.1, each on a fresh fixture:
- Links that leave the bind (relative, absolute,
/etc/passwd,-> /), dangling links, self-loops and long chains:test -Lis 0 andrmremoves only the link. Nothing outside the bind source changed. - A link reached through a directory link that leaves the bind:
test -L,rm,rm -rf,rmdir,cat,ls -l,readlinkare all refused, same as base. - Read-only bind: 17 commands plus Lua
os.removeall fail with "read-only bind mount" and the host is unchanged. - Cyclic links (
d/back -> ..):findandrm -rfinish on head; base hits an 8-second timeout. - A host process swapping the leaf between an escaping link, an in-bind link and a file during 120,000
rm -fruns and 80,000rm -rfruns: no escape. The interleaving is uncontrolled, so this is not proof.
- Links that leave the bind (relative, absolute,
- CI is green on all jobs, including
Rust (windows-latest)andRust (macos-latest).
Not run: Windows, macOS and wasm32 locally; the Python and Node bindings; clippy.
Questions: none blocking.
Reading order: resolve_host_symlink (src/vfs_kernel.rs:138), then its three callers (:542, :661, :683), then host_symlink_stat (:1250), then the two tests.
Appendix, non-blocking (4)
- ⚪
src/commands/readlink.rs:209says "Direct-bind metadata follows links". That is no longer true after this PR, and the unconditionalread_linkbelow it could go back toif st.is_symlink. - ⚪ The description says everything else on binds works as before. A few other things change, all toward GNU:
ls -lshows links asl… -> target,findno longer descends into linked directories and-type lmatches, andremove_filefrom the Rust, Python, Node and Lua APIs unlinks the link. Worth a line in the description. - ⚪ Windows, not run:
std::fs::remove_fileis documented to fail on a directory symlink there, sorm dirlinkmay error whilermdir dirlinksucceeds. Both new tests are#[cfg(unix)]. - Pre-existing, filed: #385 (
rm -rof an in-memory link to a bind root deletes the bound host directory), #386 (mvof a link on a bind renames the target), #387 (the./ trailing-slash behaviour in the in-memory filesystem).
| if let Some((_, _, meta)) = Self::resolve_host_symlink(&vfs, &abs) { | ||
| return host_symlink_stat(&meta); | ||
| } |
There was a problem hiding this comment.
🔴 find . and ls -l inside a symlinked directory on a bind now see the link, not its contents
With ld -> d as a host symlink in a direct bind, cd /b/ld && find . -name keep prints nothing and exits 0, and ls -l prints . -> d. On main and in bash both list the contents of d. ls -l /b/ld/ and find /b/ld/ change the same way. vfs::absolute drops the final . and the trailing / before this branch runs, so lstat reports the link where POSIX follows it. A search run from inside a symlinked directory silently returns no results.
Suggestion: take the symlink branch only when the caller's path names the link itself.
| if let Some((_, _, meta)) = Self::resolve_host_symlink(&vfs, &abs) { | |
| return host_symlink_stat(&meta); | |
| } | |
| // `link/` and `link/.` name the directory the link points at, not the link. | |
| let names_link = | |
| !path.ends_with('/') && !matches!(path.rsplit('/').next(), Some("." | "..")); | |
| if names_link && let Some((_, _, meta)) = Self::resolve_host_symlink(&vfs, &abs) { | |
| return host_symlink_stat(&meta); | |
| } |
One consequence to decide on: with this, rm -r /b/ld/ (trailing slash) deletes the contents of d and then fails with "Not a directory", as GNU rm and main do. At this commit it removes only the link. The in-memory filesystem has the same . / trailing-slash behaviour on main (#387), so a check placed ahead of both branches of lstat would fix that too.
Reproduction details
Reviewed 8428bd7; compared with base dbb6189 and bash 5.2 on Debian 12 aarch64. Debug builds, one target directory per revision.
Setup:
mkdir -p t1/src/d/sub && echo k > t1/src/d/keep && echo deep > t1/src/d/sub/deep && ln -s d t1/src/ld
printf '[[bind]]\nmode = "direct"\nsource = "%s/t1/src"\ndestination = "/b"\n' "$PWD" > t1.toml
strands-shell --config t1.toml -c 'cd /b/ld && find . -name keep; ls -l; ls -l /b/ld/; find /b/ld/ -name keep; test -L /b/ld/; echo $?'| Command | base | head | bash | head + suggestion |
|---|---|---|---|---|
cd /b/ld && find . -name keep |
./keep |
nothing, exit 0 | ./keep |
./keep |
cd /b/ld && ls -l |
lists keep, sub |
. -> d |
lists keep, sub |
lists keep, sub |
ls -l /b/ld/ |
lists contents | /b/ld/ -> d |
lists contents | lists contents |
find /b/ld/ -name keep |
/b/ld/keep |
nothing | /b/ld/keep |
/b/ld/keep |
test -L /b/ld/ |
1 | 0 | 1 | 1 |
rm -r /b/ld/ |
not rerun | exit 0, link removed, d intact |
exit 1, d emptied, link kept |
exit 1, d emptied, link kept |
With the suggestion applied to 8428bd7:
cargo test --workspace --all-targetsexits 0 with 2735 passed, 0 failed, andcargo fmt --checkpasses.- The PR's behaviour is kept:
test -L /b/ldis 0,rm /b/lfandrm /b/ldremove only the link, andrmdir /b/ldfails with "Not a directory". rm /b/ld/fails with "is a directory", as in bash.
No existing test covers these forms on a bind: the suite passes with and without the suggestion.
There was a problem hiding this comment.
Fixed in 29e4008 with your suggestion: lstat only reports the link when the path names it, so link/ and link/. follow it. Added a test for cd link && find ., find link/, ls link/ and test -L on both forms; it failed on 8428bd7 and passes now. I kept removal acting on the link itself, so rm -r /b/ld/ now matches GNU and main (empties d, then fails on the link).
There was a problem hiding this comment.
Verified at 29e4008: all six rows from the table now match bash, the new test fails on 8428bd7 and passes here, and removing the ./trailing-slash check makes it fail again. rm -r /b/ld/ matches GNU as you describe.
| assert!(dir.join("e").is_dir()); | ||
| assert!(dir.join("le").symlink_metadata().is_ok()); | ||
| })); | ||
| } |
There was a problem hiding this comment.
🟡 Nothing tests the new symlink branch on a read-only bind, or for links that leave the bind
Both tests plant only links that stay inside a writable bind. Two deliberate breakages of the new code pass the whole suite (2735 passed, 0 failed):
- ignoring the read-only flag for symlinks in
remove_file/remove_dir, after whichrm /b/lfdeletes a link from areadonly = truebind; - dropping the parent containment check from
resolve_host_symlink, after whichrm /b/parentlink/innerlinkdeletes a link outside the bind.
The code in this PR handles both correctly. The gap is that a later change could break either without a test failing, in a file where read-only and containment are the security boundary.
Suggestion: add cases beside this one, in the style of the bind_direct_readlink_* tests above.
- With
bind_direct_readonly:rm /b/lf,rmdir /b/leandrm -r /b/weach fail and the host tree is unchanged. - A link to a file outside the bind, and a dangling link:
test -Lis 0,rmremoves the link, the outside file survives. - A link reached through a directory link that leaves the bind:
test -Lis 1,rmfails, the outside link survives.
Reproduction details
Run on disposable copies of 8428bd7, never on the PR branch. Debian 12 aarch64, Rust 1.100.0.
| Change made to the copy | cargo test --workspace --all-targets |
Probe that does catch it |
|---|---|---|
| none (control) | 2735 passed, 0 failed | n/a |
resolve_host_symlink returns ro = false |
2735 passed, 0 failed | rm /b/lf on a readonly = true bind deletes the host link |
resolve_host_symlink skips the parent containment check |
2735 passed, 0 failed | rm /b/parentlink/innerlink deletes a link outside the bind source |
On the unmodified head the same probes behave correctly: every removal on a read-only bind prints "read-only bind mount" with exit 1 and leaves the host unchanged, and paths through an escaping parent are refused.
There was a problem hiding this comment.
Added in 9aa9d8f: a read-only bind test (rm, rmdir and rm -r on links all fail and the host is unchanged), and a test for links leaving the bind (outside and dangling links are removed and the outside file survives; a link reached through an escaping directory link is refused and survives). I checked the read-only test by forcing the flag off in the new resolver, and it failed.
There was a problem hiding this comment.
Verified at 29e4008: dropping the read-only flag fails bind_direct_readonly_keeps_symlinks, and dropping parent containment or hiding escaping links fails bind_direct_rm_of_links_leaving_the_bind. Both breakages passed the whole suite before.
| fn resolve_host_symlink( | ||
| vfs: &Vfs, | ||
| abs_path: &str, | ||
| ) -> Option<(PathBuf, bool, std::fs::Metadata)> { | ||
| let (host_path, ro) = Self::resolve_host_nofollow(vfs, abs_path)?; | ||
| let meta = std::fs::symlink_metadata(&host_path).ok()?; | ||
| meta.is_symlink().then_some((host_path, ro, meta)) | ||
| } |
There was a problem hiding this comment.
🟡 Every lstat and removal on a bind now resolves the host path twice
resolve_host_symlink canonicalizes the parent and stats the leaf. For anything that is not a symlink, the caller then runs resolve_host on the full path as well. On a bind with about 20,000 entries, find /b went from about 1.6 s to 2.4 s, and ls -lR and rm -r slowed by a similar 45–56%. These are debug builds; release was not measured.
Would it be simpler to resolve once? For a regular file or directory, resolve_host_nofollow plus symlink_metadata already gives the same host path resolve_host would (the parent is canonical and contained, and the leaf is not a link). resolve_host would then only be needed when the no-follow form returns None, which is the bind root. lstat could also build its FileStat from that one metadata call, instead of host_symlink_stat becoming a third copy of the builder in host_stat and host_stat_sync.
I have not built or tested that restructuring.
Reproduction details
Base dbb6189 vs head 8428bd7, debug binaries, Debian 12 aarch64, a direct bind over a host tree of 20,104 entries. Times in ms, one column per run.
| Command | base | head | Change |
|---|---|---|---|
find /b |
1526 / 1621 / 1587 | 2391 / 2371 / 2366 | +50% |
ls -lR /b |
1704 / 1664 / 1603 | 2414 / 2402 / 2414 | +45% |
rm -r /b/a |
2813 / 2750 | 4327 / 4351 | +56% |
A second, independent measurement on a 20,000-file bind gave find /b | wc -l 1575 / 1605 / 1589 on base and 2391 / 2649 / 2408 on head.
There was a problem hiding this comment.
Done in 29e4008. lstat and the removals now resolve once: the no-follow path plus one metadata call, falling back to the old resolver only when that finds nothing (the bind root or a missing path). lstat builds its result from that metadata, and all three callers now share one stat builder instead of each keeping a copy. I did not rerun your timing benchmark.
There was a problem hiding this comment.
Verified at 29e4008. I reran the benchmark on a 20,104-entry bind: find /b about 1.2 s (1.9 s on main, 2.7 s on 8428bd7), with ls -lR and rm -r improved similarly. Containment and read-only probes still hold with the single resolution.
…and links leaving the bind
|
@strandly-the-agent Please review this PR |
strandly-the-agent
left a comment
There was a problem hiding this comment.
Approve. All three findings from my last review are fixed at 29e4008, and I found nothing new.
- 🔴 Symlinked directory named as
.or with a trailing/: fixed.cd /b/ld && find .,ls -l /b/ld/,find /b/ld/andtest -Lon those forms now match bash. The new test fails on8428bd7and passes here. - 🟡 Test gaps: fixed. Six deliberate breakages of the new resolver each fail at least one
bind_direct_*test, including the two that passed the whole suite last time. - 🟡 Double resolution: fixed. On a 20,104-entry bind
find /btakes about 1.2 s, against 1.9 s onmainand 2.7 s on8428bd7(debug builds).
Since the resolver now handles every file type and not only links, I re-ran the containment and read-only probes as well as the repros. Nothing outside the bind source changed in 302 cases.
What was verified, what was not run, appendix
Revisions: new head 29e4008, old head 8428bd7, base dbb6189. Debian 12 aarch64, Rust 1.100.0, debug builds, one target directory per revision.
Suite and lint on 29e4008
cargo test --workspace --all-targets: exit 0, 2738 passed, 0 failed.cargo fmt --checkpasses.cargo clippy --workspace --all-targetsexits 0 with no warnings insrc/vfs_kernel.rsor the new tests.- Old head plus only the new tests:
bind_direct_trailing_slash_or_dot_follows_a_directory_linkfails at:9084(find . -name keepprints nothing). The other 15bind_directtests pass there, as expected, since read-only and containment were already correct.
Mutation check of the new tests (env-gated changes in a disposable copy of 29e4008, cargo test --test shell_integration bind_direct)
| Change | Result | Caught by |
|---|---|---|
| none (control) | 16 passed | n/a |
resolve_host_leaf drops the read-only flag |
1 failed | bind_direct_readonly_keeps_symlinks |
names_leaf always true (last review's regression) |
1 failed | bind_direct_trailing_slash_or_dot_follows_a_directory_link |
resolve_host_leaf skips parent containment |
1 failed | bind_direct_rm_of_links_leaving_the_bind |
links that dangle or leave the bind hidden, as on main |
1 failed | bind_direct_rm_of_links_leaving_the_bind |
| removals follow links again | 2 failed | the two rm tests |
lstat follows links again (#254) |
3 failed | test_h, and the two rm tests |
Behaviour sweep (old head, new head and bash 5.2, each case on a fresh fixture)
- 302 cases. No change under the fixture's
outside/directory on the new head in any of them. - Old and new head differ only in two groups. The
.and trailing-slash forms on in-bind directory links now match bash. A directory link that leaves the bind, named with a trailing slash, is now reported as missing instead of as a link (bash follows it out of the bind; the sandbox refuses). - 41 removal and write cases on a
readonly = truebind: host unchanged in every one. - The only cases where the new head removes more than the old head are
rm -r /b/ld/andrm -rf /b/ld/, which emptydand keep the link. That is the GNU behaviour chosen in the thread. - Nested binds are still rejected at config load, and bind roots behave the same on old and new head.
Timing (ms, 20,104-entry bind, three runs, repeated once with the same picture)
| Command | base | old head | new head |
|---|---|---|---|
find /b |
2060 / 1925 / 1889 | 2666 / 2747 / 2673 | 1243 / 1231 / 1245 |
ls -lR /b |
2046 / 2020 / 2031 | 2750 / 2876 / 2882 | 1298 / 1405 / 1301 |
rm -r /b/a |
2807 / 2851 | 4367 / 4360 | 2047 / 2060 |
Races on the new head: a host process flipping the leaf between an escaping link, an in-bind link and a file during 40,000 rm -f runs, and swapping a directory for an escaping link during 30,000 rm -rf runs. No escape. The interleaving is uncontrolled, so this is not proof.
CI: every build and test job is green on 29e4008, including Rust (windows-latest) and Rust (macos-latest). The rollup shows failure only because two superseded runs of "2 Approvers for Bots" were cancelled.
Not run
- wasm32: a compile check is blocked on both base and this head because
mlua-sysneeds wasi-sdk, and CI does not build wasm. The wasm-onlyhost_stat_syncnow delegates tohost_meta_stat; I checked that by reading only. - Windows and macOS locally, and the Python and Node bindings. The Windows note from last time (removing a directory symlink with
remove_file) is still untested.
Appendix, non-blocking (2)
- ⚪
rm -rf /b/ld/exits 1 with "Not a directory" after emptyingd; GNUrm -rf ld/does the same work but exits 0.rm -rwithout-fmatches GNU exactly. - ⚪
src/commands/readlink.rs:209still says "Direct-bind metadata follows links", which is no longer true.
On a direct bind, asking whether a path is a symlink always looked through the link to whatever it pointed at. So
test -Land[ -h ]said no for every link, and removal acted on the link's target instead of the link:rm linkdeleted the file it pointed to,rmdir linkdeleted the target directory, andrm -rof a directory holding a link to a sibling directory emptied that sibling and then failed.Now those checks and removals look at the link itself on direct binds, the same way
readlinkalready does. A link is reported as a link and removing it removes only the link, whilelink/andlink/.still mean the directory it points to. The containment and read-only checks are unchanged. A few other things move toward GNU along with this:ls -lshows links as links,findno longer descends into linked directories and-type lmatches them, and removing a file through the Rust, Python, Node and Lua APIs unlinks the link.I added tests for
test -L/-h,rm,rm -randrmdiron host symlinks inside a direct bind, plus read-only binds, links that leave the bind, and paths through a linked directory. On main the symlink tests failed, and I confirmed that the link targets were really deleted. With the fix they pass, and the full test suite passes. I also added a bash comparison case fortest -L/-h. It already matched on the in-memory filesystem, so it's there as a parity guard. I only ran it on Linux. On Windows, removing a symlink to a directory withrmmay still fail, and the new tests are Unix-only.Fixes #254