Skip to content

Workaround Fuse-T readdir behavior on macOS#530

Open
LoganDark wants to merge 6 commits intogittup:masterfrom
LoganDark:fuse-nfs-workaround
Open

Workaround Fuse-T readdir behavior on macOS#530
LoganDark wants to merge 6 commits intogittup:masterfrom
LoganDark:fuse-nfs-workaround

Conversation

@LoganDark
Copy link
Copy Markdown

With Fuse-T, readdir seems to also stat every directory entry, causing tup to assume false input dependencies; potentially due to Fuse-T's usage of NFS. Due to this, while bootstrapping succeeds, Fuse-T build/tup will report errors like this on my machine:

* 43) src/luabuiltin: ../lua/lua xxd.lua builtin.lua luabuiltin.h
 *** tup messages ***
tup error: Missing input dependency - a file was read from, and was not specified as an input link for the command. This is an issue because the file was created from another command, and without the input link the commands may execute out of order. You should add this file as an input, since it is possible this could randomly break in the future.
 - [2417] src/lua/.gitignore
 *** Command ran successfully, but failed due to errors processing input dependencies.
 42) [5.027s] src/sqlite3: CC sqlite3.c
 [                             ETA~=5s  Remaining=42 Active=0                               ]  53%
 *** tup: 1 job failed.
(full transcript)
Initializing .tup in /Users/LoganDark/Documents/Projects/tup
.tup repository initialized: .tup/db
[ tup ] [0.000s] Scanning filesystem...
[ tup ] [0.020s] Reading in new environment variables...
[ tup ] [0.021s] Parsing Tupfiles...
 58) [0.006s] src/lua
 57) [0.005s] src/luabuiltin
 56) [0.007s] src/tup
 55) [0.005s] src/tup/flock
 54) [0.005s] src/tup/tup
 53) [0.005s] src/tup/monitor
 52) [0.004s] src/tup/server
 51) [0.004s] src/inih
 50) [0.003s] src/compat
 49) [0.003s] src/sqlite3
 48) [0.006s] .
 47) [0.002s] test
 46) [0.001s] test/make_v_tup
 45) [0.001s] docs
 44) [0.002s] docs/html
 43) [0.003s] docs/html/pub
 42) [0.002s] docs/html/pub/win32
 41) [0.001s] contrib
 40) [0.001s] contrib/debian
 39) [0.001s] contrib/debian/source
 38) [0.001s] contrib/syntax
 37) [0.001s] libfuse
 36) [0.001s] libfuse/example
 35) [0.001s] libfuse/include
 34) [0.001s] libfuse/include/old
 33) [0.001s] libfuse/lib
 32) [0.001s] libfuse/lib/modules
 31) [0.001s] libfuse/doc
 30) [0.001s] .jj
 29) [0.001s] .jj/working_copy
 28) [0.001s] .jj/repo
 27) [0.001s] .jj/repo/op_store
 26) [0.002s] .jj/repo/op_store/operations
 25) [0.002s] .jj/repo/op_store/views
 24) [0.002s] .jj/repo/workspace_store
 23) [0.002s] .jj/repo/op_heads
 22) [0.002s] .jj/repo/op_heads/heads
 21) [0.001s] .jj/repo/index
 20) [0.002s] .jj/repo/index/op_links
 19) [0.002s] .jj/repo/index/changed_paths
 18) [0.002s] .jj/repo/index/segments
 17) [0.001s] .jj/repo/submodule_store
 16) [0.001s] .jj/repo/store
 15) [0.002s] .jj/repo/store/extra
 14) [0.002s] .jj/repo/store/extra/heads
 13) [0.001s] .github
 12) [0.001s] .github/workflows
 11) [0.003s] lib32
 10) [0.001s] local-build-configs
  9) [0.001s] build
  8) [0.001s] build/luabuiltin
  7) [0.001s] src
  6) [0.003s] src/compat/win32
  5) [0.004s] src/compat/win32/detect
  4) [0.002s] src/compat/win32/sys
  3) [0.003s] src/dllinject
  2) [0.001s] src/bsd
  1) [0.003s] src/pcre
  0) [0.003s] src/ldpreload
 [                  ETA~=<1s Remaining=0                     ] 100%
[ tup ] [0.195s] No files to delete.
[ tup ] [0.195s] Generating .gitignore files...
[ tup ] [0.199s] Executing Commands...
 89) [0.124s] src/lua: CC lctype.c
 88) [0.151s] src/lua: CC linit.c
 87) [0.155s] src/lua: CC ldblib.c
 86) [0.163s] src/lua: CC lcorolib.c
 85) [0.180s] src/lua: CC ldump.c
 84) [0.192s] src/lua: CC lfunc.c
 83) [0.202s] src/lua: CC lmathlib.c
 82) [0.211s] src/lua: CC liolib.c
 81) [0.212s] src/lua: CC lauxlib.c
 80) [0.214s] src/lua: CC lbaselib.c
 79) [0.240s] src/lua: CC llex.c
 78) [0.245s] src/lua: CC lapi.c
 77) [0.246s] src/lua: CC ldo.c
 76) [0.261s] src/lua: CC ldebug.c
 75) [0.276s] src/lua: CC lcode.c
 74) [0.120s] src/lua: CC lopcodes.c
 73) [0.295s] src/lua: CC lgc.c
 72) [0.184s] src/lua: CC lmem.c
 71) [0.163s] src/lua: CC loadlib.c
 70) [0.028s] CP src/tup/vardict.h -> tup_client.h
 69) [0.171s] src/lua: CC loslib.c
 68) [0.223s] src/lua: CC lobject.c
 67) [0.144s] src/lua: CC ltablib.c
 66) [0.138s] src/lua: CC lutf8lib.c
 65) [0.194s] src/lua: CC lstring.c
 64) [0.110s] src/tup/flock: CC fcntl.c
 63) [0.225s] src/lua: CC lstate.c
 62) [0.190s] src/lua: CC ltm.c
 61) [0.153s] src/lua: CC lzio.c
 60) [0.149s] src/lua: CC lua.c
 59) [0.233s] src/lua: CC ltable.c
 58) [0.096s] src/tup/monitor: CC null.c
 57) [0.203s] src/lua: CC lundump.c
 56) [0.238s] src/lua: CC lstrlib.c
 55) [0.080s] src/compat: CC dummy.c
 54) [0.102s] src/inih: CC ini.c
 53) [0.089s] src/compat: CC clearenv.c
 52) [0.339s] src/lua: CC lparser.c
 51) [0.162s] src/tup/server: CC symlink.c
 50) [0.201s] src/tup/server: CC master_fork.c
 49) [0.227s] src/tup/server: CC fuse_server.c
 48) [0.222s] src/tup/server: CC fuse_fs.c
 47) [0.271s] src/tup/tup: CC main.c
 46) [0.351s] src/lua: CC lvm.c
 45) [0.072s] src/lua: AR liblua.a
 44) [0.081s] src/lua: LINK lua
* 43) src/luabuiltin: ../lua/lua xxd.lua builtin.lua luabuiltin.h
 *** tup messages ***
tup error: Missing input dependency - a file was read from, and was not specified as an input link for the command. This is an issue because the file was created from another command, and without the input link the commands may execute out of order. You should add this file as an input, since it is possible this could randomly break in the future.
 - [2417] src/lua/.gitignore
 *** Command ran successfully, but failed due to errors processing input dependencies.
 42) [5.027s] src/sqlite3: CC sqlite3.c
 [                             ETA~=5s  Remaining=42 Active=0                               ]  53%
 *** tup: 1 job failed.

With this PR, on macOS when Fuse-T is likely in use, we now test at startup for this behavior by listing a virtual directory and watching for stat of a virtual sentinel file. We always ensure the behavior at runtime before enabling the workaround, because there could be forks of Fuse-T that fix it or other implementations that never had it (such as macFUSE). The virtual directory itself deactivates after being listed once. Then, if we observe a stat on the sentinel, we assume it was performed automatically as a consequence of the readdir, and enable the workaround. We then stat a virtual "done" file to conclude the probe, deactivating both virtual files to prevent the workaround from being enabled when the behavior is not present. Thus the workaround is only enabled when a stat reaches the sentinel following the readdir but before our stat to the done file.

When the workaround is enabled, readdir will record each result into a cache that tells getattr to ignore one next stat. By ignoring the spurious call to getattr, we avoid assuming a corresponding spurious access, because in this case, the first stat for a child following its parent directory's listing is from the filesystem layer itself, not the running task.

This is my first contribution, so let me know if I'll need to sign a CLA.

@LoganDark LoganDark force-pushed the fuse-nfs-workaround branch 2 times, most recently from d25849d to b3d1e47 Compare April 20, 2026 19:00
@LoganDark LoganDark force-pushed the fuse-nfs-workaround branch from b3d1e47 to 1e2b342 Compare April 20, 2026 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant