Skip to content

Fix rewatch losing watches when an overlapping source folder is removed - #8839

Open
cknitt wants to merge 1 commit into
masterfrom
rewatch-overlapping-unwatch
Open

cknitt wants to merge 1 commit into
masterfrom
rewatch-overlapping-unwatch

Conversation

@cknitt

@cknitt cknitt commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

rescript watch registers each source folder as its own watch, and source folders may overlap. In Rust rewatch (rescript-rust), a full rebuild (after a rescript.json change, but also when a source file is added, removed or renamed) unwatches the paths that went away and, since #8730, registers only the new ones. With inotify on Linux that loses watches: notify keeps one watch per directory, shared by every registered path that reaches it, so unwatching one path can drop another that is still configured.

For example, with a package's sources changed from [{"dir": "src", "subdirs": true}, "src/retained"] to ["src", "src/retained"] while the watcher runs, unwatching the recursive src also dropped src/retained, and edits there were no longer compiled (#8825). The same can happen through other spellings of a folder (.., symlinks) and through directory symlinks inside a recursive folder.

update_watches still unwatches only the stale watches first (paths no longer configured, or whose directory was replaced). After that it depends on the platform:

  • macOS (FSEvents): only the new paths are registered, as before. FSEvents restarts its event stream on every watch/unwatch call and loses the events in between, which is why Fix watcher missing watch.lock removal during full rebuilds #8730 made updates incremental. Its watches are independent paths, so an unwatch never drops another one.
  • Linux (inotify) and other platforms: every configured path is registered again. Registering a directory that is already watched only updates its existing watch, so unchanged watches keep reporting events without a gap, and any watch an unwatch dropped is restored. This needs no reasoning about how watches overlap.
  • Windows takes the same path. It has no stable directory identity, so every watch counts as stale there and is re-registered, as before.

The doc comment of update_watches explains these differences.

Watches are also registered non-recursive first, on every platform. notify keeps one recursive flag per directory, set by the last registration that reaches it, and only watches a folder created later if its parent's flag is recursive. Registered in HashMap order, a non-recursive src/retained could clear the flag that the recursive src set, so files in folders created inside src/retained were missed. That already happened on master's first registration; with every path registered again, it could also happen on each full rebuild.

One window remains: after a config change that removes a recursive folder overlapping a kept one, the kept folder's watch is dropped by the unwatch and restored when every path is registered again, so an edit in between is picked up only with the next change. Ordinary full rebuilds unwatch nothing. Closing it would need a rescan after registration, which isn't worth it for the Rust fallback.

OCaml rewatch (rescript), which ships as the default since #8780, isn't affected: it resolves every watched folder to its canonical directory, keeps one libuv handle per directory, and on a refresh closes only the directories that are no longer wanted.

New test: rewatch/tests/watch/08-watch-overlapping-source-removed.sh in the shared suite runs against both executables. With the overlapping configuration, it checks that a file in a folder created inside src/retained is compiled. It then reproduces the example above through the CLI and checks that an edit in src/retained is compiled after the change. With master, rescript-rust fails the edit check (2 of 2 runs); without the registration order, the new-folder check failed 7 of 8 runs. rescript passes both.

Fixes #8825

🤖 Generated with Claude Code

https://claude.ai/code/session_01BC5F5QPeQh5cHBjGBnvJ2T

@cknitt
cknitt marked this pull request as ready for review October 12, 2026 19:21
@cknitt
cknitt force-pushed the rewatch-overlapping-unwatch branch from 010c1ae to f456adc Compare October 12, 2026 19:21
@pkg-pr-new

pkg-pr-new Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

rescript

npm i https://pkg.pr.new/rescript@8839

@rescript/belt

npm i https://pkg.pr.new/@rescript/belt@8839

@rescript/darwin-arm64

npm i https://pkg.pr.new/@rescript/darwin-arm64@8839

@rescript/darwin-x64

npm i https://pkg.pr.new/@rescript/darwin-x64@8839

@rescript/linux-arm64

npm i https://pkg.pr.new/@rescript/linux-arm64@8839

@rescript/linux-x64

npm i https://pkg.pr.new/@rescript/linux-x64@8839

@rescript/runtime

npm i https://pkg.pr.new/@rescript/runtime@8839

@rescript/win32-x64

npm i https://pkg.pr.new/@rescript/win32-x64@8839

commit: d3591fd

@github-actions

Copy link
Copy Markdown

@cknitt

cknitt commented Oct 13, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-16T19:32:21.124364Z d3591fd Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f456adc3e9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread changes/rewatch-overlapping-unwatch.fix.md
Comment thread rewatch/src/watcher.rs Outdated
@cknitt

cknitt commented Oct 13, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 94a90ef802

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rewatch/src/watcher.rs Outdated
Comment thread rewatch/src/watcher.rs Outdated
@cknitt

cknitt commented Oct 13, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a1c8248246

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rewatch/src/watcher.rs Outdated
@cknitt

cknitt commented Oct 13, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cd0c132ece

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rewatch/src/watcher.rs Outdated
@cknitt

cknitt commented Oct 13, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ccacd4b908

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rewatch/src/watcher.rs Outdated
@cknitt

cknitt commented Oct 13, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: 765184e889

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@cknitt cknitt changed the title Rewatch: re-register watches dropped by an overlapping unwatch Re-register all watches outside macOS Oct 13, 2026
@cknitt
cknitt force-pushed the rewatch-overlapping-unwatch branch from 765184e to a5af682 Compare October 13, 2026 11:43
@cknitt

cknitt commented Oct 13, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a5af682777

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rewatch/src/watcher.rs Outdated
@cknitt
cknitt force-pushed the rewatch-overlapping-unwatch branch from a5af682 to e3ada4d Compare October 15, 2026 07:33
@cknitt cknitt changed the title Re-register all watches outside macOS Fix rewatch losing watches when an overlapping source folder is removed Oct 15, 2026
@cknitt

cknitt commented Oct 15, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e3ada4d2cb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rewatch/src/watcher.rs
Comment thread rewatch/src/watcher.rs
cknitt added a commit that referenced this pull request Oct 16, 2026
Rust rewatch (`rescript-rust`) watches each source folder separately, and
source folders may overlap. On a full rebuild it unwatched the paths that
went away and registered only the new ones. With inotify on Linux, notify
keeps one watch per directory, shared by every registered path that reaches
it, so unwatching a removed recursive folder also dropped a folder that is
still configured, and edits there were no longer compiled. Outside macOS,
every configured path is now registered again after the stale ones are
unwatched; registering an already watched directory only updates its watch,
so unchanged watches have no gap. macOS keeps registering only the new
paths, because FSEvents restarts its event stream on every watch change and
its watches don't depend on each other.

Example (text):
    rescript.json of a package, while `rescript-rust watch` runs:
      "sources": [{"dir": "src", "subdirs": true}, "src/retained"]
    changed to
      "sources": ["src", "src/retained"]
    then src/retained/Retained.res is edited.

Before (text):
    The edit isn't compiled (Linux).

After (text):
    The edit is compiled.

Source: rewatch/tests/watch/08-watch-overlapping-source-removed.sh, run
against master's rescript-rust (fails 2 of 2) and this branch's (passes).

Changelog entry: - Rewatch: keep watching a source folder on Linux when an overlapping recursive source folder is removed from `rescript.json`. #8839

Review: Is OCaml rewatch (`rescript`) affected? No. It resolves every
watched folder to its canonical directory, keeps one libuv handle per
directory, and on a refresh closes only the directories no longer wanted.
The new test runs against both executables in the shared suite and passes
for `rescript` on master already.

Review: Why not keep the incremental update everywhere? Watches can share
inotify state through overlapping folders, `..` and symlinks, including
directory symlinks inside a recursive folder; working out every way an
unwatch can drop another watch took several review rounds. Registering every
path again needs no such reasoning and costs no gap on inotify.

Review: Why does macOS keep the incremental update? #8730 introduced it
because FSEvents restarts its event stream on every watch or unwatch call
and loses the events in between. FSEvents watches are independent paths, so
an unwatch never drops another watch there.

Area: rewatch tests
Changelog: fix
Output: unchanged
Tests: +rewatch/tests/watch/08-watch-overlapping-source-removed.sh
 ~rewatch/tests/suite.sh
PR: #8839
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BC5F5QPeQh5cHBjGBnvJ2T
Signed-off-by: Christoph Knittel <ck@cca.io>
@cknitt
cknitt force-pushed the rewatch-overlapping-unwatch branch from e3ada4d to e24b200 Compare October 16, 2026 16:33
@cknitt

cknitt commented Oct 16, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: e24b2007a3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@cknitt
cknitt requested a review from cristianoc October 16, 2026 19:09
Rust rewatch (`rescript-rust`) watches each source folder separately, and
source folders may overlap. On a full rebuild it unwatched the paths that
went away and registered only the new ones. With inotify on Linux, notify
keeps one watch per directory, shared by every registered path that reaches
it, so unwatching a removed recursive folder also dropped a folder that is
still configured, and edits there were no longer compiled. Outside macOS,
every configured path is now registered again after the stale ones are
unwatched; registering an already watched directory only updates its watch,
so unchanged watches have no gap. macOS keeps registering only the new
paths, because FSEvents restarts its event stream on every watch change and
its watches don't depend on each other.

Watches are also registered non-recursive first. notify keeps one recursive
flag per directory, set by the last registration that reaches it, so a
non-recursive source folder inside a recursive one could clear the flag, and
folders created there later weren't watched.

Example (text):
    rescript.json of a package, while `rescript-rust watch` runs:
      "sources": [{"dir": "src", "subdirs": true}, "src/retained"]
    changed to
      "sources": ["src", "src/retained"]
    then src/retained/Retained.res is edited. Before the change, a file is
    also created in a new folder src/retained/nested.

Before (text):
    The edit isn't compiled (Linux). The new folder's file is missed in most
    runs.

After (text):
    Both are compiled.

Source: rewatch/tests/watch/08-watch-overlapping-source-removed.sh, run
against master's rescript-rust (the edit fails 2 of 2), this branch without
the registration order (the new folder fails 7 of 8) and this branch (passes
8 of 8).

Changelog entry: - Rewatch: keep watching a source folder on Linux when an overlapping recursive source folder is removed from `rescript.json`. #8839

Review: Is OCaml rewatch (`rescript`) affected? No. It resolves every
watched folder to its canonical directory, keeps one libuv handle per
directory, and on a refresh closes only the directories no longer wanted.
The new test runs against both executables in the shared suite and passes
for `rescript` on master already.

Review: Why not keep the incremental update everywhere? Watches can share
inotify state through overlapping folders, `..` and symlinks, including
directory symlinks inside a recursive folder; working out every way an
unwatch can drop another watch took several review rounds. Registering every
path again needs no such reasoning and costs no gap on inotify.

Review: Can an edit be missed while the watches are updated? Only after a
config change that removes a recursive folder overlapping a kept one: the
kept folder's watch is dropped by the unwatch and restored when every path
is registered again, so an edit in between is picked up only with the next
change. Ordinary full rebuilds unwatch nothing. Closing that window would
need a rescan after registration, which isn't worth it for the Rust
fallback.

Review: Why does macOS keep the incremental update? #8730 introduced it
because FSEvents restarts its event stream on every watch or unwatch call
and loses the events in between. FSEvents watches are independent paths, so
an unwatch never drops another watch there.

Area: rewatch tests
Changelog: fix
Output: unchanged
Tests: +rewatch/tests/watch/08-watch-overlapping-source-removed.sh
 ~rewatch/tests/suite.sh
PR: #8839
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BC5F5QPeQh5cHBjGBnvJ2T
Signed-off-by: Christoph Knittel <ck@cca.io>
@cknitt
cknitt force-pushed the rewatch-overlapping-unwatch branch from e24b200 to d3591fd Compare October 16, 2026 19:29
@github-actions

Copy link
Copy Markdown

@cknitt

cknitt commented Oct 16, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: d3591fd089

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rewatch watch on Linux stops watching a source folder when an overlapping recursive source folder is removed from rescript.json

1 participant