feat: add filesystem watch support to local_file source by bashandbone · Pull Request #97 · knitli/recoco

bashandbone · 2026-03-13T20:31:36Z

Summary

Port upstream feature from cocoindex-io/cocoindex#1669 to enable real-time change detection for the LocalFile source using the notify crate.

Changes

Add notify 8.2.0 dependency to workspace and recoco-core
Wire notify into source-local-file feature
Add optional watch_changes field to Spec (defaults to false)
Add watch_changes field to Executor
Implement change_stream() method using notify::RecommendedWatcher
Add Clone derive to PatternMatcher for use in async stream
Filter filesystem events through existing PatternMatcher

The feature is opt-in and fully backward-compatible. When enabled, the source bridges filesystem events via tokio::sync::mpsc into the change_stream() interface for low-latency continuous indexing.

Closes #51
Related to #27

Generated with Claude Code

Port upstream feature from cocoindex-io/cocoindex#1669 to enable real-time change detection for the LocalFile source using the notify crate. Changes: - Add notify 8.2.0 dependency to workspace and recoco-core - Wire notify into source-local-file feature - Add optional watch_changes field to Spec (defaults to false) - Add watch_changes field to Executor - Implement change_stream() method using notify::RecommendedWatcher - Add Clone derive to PatternMatcher for use in async stream - Filter filesystem events through existing PatternMatcher The feature is opt-in and fully backward-compatible. When enabled, the source bridges filesystem events via tokio::sync::mpsc into the change_stream() interface for low-latency continuous indexing. Related to #27 Co-authored-by: Adam Poulemanos <bashandbone@users.noreply.github.com>

github-actions · 2026-03-13T20:32:58Z

🤖 Hi @bashandbone, I've received your request, and I'm working on it now! You can track my progress in the logs for more details.

Copilot

Pull request overview

Ports upstream filesystem watch support to the LocalFile source so it can emit real-time change notifications (opt-in via watch_changes) using the notify crate, integrated behind the source-local-file feature.

Changes:

Add notify = 8.2.0 as a workspace dependency and wire it into recoco-core’s source-local-file feature.
Extend LocalFile spec/executor with watch_changes and implement change_stream() using notify::RecommendedWatcher.
Make PatternMatcher clonable so it can be moved into the async change stream.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
crates/recoco-core/src/ops/sources/shared/pattern_matcher.rs	Derive `Clone` for `PatternMatcher` to support async streaming usage.
crates/recoco-core/src/ops/sources/local_file.rs	Add `watch_changes` and implement filesystem-based `change_stream()` for LocalFile.
crates/recoco-core/Cargo.toml	Add optional `notify` dep and include it in `source-local-file` feature.
Cargo.toml	Add `notify` to workspace dependencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

crates/recoco-core/src/ops/sources/local_file.rs

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Adam Poulemanos <89049923+bashandbone@users.noreply.github.com>

cloudflare-workers-and-pages · 2026-03-13T21:05:05Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Updated (UTC)
❌ Deployment failed View logs	recoco-docs	`e836298`	Mar 16 2026, 02:55 AM

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Adam Poulemanos <89049923+bashandbone@users.noreply.github.com>

github-actions · 2026-03-13T21:05:47Z

👋 Hey @bashandbone,

Thanks for your contribution to codeweaver! 🧵

You need to agree to the CLA first... 🖊️

Before we can accept your contribution, you need to agree to our Contributor License Agreement (CLA).

To agree to the CLA, please comment:

I read the contributors license agreement and I agree to it.

Those exact words are important¹, so please don't change them. 😉

You can read the full CLA here: Contributor License Agreement

✅ @bashandbone has signed the CLA.

_{You can retrigger this bot by commenting recheck in this Pull Request.}_{Posted by the CLA Assistant Lite bot.}

Our bot needs those exact words to recognize that you agree to the CLA. ↩

Copilot

Pull request overview

Adds opt-in filesystem watch support to the LocalFile source in recoco-core, enabling real-time change detection via notify when watch_changes is enabled.

Changes:

Add notify as an optional workspace + recoco-core dependency and wire it into the source-local-file feature.
Extend LocalFile Spec/Executor with watch_changes and implement change_stream() using notify::RecommendedWatcher.
Derive Clone for PatternMatcher to support use inside the async change stream.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
Cargo.toml	Adds `notify = 8.2.0` to workspace dependencies.
crates/recoco-core/Cargo.toml	Adds optional `notify` dep and includes it in `source-local-file` feature.
crates/recoco-core/src/ops/sources/local_file.rs	Adds `watch_changes` config and implements `change_stream()` using `notify`.
crates/recoco-core/src/ops/sources/shared/pattern_matcher.rs	Adds `Clone` derive to allow matcher cloning into the stream.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

crates/recoco-core/src/ops/sources/local_file.rs

+        let (tx, mut rx) = mpsc::channel::<PathBuf>(100);
+
+        let mut watcher = RecommendedWatcher::new(
+            move |res: notify::Result<notify::Event>| {
+                if let Ok(event) = res {
+                    for path in event.paths {
+                        if let Err(err) = tx.try_send(path) {
+                            use tokio::sync::mpsc::error::TrySendError;
+                            match err {
+                                TrySendError::Full(_) => {
+                                    warn!("File watcher channel is full; dropping file change event");
+                                }
+                                TrySendError::Closed(_) => {


crates/recoco-core/src/ops/sources/local_file.rs

+                // Filter through pattern matcher
+                if pattern_matcher.is_file_included(relative_path) {
+                    yield Ok(SourceChangeMessage {
+                        changes: vec![SourceChange {
+                            key: KeyValue::from_single_part(relative_path.to_string()),


crates/recoco-core/src/ops/sources/local_file.rs

+                if let Ok(event) = res {
+                    for path in event.paths {
+                        if let Err(err) = tx.try_send(path) {
+                            use tokio::sync::mpsc::error::TrySendError;
+                            match err {
+                                TrySendError::Full(_) => {
+                                    warn!("File watcher channel is full; dropping file change event");
+                                }
+                                TrySendError::Closed(_) => {
+                                    warn!("File watcher channel is closed; dropping file change event");
+                                }
+                            }
+                        }
+                    }


Cargo.toml

 json5 = "1.3.0"
 log = "0.4.29"
 neo4rs = "0.8.0"
+notify = "8.2.0"


bashandbone · 2026-03-13T21:28:13Z

@copilot open a new pull request to apply changes based on the comments in this thread

Copilot · 2026-03-13T21:28:26Z

@bashandbone I've opened a new pull request, #98, to work on those changes. Once the pull request is ready, I'll request review from you.

…mplementation (#98) * Initial plan * fix: address review feedback on change_stream() in local_file source Co-authored-by: bashandbone <89049923+bashandbone@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Adam Poulemanos <89049923+bashandbone@users.noreply.github.com> --------- Signed-off-by: Adam Poulemanos <89049923+bashandbone@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: bashandbone <89049923+bashandbone@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

…310-2131

socket-security · 2026-03-16T02:54:54Z

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	cargo/notify@8.2.0

View full report

bashandbone added enhancement New feature or request upstream-sync Issues for syncing updates with our upstream (cocoindex-io/cocoindex) labels Mar 13, 2026

Copilot AI review requested due to automatic review settings March 13, 2026 20:31

github-project-automation bot added this to Recoco v1.0.0 Mar 13, 2026

github-project-automation bot moved this to Backlog in Recoco v1.0.0 Mar 13, 2026

bashandbone removed this from Recoco v1.0.0 Mar 13, 2026

Copilot started reviewing on behalf of bashandbone March 13, 2026 20:34 View session

Copilot AI reviewed Mar 13, 2026

View reviewed changes

crates/recoco-core/src/ops/sources/local_file.rs Show resolved Hide resolved

crates/recoco-core/src/ops/sources/local_file.rs Outdated Show resolved Hide resolved

Potential fix for pull request finding

a49e9ae

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Adam Poulemanos <89049923+bashandbone@users.noreply.github.com>

Potential fix for pull request finding

132ea49

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Adam Poulemanos <89049923+bashandbone@users.noreply.github.com>

bashandbone requested a review from Copilot March 13, 2026 21:05

Copilot started reviewing on behalf of bashandbone March 13, 2026 21:08 View session

Copilot AI reviewed Mar 13, 2026

View reviewed changes

Copilot AI mentioned this pull request Mar 13, 2026

fix(local_file): address review feedback on change_stream() watcher implementation #98

Merged

Copilot AI and others added 3 commits March 15, 2026 22:39

Merge remote-tracking branch 'origin/main' into claude/issue-51-20260…

b4d81f9

…310-2131

update lockfile

e836298

bashandbone merged commit 2f82c4d into main Mar 16, 2026
8 of 10 checks passed

bashandbone deleted the claude/issue-51-20260310-2131 branch March 16, 2026 02:59

github-actions bot mentioned this pull request Mar 16, 2026

[upstream-sync] LocalFile source: ignore errors and warn in list() for resilient directory listing (upstream PR #1753) #102

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add filesystem watch support to local_file source#97

feat: add filesystem watch support to local_file source#97
bashandbone merged 6 commits intomainfrom
claude/issue-51-20260310-2131

bashandbone commented Mar 13, 2026

Uh oh!

github-actions bot commented Mar 13, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

cloudflare-workers-and-pages bot commented Mar 13, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 13, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

bashandbone commented Mar 13, 2026

Uh oh!

Copilot AI commented Mar 13, 2026

Uh oh!

socket-security bot commented Mar 16, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

bashandbone commented Mar 13, 2026

Summary

Changes

Uh oh!

github-actions bot commented Mar 13, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

cloudflare-workers-and-pages bot commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying with Cloudflare Workers

Uh oh!

github-actions bot commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Thanks for your contribution to codeweaver! 🧵

You need to agree to the CLA first... 🖊️

To agree to the CLA, please comment:

Footnotes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

bashandbone commented Mar 13, 2026

Uh oh!

Copilot AI commented Mar 13, 2026

Uh oh!

socket-security bot commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cloudflare-workers-and-pages bot commented Mar 13, 2026 •

edited

Loading

github-actions bot commented Mar 13, 2026 •

edited

Loading

socket-security bot commented Mar 16, 2026 •

edited

Loading