feat: Use BPF iterator instead of procfs#336
feat: Use BPF iterator instead of procfs#336vadorovsky wants to merge 3 commits intofix-race-conditionfrom
Conversation
4d6e4b4 to
acf7493
Compare
8dad79c to
87966a5
Compare
4e8a25d to
cdf890c
Compare
4933373 to
949f7b1
Compare
There was a problem hiding this comment.
Pull Request Overview
This PR replaces the process tracker initialization from procfs parsing to a BPF iterator approach to improve reliability and simplify the implementation.
- Introduces a new enum ContainerEngineKind in pulsar-core to represent container engines.
- Removes the duplicate ContainerEngineKind from process-monitor and updates the process tree loading logic to use BPF iterator output.
- Updates the initializer to use the new process tree loader and changes the aya dependency configuration in Cargo.toml.
Reviewed Changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| crates/pulsar-core/src/event.rs | Added ContainerEngineKind enum for container engine distinction. |
| crates/modules/process-monitor/src/lib.rs | Updated import statements by removing duplicate enum definition. |
| crates/bpf-filtering/src/process_tree.rs | Replaced procfs parsing with reading from a BPF iterator and updated error handling for incorrect output formats. |
| crates/bpf-filtering/src/initializer.rs | Updated process tree initialization to use the new BPF iterator loader. |
| Cargo.toml | Updated aya dependency to a specific git revision. |
Files not reviewed (1)
- crates/modules/process-monitor/probes.bpf.c: Language not supported
949f7b1 to
32b28ef
Compare
595d168 to
ad0ed39
Compare
c1915f0 to
679c994
Compare
|
On the start I always see [2025-05-20T11:05:59Z WARN bpf_filtering::initializer] process 0 not found while building map_interest
[2025-05-20T11:05:59Z WARN bpf_filtering::initializer] process 0 not found while building map_interestand sometimes [2025-05-20T11:07:19Z ERROR file-system-monitor] Process not found in tracker 63980: process not found
[2025-05-20T11:07:19Z ERROR file-system-monitor] Process not found in tracker 63980: process not foundthe goal of BPF iterators should be to solve the |
Hmm... this is because we use PID 0 as a ppid for PID 1. Good catch, that doesn't make sense. We should define ppid as
Interesting. Haven't seen these on my end, but will try to reproduce.
|
|
@banditopazzo Are you seeing these errors after, let's say, the first 15s of Pulsar's runtime? In my case, I see them only at the very beginning and then never again. And after dumping the process tree to a file with the following diff diff --git a/crates/bpf-filtering/src/process_tree.rs b/crates/bpf-filtering/src/process_tree.rs
index 014fcf6..c7622fa 100644
--- a/crates/bpf-filtering/src/process_tree.rs
+++ b/crates/bpf-filtering/src/process_tree.rs
@@ -171,6 +171,8 @@ impl ProcessTree {
ProcessData::from_str(&line)
})
.collect::<Result<Vec<ProcessData>, Error>>()?;
+ std::fs::write("/tmp/processes", format!("{processes:#?}")).unwrap();
+ log::error!("CONSUMED BPF ITERATOR");
Ok(Self { processes })
}I see that all the PIDs mentioned in the errors, like: are present in the map created from the iterator: What's funny, is that it's almost always the pulsar process generating this error. After adding some more logs I'm 99% sure that the issue is a race condition with other modules. It seems like file-system-monitor (and all other probes) are starting and generating events before the process tree and process tracker are initialized. I will keep thinking about the solution, but what I would try for now is blocking all the other modules from starting and loading BPF programs before the process tracker is initialized. |
Allow modules to be dependent on others by adding the `DEPENDS_ON` constant to `PulsarModule` and `SimplePulsarModule` traits. This constant can be set either as empty (`&[]`) or can specify dependencies (e.g. `&["process-monitor"]`, `&["process-monitor", "network-monitor"]`). Make `file-system-monitor` and `network-monitor` dependent on `process-monitor`. This way we get rid of the race condition, where these modules start before `process-monitor`, generating errors like: ``` Process not found in tracker 63980: process not found ``` These errors are not there anymore after introducing the dependency.
|
@banditopazzo My assumption was right, all the errors you mentioned were a race condition between modules. I fixed that in #347 and rebased this PR on top of it. Can you check again if the issue occurs to you? |
Replace the process tracker initialization process from procfs parsing to BPF iterators, which are less erronous and result in an easier implementation.
9b9a1c7 to
7aef8fe
Compare
Using a `Vec` is not efficient, because lookups for the given process are done with O(n) complexity. Changing it to `HashMap` makes the lookup an O(1) operation.
Replace the process tracker initialization process from procfs parsing to BPF iterators, which are less erronous and result in an easier implementation.