Make gazelle consider only BUILD.bazel files#48455
Make gazelle consider only BUILD.bazel files#48455gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit intomainfrom
gazelle consider only BUILD.bazel files#48455Conversation
### What does this PR do? Configure the `//:gazelle` target with `-build_file_name=BUILD.bazel`. ### Motivation When `gazelle` runs inside a Linux container on macOS via Docker Desktop, the workspace is typically mounted on a case-insensitive filesystem where `BUILD` and `build` are indistinguishable. This causes spurious errors such as: ```sh bazel run //:gazelle; echo $? ... gazelle: open /dda/.gitlab/windows/build: no such file or directory open /dda/rtloader/build: no such file or directory 1 ``` Whereas the cause strictly resides in a filesystem layer/driver (docker/desktop-feedback#251, recently imported from docker/for-mac#7218), portable tools have been taking countermeasures. Bazel 9, for instance, revamped their filesystem cache to better accomodate case-insensitive filesystems (bazelbuild/bazel#26842), which led us to switch to it (#47742), but that only covers Bazel's own I/O, not the processes it spawns, of which `gazelle`. For that purpose, Gazelle happens to explicitly recommend setting its `build_file_name` directive to `BUILD.bazel`: > If a repository contains files named `build` that aren't related to Bazel, it may help to set this to `"BUILD.bazel"`, especially on case-insensitive file systems. ### Additional Notes For consistency with the narrowed scope, this change renames `compliance/rules/BUILD` to `compliance/rules/BUILD.bazel` (the only plain `BUILD` file still present in the tree).
Files inventory check summaryFile checks results against ancestor 93f15781: Results for datadog-agent_7.79.0~devel.git.213.fb21d42.pipeline.104629755-1_amd64.deb:No change detected |
Regression DetectorRegression Detector ResultsMetrics dashboard Baseline: 867176f Optimization Goals: ✅ No significant changes detected
|
| perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
|---|---|---|---|---|---|---|
| ➖ | docker_containers_cpu | % cpu utilization | -1.33 | [-4.31, +1.65] | 1 | Logs |
Fine details of change detection per experiment
| perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
|---|---|---|---|---|---|---|
| ➖ | quality_gate_metrics_logs | memory utilization | +1.93 | [+1.69, +2.17] | 1 | Logs bounds checks dashboard |
| ➖ | uds_dogstatsd_20mb_12k_contexts_20_senders | memory utilization | +0.57 | [+0.51, +0.62] | 1 | Logs |
| ➖ | ddot_metrics | memory utilization | +0.56 | [+0.38, +0.74] | 1 | Logs |
| ➖ | quality_gate_idle | memory utilization | +0.24 | [+0.18, +0.29] | 1 | Logs bounds checks dashboard |
| ➖ | otlp_ingest_metrics | memory utilization | +0.21 | [+0.05, +0.37] | 1 | Logs |
| ➖ | file_tree | memory utilization | +0.16 | [+0.11, +0.21] | 1 | Logs |
| ➖ | file_to_blackhole_500ms_latency | egress throughput | +0.05 | [-0.35, +0.44] | 1 | Logs |
| ➖ | tcp_dd_logs_filter_exclude | ingress throughput | +0.01 | [-0.10, +0.12] | 1 | Logs |
| ➖ | uds_dogstatsd_to_api | ingress throughput | +0.01 | [-0.18, +0.20] | 1 | Logs |
| ➖ | uds_dogstatsd_to_api_v3 | ingress throughput | -0.01 | [-0.20, +0.19] | 1 | Logs |
| ➖ | tcp_syslog_to_blackhole | ingress throughput | -0.01 | [-0.17, +0.16] | 1 | Logs |
| ➖ | ddot_metrics_sum_cumulativetodelta_exporter | memory utilization | -0.02 | [-0.24, +0.21] | 1 | Logs |
| ➖ | quality_gate_idle_all_features | memory utilization | -0.02 | [-0.06, +0.01] | 1 | Logs bounds checks dashboard |
| ➖ | file_to_blackhole_100ms_latency | egress throughput | -0.03 | [-0.12, +0.06] | 1 | Logs |
| ➖ | file_to_blackhole_1000ms_latency | egress throughput | -0.04 | [-0.47, +0.38] | 1 | Logs |
| ➖ | file_to_blackhole_0ms_latency | egress throughput | -0.06 | [-0.56, +0.44] | 1 | Logs |
| ➖ | ddot_logs | memory utilization | -0.09 | [-0.15, -0.02] | 1 | Logs |
| ➖ | ddot_metrics_sum_cumulative | memory utilization | -0.11 | [-0.25, +0.03] | 1 | Logs |
| ➖ | otlp_ingest_logs | memory utilization | -0.20 | [-0.30, -0.10] | 1 | Logs |
| ➖ | quality_gate_logs | % cpu utilization | -0.25 | [-1.91, +1.40] | 1 | Logs bounds checks dashboard |
| ➖ | ddot_metrics_sum_delta | memory utilization | -0.30 | [-0.46, -0.13] | 1 | Logs |
| ➖ | docker_containers_cpu | % cpu utilization | -1.33 | [-4.31, +1.65] | 1 | Logs |
| ➖ | docker_containers_memory | memory utilization | -2.16 | [-2.33, -1.99] | 1 | Logs |
Bounds Checks: ✅ Passed
| perf | experiment | bounds_check_name | replicates_passed | observed_value | links |
|---|---|---|---|---|---|
| ✅ | docker_containers_cpu | simple_check_run | 10/10 | 708 ≥ 26 | |
| ✅ | docker_containers_memory | memory_usage | 10/10 | 275.42MiB ≤ 370MiB | |
| ✅ | docker_containers_memory | simple_check_run | 10/10 | 682 ≥ 26 | |
| ✅ | file_to_blackhole_0ms_latency | memory_usage | 10/10 | 0.19GiB ≤ 1.20GiB | |
| ✅ | file_to_blackhole_0ms_latency | missed_bytes | 10/10 | 0B = 0B | |
| ✅ | file_to_blackhole_1000ms_latency | memory_usage | 10/10 | 0.23GiB ≤ 1.20GiB | |
| ✅ | file_to_blackhole_1000ms_latency | missed_bytes | 10/10 | 0B = 0B | |
| ✅ | file_to_blackhole_100ms_latency | memory_usage | 10/10 | 0.19GiB ≤ 1.20GiB | |
| ✅ | file_to_blackhole_100ms_latency | missed_bytes | 10/10 | 0B = 0B | |
| ✅ | file_to_blackhole_500ms_latency | memory_usage | 10/10 | 0.21GiB ≤ 1.20GiB | |
| ✅ | file_to_blackhole_500ms_latency | missed_bytes | 10/10 | 0B = 0B | |
| ✅ | quality_gate_idle | intake_connections | 10/10 | 3 = 3 | bounds checks dashboard |
| ✅ | quality_gate_idle | memory_usage | 10/10 | 174.39MiB ≤ 175MiB | bounds checks dashboard |
| ✅ | quality_gate_idle_all_features | intake_connections | 10/10 | 3 = 3 | bounds checks dashboard |
| ✅ | quality_gate_idle_all_features | memory_usage | 10/10 | 493.23MiB ≤ 550MiB | bounds checks dashboard |
| ✅ | quality_gate_logs | intake_connections | 10/10 | 4 ≤ 6 | bounds checks dashboard |
| ✅ | quality_gate_logs | memory_usage | 10/10 | 205.36MiB ≤ 220MiB | bounds checks dashboard |
| ✅ | quality_gate_logs | missed_bytes | 10/10 | 0B = 0B | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | cpu_usage | 10/10 | 351.73 ≤ 2000 | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | intake_connections | 10/10 | 4 ≤ 6 | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | memory_usage | 10/10 | 410.58MiB ≤ 475MiB | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | missed_bytes | 10/10 | 0B = 0B | bounds checks dashboard |
Explanation
Confidence level: 90.00%
Effect size tolerance: |Δ mean %| ≥ 5.00%
Performance changes are noted in the perf column of each table:
- ✅ = significantly better comparison variant performance
- ❌ = significantly worse comparison variant performance
- ➖ = no significant change in performance
A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".
For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:
-
Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
-
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
-
Its configuration does not mark it "erratic".
CI Pass/Fail Decision
✅ Passed. All Quality Gates passed.
- quality_gate_metrics_logs, bounds check cpu_usage: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check missed_bytes: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_idle, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_idle, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_idle_all_features, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_idle_all_features, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check missed_bytes: 10/10 replicas passed. Gate passed.
Static quality checks✅ Please find below the results from static quality gates 30 successful checks with minimal change (< 2 KiB)
On-wire sizes (compressed)
|
### What does this PR do? Add `**/build` to `REPO.bazel`'s `ignore_directories()`, replacing `.bazelignore`, and adjust `CODEOWNERS` accordingly. ### Motivation When `bazel` runs inside a Linux container on macOS via Docker Desktop, the workspace is typically mounted on a case-insensitive filesystem where `BUILD` and `build` are indistinguishable (docker/desktop-feedback#251). #48455 addressed this for `gazelle` by restricting it to `BUILD.bazel` files only. Bazel's own directory traversal, however, remained exposed to the same confusion. `REPO.bazel` gained `ignore_directories()` in Bazel **8.0*** (bazelbuild/bazel#24203), introduced as a complement to `.bazelignore` > to provide a migration path off of that weird single-purpose configuration file. It supports Bazel's native `glob()` semantics, making `**/build` a natural fit. ### Additional Notes Coming from Bazel 7, I was not aware of `REPO.bazel`, otherwise I would have added it in the first place instead of `.bazelignore` (#40153). In `CODEOWNERS`, `/MODULE.bazel*` is replaced by `/*.bazel*` to cover `REPO.bazel` as well as any future root-level `.bazel` files.
### What does this PR do? Add `**/build` to `REPO.bazel`'s `ignore_directories()`, replacing `.bazelignore`, and adjust `CODEOWNERS` accordingly. ### Motivation When `bazel` runs inside a Linux container on macOS via Docker Desktop, the workspace is typically mounted on a case-insensitive filesystem where `BUILD` and `build` are indistinguishable (docker/desktop-feedback#251). #48455 addressed this for `gazelle` by restricting it to `BUILD.bazel` files only. Bazel's own directory traversal, however, remained exposed to the same confusion. `REPO.bazel` gained `ignore_directories()` in Bazel **8.0*** (bazelbuild/bazel#24203), introduced as a complement to `.bazelignore` > to provide a migration path off of that weird single-purpose configuration file. It supports Bazel's native `glob()` semantics, making `**/build` a natural fit. ### Additional Notes Coming from Bazel 7, I was not aware of `REPO.bazel`, otherwise I would have added it in the first place instead of `.bazelignore` (#40153). In `CODEOWNERS`, `/MODULE.bazel*` is replaced by `/*.bazel*` to cover `REPO.bazel` as well as any future root-level `.bazel` files.
### What does this PR do? Add `**/build` to `REPO.bazel`'s `ignore_directories()`, replacing `.bazelignore`, and adjust `CODEOWNERS` accordingly. ### Motivation When `bazel` runs inside a Linux container on macOS via Docker Desktop, the workspace is typically mounted on a case-insensitive filesystem where `BUILD` and `build` are indistinguishable (docker/desktop-feedback#251). #48455 addressed this for `gazelle` by restricting it to `BUILD.bazel` files only. Bazel's own directory traversal, however, remained exposed to the same confusion. `REPO.bazel` gained `ignore_directories()` in Bazel **8.0*** (bazelbuild/bazel#24203), introduced as a complement to `.bazelignore` > to provide a migration path off of that weird single-purpose configuration file. It supports Bazel's native `glob()` semantics, making `**/build` a natural fit. ### Additional Notes Coming from Bazel 7, I was not aware of `REPO.bazel`, otherwise I would have added it in the first place instead of `.bazelignore` (#40153). In `CODEOWNERS`, `/MODULE.bazel*` is replaced by `/*.bazel*` to cover `REPO.bazel` as well as any future root-level `.bazel` files.
### What does this PR do? Add `**/build` to `REPO.bazel`'s `ignore_directories()`, partially replacing `.bazelignore`, and adjust `CODEOWNERS` accordingly. ### Motivation When `bazel` runs inside a Linux container on macOS via Docker Desktop, the workspace is typically mounted on a case-insensitive filesystem where `BUILD` and `build` are indistinguishable (docker/desktop-feedback#251). #48455 addressed this for `gazelle` by restricting it to `BUILD.bazel` files only. Bazel's own directory traversal, however, remained exposed to the same confusion. `REPO.bazel` gained `ignore_directories()` in Bazel **8.0** (bazelbuild/bazel#24203), introduced as a complement to `.bazelignore` > to provide a migration path off of that weird single-purpose configuration file. It supports Bazel's native `glob()` semantics, making `**/build` a natural fit. Unfortunately (there's always a _but_...), it doesn't always work for symlinked directories ([example](https://gitlab.ddbuild.io/DataDog/datadog-agent/-/jobs/1544684745#L117)): ``` ERROR: infinite symlink expansion detected [start of symlink chain] /go/src/github.com/DataDog/datadog-agent /go/src/github.com/DataDog/datadog-agent/.cache/bazel/_bazel_root/8595a61ca643e2561dc364b98a410786/external/_main [end of symlink chain] ``` ... which was reported here: bazelbuild/bazel#24203 (comment) So, for the time being, we also have to keep `.bazelignore`, but shrunk down to the bare minimum, until the issue is fixed upstream. ### Additional Notes Coming from Bazel 7, I was not aware of `REPO.bazel`, otherwise I would have added it in the first place instead of `.bazelignore` (#40153). In `CODEOWNERS`, `/MODULE.bazel*` is replaced by `/*.bazel*` to cover `REPO.bazel` as well as any future root-level `.bazel` files. In order to reproduce the `infinite symlink expansion detected` mentioned above, I had to configure my local `bazel` command line invocation as in CI, which surfaced the fact that `tools/bazel*` scripts had to be adjusted for this purpose. In particular, `--repo_contents_cache=` must be set when `XDG_CACHE_HOME=$PWD/.cache` (bazelbuild/bazel#26384). This is now fixed as well. Co-authored-by: regis.desgroppes <regis.desgroppes@datadoghq.com>
…nsistencies (#48279) ### What does this PR do? Remove `gazelle` exclusions that had to be put in place pending a fix for `gazelle` confused by `build` directories on "broken" case-insensitive filesystem stacks (docker/desktop-feedback#251). Doing so, I realized there were inconsistencies in the `gazelle` setup and took the opportunity to fix them. ### Motivation #48455 indeed allows to get rid of the following exclusions: ``` # TODO(agent-build): This exclusion is ready for removal, blocked by issues caused by the interaction of # Bazel with directories named `build` created by the CMake builds. We can remove this once we remove CMake # support or we upgrade to a Bazel version containing a fix: bazelbuild/bazel#26842 # In the meantime, temporarily remove this exclusion locally if regeneration of BUILD files is required # gazelle:exclude rtloader ... # gazelle:exclude .gitlab ``` Nearby inconsistencies in the `gazelle` setup: - `build_tags` is passed by attribute (`build_tags = ALL_BUILD_TAGS`) => OK (**canonical**), - `external` was passed by `args` (`-external=static`) => pass by attribute (`external = "static"`), - `prefix` was passed by directive (`# gazelle:prefix github.com/DataDog/datadog-agent`) => pass by attribute (`prefix = "github.com/DataDog/datadog-agent"`), - `args` happens to be a [compatibility shim](bazel-contrib/bazel-gazelle#62) => use `extra_args` (**canonical**) when there's no corresponding attribute yet: - `build_file_name` was passed by `args` and doesn't have a corresponding attribute yet => pass by `extra_args`, - `strict` was passed from a single GitLab job and doesn't have a corresponding attribute yet => pass by `extra_args`. ### Describe how you validated your changes Kind assistance of @alopezz and/or @JSGette who run Linux containers on their macOS-powered laptop. Co-authored-by: regis.desgroppes <regis.desgroppes@datadoghq.com>
What does this PR do?
Configure the
//:gazelletarget with-build_file_name=BUILD.bazel.Motivation
When
gazelleruns inside a Linux container on macOS via Docker Desktop, the workspace is typically mounted on a case-insensitive filesystem whereBUILDandbuildare indistinguishable.This causes spurious errors such as:
Whereas the cause strictly resides in a filesystem layer/driver (docker/desktop-feedback#251, recently imported from docker/for-mac#7218), portable tools have been taking countermeasures. Bazel 9, for instance, revamped their filesystem cache to better accomodate case-insensitive filesystems (bazelbuild/bazel#26842), which led us to switch to it (#47742), but that only covers Bazel's own I/O, not the processes it spawns, of which
gazelle.For that purpose, Gazelle happens to explicitly recommend setting its
build_file_namedirective toBUILD.bazel:Additional Notes
For consistency with the narrowed scope, this change renames
compliance/rules/BUILDtocompliance/rules/BUILD.bazel(the only plainBUILDfile still present in the tree).