From 853def8f6ef750475e728b2ee1bbce9ec7c49630 Mon Sep 17 00:00:00 2001 From: Mike Williams Date: Wed, 11 Feb 2026 09:30:38 -0500 Subject: [PATCH] chore: clarify how metric config changes will apply to existing experiments --- docs/data-analysis/jetstream/overview.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/data-analysis/jetstream/overview.md b/docs/data-analysis/jetstream/overview.md index bc0e05ac4..56605df43 100644 --- a/docs/data-analysis/jetstream/overview.md +++ b/docs/data-analysis/jetstream/overview.md @@ -114,12 +114,17 @@ It prevents sudden changes of how results are computed after tooling (such as [m ### How to Use the Latest Tooling and Metric Definitions? When tooling or metrics are updated, there are a few options with different consequences for how Jetstream treats existing experiments: -1. **do nothing**: only experiments that are launched after the new tooling release will use the most recent version of the tooling +1. **do nothing**: only experiments that are launched after the new tooling release will use the most recent version of the tooling + - NOTE: this does not apply to metric changes, which automatically get rerun or rerun-skip depending on how changes are landed. 1. **use new tooling version without rerunning**: install the [jetstream command line tooling](https://pypi.org/project/mozilla-jetstream/) locally and run `jetstream rerun-skip --experiment_slug=`. This command pretends to rerun the experiment without actually re-running the queries, which will update the date used to determine which tooling and metric versions to use. Rerunning an experiment will always force new versions to be used for experiments. + - NOTE: this option will apply for metric changes if the metric-hub PR is landed **with** `[ci rerun-skip]` in the title. 1. **rerun experiment**: re-running an experiment will always use the most recent version of the tooling on the rerun and update the last updated timestamps of the result tables + - NOTE: this option will apply for metric changes if the metric-hub PR is landed **without** `[ci rerun-skip]` in the title. More detailled information around versioning is available in [Jetstream Operations]. +Also see the [metric-hub README] for more information on how and when to use `[ci rerun-skip]` when landing configuration changes. + ## Datasets The datasets that back the Experimenter results dashboards @@ -135,3 +140,4 @@ is available in the Mozilla data docs. [outcome]: ./outcomes [mozanalysis]: https://github.com/mozilla/mozanalysis [Jetstream Operations]: ./operations#tooling-and-metric-versioning +[metric-hub README]: https://github.com/mozilla/metric-hub?tab=readme-ov-file#experiment-analysis-considerations