Provenance #321

cole-h · 2026-01-22T19:36:43Z

Rather than having to call this, is there any way we could maybe make this automatic?

(Mostly thinking about protecting from developer error -- if I have to write code here for whatever reason, and add a new path where I should have called setRootProvenance but forgot to, that's probably not great. But if it's integrated into the construction of e.g. attrPath, that means we can't forget it because it'll always happen)

coderabbitai · 2026-01-21T16:25:09Z

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash # Search for "fetchurl" references in the codebase echo "=== Searching for 'fetchurl' ===" rg -n "fetchurl" --type-list | head -5 rg -n "fetchurl" 2>/dev/null | head -20 echo -e "\n=== Searching for Provenance deserialization patterns ===" rg -n "from_json|Provenance::from_json" 2>/dev/null | head -20 echo -e "\n=== Searching for type dispatch/registry in provenance ===" rg -n "\"type\"" src/libutil src/libfetchers src/libstore 2>/dev/null | grep -i provenance | head -20

Repository: DeterminateSystems/nix-src

Length of output: 4928

🏁 Script executed:

#!/bin/bash # Look for Provenance class definition and from_json implementation echo "=== Finding Provenance class definition ===" fd -e "\.hh" -e "\.cc" | xargs rg -l "class Provenance" 2>/dev/null echo -e "\n=== Search for from_json in provenance files ===" fd -e "\.hh" -e "\.cc" | xargs rg -A 10 "from_json.*Provenance" 2>/dev/null | head -40

Repository: DeterminateSystems/nix-src

Length of output: 163

🏁 Script executed:

#!/bin/bash # Look for JSON type dispatching for provenance echo "=== Searching for provenance type mapping ===" rg -n "type.*provenance\|provenance.*type" -i src/ 2>/dev/null | head -20 echo -e "\n=== Checking for nlohmann json deserialization ===" rg -n "nlohmann::json\|\.get\(\)" src/libfetchers src/libutil -A 2 2>/dev/null | grep -A 2 "type\|fetchurl" | head -30

Repository: DeterminateSystems/nix-src

Length of output: 167

🏁 Script executed:

#!/bin/bash # Read the from_json implementation echo "=== src/libutil/provenance.cc from_json implementation ===" sed -n '32,80p' src/libutil/provenance.cc echo -e "\n=== Check full provenance.cc ===" wc -l src/libutil/provenance.cc src/libfetchers/provenance.cc src/libstore/provenance.cc 2>/dev/null echo -e "\n=== Read src/libutil/provenance.cc entirely ===" cat -n src/libutil/provenance.cc

Repository: DeterminateSystems/nix-src

Length of output: 2603

🏁 Script executed:

#!/bin/bash # Check what files include/define FetchurlProvenance echo "=== Files mentioning FetchurlProvenance ===" rg -n "FetchurlProvenance" --type cc --type hh 2>/dev/null echo -e "\n=== Check src/libfetchers/provenance.cc ===" cat -n src/libfetchers/provenance.cc 2>/dev/null || echo "File not found" echo -e "\n=== Check all provenance related headers ===" fd -e ".hh" | xargs rg -l "Provenance" 2>/dev/null | head -10

Repository: DeterminateSystems/nix-src

Length of output: 1529

🏁 Script executed:

#!/bin/bash # Read provenance headers echo "=== src/libutil/include/nix/util/provenance.hh ===" cat -n src/libutil/include/nix/util/provenance.hh echo -e "\n=== src/libstore/include/nix/store/provenance.hh ===" cat -n src/libstore/include/nix/store/provenance.hh

Repository: DeterminateSystems/nix-src

Length of output: 3463

🏁 Script executed:

#!/bin/bash # Check for registration patterns echo "=== Looking for RegisteredTypes registration ===" rg -n "registeredTypes\|RegisteredTypes\|register.*[Pp]rovenance" -A 3 -B 1 --type cc --type hh 2>/dev/null | head -60 echo -e "\n=== Check tarball.cc for FetchurlProvenance context ===" sed -n '1,50p' src/libfetchers/tarball.cc

Repository: DeterminateSystems/nix-src

Length of output: 1265

🏁 Script executed:

#!/bin/bash # Search for Provenance::Register usage pattern echo "=== Looking for Provenance::Register instantiation ===" rg -n "Provenance::Register\|Register.*type" -B 2 -A 5 --type cc --type hh 2>/dev/null echo -e "\n=== Check full src/libstore/provenance.cc ===" cat -n src/libstore/provenance.cc echo -e "\n=== Search for registration in fetchers/provenance ===" cat -n src/libfetchers/provenance.cc

Repository: DeterminateSystems/nix-src

Length of output: 2218

🏁 Script executed:

#!/bin/bash # Look for how other provenance types register themselves echo "=== Check for registrations across all provenance implementations ===" find . -name "provenance*" -type f \( -name "*.cc" -o -name "*.hh" \) 2>/dev/null | xargs cat -n 2>/dev/null | grep -A 10 "Register" echo -e "\n=== Full tarball.cc provenance section ===" sed -n '12,100p' src/libfetchers/tarball.cc

Repository: DeterminateSystems/nix-src

Length of output: 3819

Register FetchurlProvenance for deserialization.

The "fetchurl" type lacks a Provenance::Register instantiation. Without it, deserialization of provenance with "type": "fetchurl" will fall back to UnknownProvenance (line 40 of src/libutil/provenance.cc), losing the typed information. Add a registration in this file or src/libfetchers/provenance.cc:

Provenance::Register registerFetchurlProvenance("fetchurl", [](nlohmann::json json) { auto & obj = getObject(json); return make_ref<FetchurlProvenance>(getString(valueAt(obj, "url"))); });

Additionally, URLs in provenance may contain credentials or signed tokens; consider redacting sensitive components before serialization.

🤖 Prompt for AI Agents

In `@src/libfetchers/tarball.cc` around lines 12 - 34, Add a Provenance::Register for the FetchurlProvenance type so deserialization of {"type":"fetchurl"} returns a FetchurlProvenance instance: register a Provenance::Register (e.g. registerFetchurlProvenance) that uses getObject(json), valueAt(obj, "url"), getString(...) and make_ref<FetchurlProvenance>(...) to construct the object; place this registration in this file (or src/libfetchers/provenance.cc) so the type is recognized at load time. Also sanitize/redact sensitive components of the url before storing/serializing in FetchurlProvenance::to_json (e.g. strip credentials or tokens) to avoid leaking credentials in provenance.

-Original file line number
+Diff line change
@@ Expand Up / @@ -179,6 +179,15 @@ $defs: @@
               The total size of this store object and every other object in its [closure](@docroot@/glossary.md#gloss-closure).
               > This field is not stored at all, but computed by traversing the other fields across all the store objects in a closure.
+          provenance:
+            oneOf:
+              - type: "null"
+              - type: object # FIXME
+            title: Provenance
+            description: |
+              An arbitrary JSON object containing provenance information about the store object, or `null` if not available.
         additionalProperties: false
       narInfo:
@@ Expand Down Expand Up / @@ -262,4 +271,13 @@ $defs: @@
               > This is an impure "`.narinfo`" field that may not be included in certain contexts.
               > This field is not stored at all, but computed by traversing the other fields across all the store objects in a closure.
+          provenance:
+            oneOf:
+              - type: "null"
+              - type: object # FIXME
+            title: Provenance
+            description: |
+              An arbitrary JSON object containing provenance information about the store object, or `null` if not available.
         additionalProperties: false

-Original file line number
+Diff line change
@@ Expand Up / @@ -72,6 +72,8 @@ struct InstallableFlake : InstallableValue @@
         ref<flake::LockedFlake> getLockedFlake() const;
         FlakeRef nixpkgsFlakeRef() const;
+        std::shared_ptr<const Provenance> makeProvenance(std::string_view attrPath) const;
     };
     /**
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -17,6 +17,7 @@ @@
     #include "nix/util/url.hh"
     #include "nix/fetchers/registry.hh"
     #include "nix/store/build-result.hh"
+    #include "nix/flake/provenance.hh"
     #include <regex>
     #include <queue>
@@ Expand Down Expand Up / @@ -84,6 +85,8 @@ DerivedPathsWithInfo InstallableFlake::toDerivedPaths() @@
         auto attrPath = attr->getAttrPathStr();
+        state->setRootProvenance(makeProvenance(attrPath));
         if (!attr->isDerivation()) {
             // FIXME: use eval cache?
@@ Expand Down Expand Up @@
         for (auto & attrPath : attrPaths) {
             debug("trying flake output attribute '%s'", attrPath);
+            state.setRootProvenance(makeProvenance(attrPath));
             auto attr = root->findAlongAttrPath(AttrPath::parse(state, attrPath));
             if (attr) {
                 res.push_back(ref(*attr));
@@ Expand Down Expand Up / @@ -212,4 +217,12 @@ FlakeRef InstallableFlake::nixpkgsFlakeRef() const @@
         return defaultNixpkgsFlakeRef();
     }
+    std::shared_ptr<const Provenance> InstallableFlake::makeProvenance(std::string_view attrPath) const
+    {
+        auto provenance = getLockedFlake()->flake.provenance;
+        if (!provenance)
+            return nullptr;
+        return std::make_shared<const FlakeProvenance>(provenance, std::string(attrPath));
+    }
     } // namespace nix

-Original file line number
+Diff line change
@@ Expand Up / @@ -3369,4 +3369,14 @@ void EvalState::waitForAllPaths() @@
         asyncPathWriter->waitForAllPaths();
     }
+    std::shared_ptr<const Provenance> EvalState::getRootProvenance()
+    {
+        return rootProvenance;
+    }
+    void EvalState::setRootProvenance(std::shared_ptr<const Provenance> provenance)
+    {
+        rootProvenance = provenance;
+    }
     } // namespace nix

-Original file line number
+Diff line change
@@ Expand Up / @@ -52,6 +52,7 @@ enum RepairFlag : bool; @@
     struct MemorySourceAccessor;
     struct MountedSourceAccessor;
     struct AsyncPathWriter;
+    struct Provenance;
     namespace eval_cache {
     class EvalCache;
@@ Expand Down Expand Up / @@ -1127,7 +1128,17 @@ private: @@
         friend struct Value;
         friend class ListBuilder;
+        std::shared_ptr<const Provenance> rootProvenance;
     public:
+        /**
+         * Set the provenance of derivations instantiated by the evaluator.
+         */
+        void setRootProvenance(std::shared_ptr<const Provenance> provenance);
+        std::shared_ptr<const Provenance> getRootProvenance();
         /**
          * Worker threads manager.
          *
@@ Expand Down @@

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Provenance #321

Diff view

Diff view

There are no files selected for viewing

cole-h Jan 22, 2026

Uh oh!

coderabbitai bot Jan 21, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

-Original file line number
+Diff line change
@@ Expand Up @@
         }
         /* Write the resulting term into the Nix store directory. */
-        auto drvPath = writeDerivation(*state.store, *state.asyncPathWriter, drv, state.repair);
+        auto drvPath =
+            writeDerivation(*state.store, *state.asyncPathWriter, drv, state.repair, false, state.getRootProvenance());
         auto drvPathS = state.store->printStorePath(drvPath);
         printMsg(lvlChatty, "instantiated '%1%' -> '%2%'", drvName, drvPathS);
@@ Expand Down Expand Up @@
                                                          ContentAddressMethod::Raw::Text,
                                                          HashAlgorithm::SHA256,
                                                          refs,
-                                                         state.repair);
+                                                         state.repair,
+                                                         state.getRootProvenance());
                                                  });
         /* Note: we don't need to add `context' to the context of the
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -27,6 +27,9 @@ nlohmann::json attrsToJSON(const Attrs & attrs) @@
     {
         nlohmann::json json;
         for (auto & attr : attrs) {
+            /* The __final attribute is purely internal, so never serialize it. */
+            if (attr.first == "__final")
+                continue;
             if (auto v = std::get_if<uint64_t>(&attr.second)) {
                 json[attr.first] = *v;
             } else if (auto v = std::get_if<std::string>(&attr.second)) {
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -4,7 +4,7 @@ @@
     #include "nix/fetchers/fetch-to-store.hh"
     #include "nix/util/json-utils.hh"
     #include "nix/fetchers/fetch-settings.hh"
-    #include "nix/fetchers/fetch-to-store.hh"
+    #include "nix/fetchers/provenance.hh"
     #include "nix/util/url.hh"
     #include "nix/util/forwarding-source-accessor.hh"
     #include "nix/util/archive.hh"
@@ Expand Down Expand Up / @@ -196,7 +196,6 @@ bool Input::contains(const Input & other) const @@
         return false;
     }
-    // FIXME: remove
     std::tuple<StorePath, ref<SourceAccessor>, Input> Input::fetchToStore(const Settings & settings, Store & store) const
     {
         if (!scheme)
@@ Expand Down Expand Up @@
                     {{"hash", store.queryPathInfo(*storePath)->narHash.to_string(HashFormat::SRI, true)}});
             }
+            accessor->provenance = std::make_shared<TreeProvenance>(*this);
             // FIXME: ideally we would use the `showPath()` of the
             // "real" accessor for this fetcher type.
             accessor->setPathDisplay("«" + to_string(true) + "»");
@@ Expand All @@
             else
                 accessor->fingerprint = result.getFingerprint(store);
+            accessor->provenance = std::make_shared<TreeProvenance>(result);
             return {accessor, std::move(result)};
         } catch (Error & e) {
             if (storePath) {
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up @@
         return next->getFingerprint(prefix / path);
     }
+    std::shared_ptr<const Provenance> FilteringSourceAccessor::getProvenance(const CanonPath & path)
+    {
+        if (provenance)
+            return SourceAccessor::getProvenance(path);
+        return next->getProvenance(prefix / path);
+    }
     void FilteringSourceAccessor::invalidateCache(const CanonPath & path)
     {
         next->invalidateCache(prefix / path);
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -52,6 +52,8 @@ struct FilteringSourceAccessor : SourceAccessor @@
         std::pair<CanonPath, std::optional<std::string>> getFingerprint(const CanonPath & path) override;
+        std::shared_ptr<const Provenance> getProvenance(const CanonPath & path) override;
         void invalidateCache(const CanonPath & path) override;
         /**
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -10,6 +10,7 @@ headers = files( @@
       'git-lfs-fetch.hh',
       'git-utils.hh',
       'input-cache.hh',
+      'provenance.hh',
       'registry.hh',
       'tarball.hh',
     )

-Original file line number
+Diff line change
@@ -0,0 +1,17 @@
+    #pragma once
+    #include "nix/util/provenance.hh"
+    #include "nix/fetchers/fetchers.hh"
+    namespace nix {
+    struct TreeProvenance : Provenance
+    {
+        ref<nlohmann::json> attrs;
+        TreeProvenance(const fetchers::Input & input);
+        nlohmann::json to_json() const override;
+    };
+    } // namespace nix

-Original file line number
+Diff line change
@@ Expand Up / @@ -49,6 +49,7 @@ sources = files( @@
       'input-cache.cc',
       'mercurial.cc',
       'path.cc',
+      'provenance.cc',
       'registry.cc',
       'tarball.cc',
     )
@@ Expand Down @@

-Original file line number
+Diff line change
@@ -0,0 +1,27 @@
+    #include "nix/fetchers/provenance.hh"
+    #include "nix/fetchers/attrs.hh"
+    #include <nlohmann/json.hpp>
+    namespace nix {
+    TreeProvenance::TreeProvenance(const fetchers::Input & input)
+        : attrs(make_ref<nlohmann::json>([&]() {
+            // Remove the narHash attribute from the provenance info, as it's redundant (it's already recorded in the store
+            // path info).
+            auto attrs2 = input.attrs;
+            attrs2.erase("narHash");
+            return fetchers::attrsToJSON(attrs2);
+        }()))
+    {
+    }
+    nlohmann::json TreeProvenance::to_json() const
+    {
+        return nlohmann::json{
+            {"type", "tree"},
+            {"attrs", *attrs},
+        };
+    }
+    } // namespace nix

-Original file line number
+Diff line change
@@ Expand Up / @@ -9,9 +9,30 @@ @@
     #include "nix/store/store-api.hh"
     #include "nix/fetchers/git-utils.hh"
     #include "nix/fetchers/fetch-settings.hh"
+    #include "nix/util/provenance.hh"
+    #include <nlohmann/json.hpp>
     namespace nix::fetchers {
+    struct FetchurlProvenance : Provenance
+    {
+        std::string url;
+        FetchurlProvenance(const std::string & url)
+            : url(url)
+        {
+        }
+        nlohmann::json to_json() const override
+        {
+            return nlohmann::json{
+                {"type", "fetchurl"},
+                {"url", url},
+            };
+        }
+    };
     DownloadFileResult downloadFile(
         Store & store,
         const Settings & settings,
@@ Expand Down Expand Up / @@ -83,6 +104,13 @@ DownloadFileResult downloadFile( @@
                 },
                 hashString(HashAlgorithm::SHA256, sink.s));
             info.narSize = sink.s.size();
+            if (experimentalFeatureSettings.isEnabled(Xp::Provenance)) {
+                auto sanitizedUrl = request.uri.parsed();
+                if (sanitizedUrl.authority)
+                    sanitizedUrl.authority->password.reset();
+                sanitizedUrl.query.clear();
+                info.provenance = std::make_shared<FetchurlProvenance>(sanitizedUrl.to_string());
+            }
             auto source = StringSource{sink.s};
             store.addToStore(info, source, NoRepair, NoCheckSigs);
             storePath = std::move(info.path);
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -269,6 +269,7 @@ static Flake readFlake( @@
             .resolvedRef = resolvedRef,
             .lockedRef = lockedRef,
             .path = flakePath,
+            .provenance = flakePath.getProvenance(),
         };
         if (auto description = vInfo.attrs()->get(state.s.description)) {
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -10,6 +10,7 @@ @@
     namespace nix {
     class EvalState;
+    struct Provenance;
     namespace flake {
@@ Expand Down Expand Up / @@ -94,6 +95,11 @@ struct Flake @@
          */
         SourcePath path;
+        /**
+         * Cached provenance of `flake.nix` (equivalent to `path.getProvenance()`).
+         */
+        std::shared_ptr<const Provenance> provenance;
         /**
          * Pretend that `lockedRef` is dirty.
          */
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -6,6 +6,7 @@ headers = files( @@
       'flake.hh',
       'flakeref.hh',
       'lockfile.hh',
+      'provenance.hh',
       'settings.hh',
       'url-name.hh',
     )

-Original file line number
+Diff line change
@@ Expand Up @@
             if (auto lockedNode = node.dynamic_pointer_cast<const LockedNode>()) {
                 n["original"] = fetchers::attrsToJSON(lockedNode->originalRef.toAttrs());
                 n["locked"] = fetchers::attrsToJSON(lockedNode->lockedRef.toAttrs());
-                /* For backward compatibility, omit the "__final"
-                   attribute. We never allow non-final inputs in lock files
-                   anyway. */
                 assert(lockedNode->lockedRef.input.isFinal() || lockedNode->lockedRef.input.isRelative());
-                n["locked"].erase("__final");
                 if (!lockedNode->isFlake)
                     n["flake"] = false;
                 if (lockedNode->buildTime)
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -45,6 +45,7 @@ sources = files( @@
       'flake.cc',
       'flakeref.cc',
       'lockfile.cc',
+      'provenance.cc',
       'settings.cc',
       'url-name.cc',
     )
@@ Expand Down @@

Provenance #321

Are you sure you want to change the base?

Provenance #321

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

cole-h Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!