Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
66498c6
refactor: Refine ADR-0004 documentation based on feedback
flyingrobots Nov 10, 2025
46ec339
docs: Clarify commit trailer format in F9-US-SEC acceptance criteria
flyingrobots Nov 10, 2025
676b6fa
docs: Clarify actor-id format in SPEC.md
flyingrobots Nov 10, 2025
7c35665
docs: Clarify size unit for OpaquePointer in SPEC.md
flyingrobots Nov 10, 2025
fce8bb1
docs: Clarify error handling for digest mismatch in SPEC.md Pointer R…
flyingrobots Nov 10, 2025
3d6bcbd
docs: Clarify PrivateStore participant as interface in TECH-SPEC.md d…
flyingrobots Nov 10, 2025
935ee36
docs: Add details for Projection Determinism test suite in TECH-SPEC.md
flyingrobots Nov 10, 2025
f643aea
docs: Update F9-US-DEV user story to BDD format in FEATURES.md
flyingrobots Nov 10, 2025
b39373c
docs: Mark gatos-compute as planned in SPEC.md system diagram
flyingrobots Nov 10, 2025
412b9b9
docs: Change PoC envelope storage requirement from SHOULD to MUST in …
flyingrobots Nov 10, 2025
c63785e
docs: Clarify purpose of gatos-kv crate in TECH-SPEC.md
flyingrobots Nov 10, 2025
1ca5ec6
docs: Mark chart data as illustrative in TECH-SPEC.md Performance Gui…
flyingrobots Nov 10, 2025
b5235c4
docs: Refine F9-US-DEV acceptance criteria to be more granular
flyingrobots Nov 10, 2025
8805e1b
docs: Simplify PrivateStore participant name in TECH-SPEC.md diagram
flyingrobots Nov 10, 2025
cc8d49e
docs: Remove redundant acceptance criteria for F9-US-DEV
flyingrobots Nov 10, 2025
f1e6f8b
docs: Add missing field description for digest in OpaquePointer
flyingrobots Nov 10, 2025
8cc6091
docs: Clarify interaction of expiration dates in Consensus Governance
flyingrobots Nov 10, 2025
f8de6f2
docs: Specify HTTP GET method for pointer resolution endpoint
flyingrobots Nov 10, 2025
e00b578
docs: Correct type casing in ADR-0004 OpaquePointer diagram
flyingrobots Nov 10, 2025
e48b7f9
docs: Further refine F9-US-DEV acceptance criteria with BDD-style gra…
flyingrobots Nov 10, 2025
b8b9504
docs: Remove redundant acceptance criteria for F9-US-DEV in FEATURES.md
flyingrobots Nov 10, 2025
5dca809
docs: Clarify sorting order for approvals in SPEC.md PoC section
flyingrobots Nov 10, 2025
a7b1a21
docs: Correct inconsistent endpoint URL in ADR-0004
flyingrobots Nov 10, 2025
d363688
docs(ADR-0004,SPEC,TECH-SPEC): switch resolver to POST + JWT; add opt…
flyingrobots Nov 10, 2025
4bbbccb
docs: harmonize DigestMismatch to 422 across SPEC/TECH-SPEC; ADR resp…
flyingrobots Nov 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ members = [
"crates/gatos-mind",
"crates/gatos-echo",
"crates/gatos-policy",
"crates/gatos-privacy",
"crates/gatos-kv",
"crates/gatosd",
"bindings/wasm",
Expand Down
10 changes: 6 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@ diagrams:
@bash -lc 'scripts/mermaid/generate_all.sh'

lint-md:
@bash -lc 'if command -v node >/dev/null 2>&1; then \
@bash -lc 'if command -v node >/dev/null 2>&1; then \
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Minor: Indentation inconsistency in lint-md and fix-md targets.

Lines 15 and 22 use mixed tabs/spaces within bash command continuation. While functionally harmless, consider normalizing indentation to project standard (tabs or spaces) for consistency with the rest of the Makefile.

Also applies to: 22-22

🤖 Prompt for AI Agents
Makefile lines 15 and 22: the bash command continuation lines use mixed tabs and
spaces; normalize these recipe lines to use the project standard (use a single
leading tab for Makefile recipe lines) and remove any stray spaces before the
backslash so the continuation lines match indentation of other targets (ensure
both lint-md and fix-md continuation lines start with a single tab and have no
mixed spaces).

npx -y markdownlint-cli "**/*.md" --config .markdownlint.json; \
elif command -v docker >/dev/null 2>&1; then \
docker run --rm -v "$$PWD:/work" -w /work node:20 bash -lc "npx -y markdownlint-cli \"**/*.md\" --config .markdownlint.json"; \
else echo "Need Node.js or Docker" >&2; exit 1; fi'

fix-md:
@bash -lc 'if command -v node >/dev/null 2>&1; then \
@bash -lc 'if command -v node >/dev/null 2>&1; then \
npx -y markdownlint-cli "**/*.md" --fix --config .markdownlint.json; \
elif command -v docker >/dev/null 2>&1; then \
docker run --rm -v "$$PWD:/work" -w /work node:20 bash -lc "npx -y markdownlint-cli \"**/*.md\" --fix --config .markdownlint.json"; \
Expand All @@ -44,7 +44,8 @@ schema-compile:
npx -y ajv-cli@5 ajv compile --spec=draft2020 --strict=true -c ajv-formats -s schemas/v1/governance/grant.schema.json -r schemas/v1/common/ids.schema.json && \
npx -y ajv-cli@5 ajv compile --spec=draft2020 --strict=true -c ajv-formats -s schemas/v1/governance/revocation.schema.json -r schemas/v1/common/ids.schema.json && \
npx -y ajv-cli@5 ajv compile --spec=draft2020 --strict=true -c ajv-formats -s schemas/v1/governance/proof_of_consensus_envelope.schema.json -r schemas/v1/common/ids.schema.json && \
npx -y ajv-cli@5 ajv compile --spec=draft2020 --strict=true -c ajv-formats -s schemas/v1/policy/governance_policy.schema.json'
npx -y ajv-cli@5 ajv compile --spec=draft2020 --strict=true -c ajv-formats -s schemas/v1/policy/governance_policy.schema.json && \
npx -y ajv-cli@5 ajv compile --spec=draft2020 --strict=true -c ajv-formats -s schemas/v1/privacy/opaque_pointer.schema.json -r schemas/v1/common/ids.schema.json'

schema-validate:
@bash -lc 'set -euo pipefail; \
Expand All @@ -57,7 +58,8 @@ schema-validate:
npx -y ajv-cli@5 ajv validate --spec=draft2020 --strict=true -c ajv-formats -s schemas/v1/governance/grant.schema.json -d examples/v1/governance/grant_min.json -r schemas/v1/common/ids.schema.json && \
npx -y ajv-cli@5 ajv validate --spec=draft2020 --strict=true -c ajv-formats -s schemas/v1/governance/revocation.schema.json -d examples/v1/governance/revocation_min.json -r schemas/v1/common/ids.schema.json && \
npx -y ajv-cli@5 ajv validate --spec=draft2020 --strict=true -c ajv-formats -s schemas/v1/governance/proof_of_consensus_envelope.schema.json -d examples/v1/governance/poc_envelope_min.json -r schemas/v1/common/ids.schema.json && \
npx -y ajv-cli@5 ajv validate --spec=draft2020 --strict=true -c ajv-formats -s schemas/v1/policy/governance_policy.schema.json -d examples/v1/policy/governance_min.json'
npx -y ajv-cli@5 ajv validate --spec=draft2020 --strict=true -c ajv-formats -s schemas/v1/policy/governance_policy.schema.json -d examples/v1/policy/governance_min.json && \
npx -y ajv-cli@5 ajv validate --spec=draft2020 --strict=true -c ajv-formats -s schemas/v1/privacy/opaque_pointer.schema.json -d examples/v1/privacy/opaque_pointer_min.json -r schemas/v1/common/ids.schema.json'

schema-negative:
@bash -lc 'set -euo pipefail; \
Expand Down
13 changes: 13 additions & 0 deletions crates/gatos-privacy/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[package]
name = "gatos-privacy"
version = "0.1.0"
edition = "2021"

[dependencies]
gatos-ledger-core = { path = "../gatos-ledger-core" }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
blake3 = { workspace = true }
hex = { workspace = true }
anyhow = { workspace = true }

42 changes: 42 additions & 0 deletions crates/gatos-privacy/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
//! gatos-privacy — Opaque Pointer types and helpers
//!
//! This crate defines the JSON-facing pointer envelope used by the
//! hybrid privacy model (ADR-0004). The struct mirrors the v1 schema
//! in `schemas/v1/privacy/opaque_pointer.schema.json`.
//!
//! Canonicalization: when computing content IDs or digests, callers
//! MUST serialize JSON using RFC 8785 JCS. This crate intentionally
//! does not take a dependency on a specific JCS implementation to
//! keep the workspace lean; higher layers may provide one.
use serde::{Deserialize, Serialize};
use serde_json::Value;

#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct OpaquePointer {
pub kind: Kind,
pub algo: Algo,
pub digest: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub ciphertext_digest: Option<String>,
Comment on lines +17 to +22

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Allow ciphertext-only pointers without digest

The new JSON schema explicitly permits opaque pointers that omit digest when ciphertext_digest is provided, but the Rust struct still models digest as a mandatory String. Any ciphertext-only pointer produced according to the schema will fail deserialization because serde expects the digest field. Either digest should be optional (Option<String>) or the schema should require it so that the type and schema remain aligned.

Useful? React with 👍 / 👎.

#[serde(skip_serializing_if = "Option::is_none")]
pub size: Option<u64>,
pub location: String,
pub capability: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub extensions: Option<Value>,
}
Comment on lines +17 to +29
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

CRITICAL: Rust struct violates schema's anyOf requirement.

Line 20 declares digest as a non-optional String, making it always required. However, schemas/v1/privacy/opaque_pointer.schema.json (lines 17-30) uses anyOf to require EITHER digest OR ciphertext_digest, allowing ciphertext-only pointers for low-entropy data per ADR-0004.

This mismatch means:

  1. Deserialization will fail for valid ciphertext-only pointers
  2. The Rust type cannot represent low-entropy privacy classes that MUST NOT expose plaintext digest

Both digest and ciphertext_digest must be Option<String>, with validation logic ensuring at least one is Some.

 #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
 #[serde(deny_unknown_fields)]
 pub struct OpaquePointer {
     pub kind: Kind,
     pub algo: Algo,
-    pub digest: String,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub digest: Option<String>,
     #[serde(skip_serializing_if = "Option::is_none")]
     pub ciphertext_digest: Option<String>,
     #[serde(skip_serializing_if = "Option::is_none")]
     pub size: Option<u64>,
     pub location: String,
     pub capability: String,
     #[serde(skip_serializing_if = "Option::is_none")]
     pub extensions: Option<Value>,
 }

Then add a validation method:

impl OpaquePointer {
    pub fn validate(&self) -> Result<(), &'static str> {
        if self.digest.is_none() && self.ciphertext_digest.is_none() {
            return Err("At least one of digest or ciphertext_digest must be present");
        }
        Ok(())
    }
}


#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum Kind {
OpaquePointer,
}
Comment on lines +31 to +35
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Consider: Single-variant enum may be premature.

The Kind enum has only one variant (OpaquePointer). Unless additional pointer kinds are planned in the near future, this could be simplified to a constant string or removed entirely, with the field being a simple &'static str or validated string.

However, if the enum is intentional for future extensibility, consider adding a doc comment explaining the planned variants.

If future kinds are planned:

/// Pointer envelope kinds.
/// 
/// Future variants may include `TransparentPointer`, `NullPointer`, etc.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum Kind {
    OpaquePointer,
    // Future: TransparentPointer, NullPointer, ...
}

If not:

// Replace enum with constant
pub const KIND: &str = "opaque_pointer";

// In struct:
pub struct OpaquePointer {
    #[serde(default = "default_kind")]
    pub kind: String,
    // ...
}

fn default_kind() -> String {
    KIND.to_string()
}
🤖 Prompt for AI Agents
In crates/gatos-privacy/src/lib.rs around lines 31 to 35, the Kind enum only has
a single variant (OpaquePointer) which is premature; if you intend to add more
pointer kinds later, keep the enum and add a doc comment above it describing
planned future variants (e.g., TransparentPointer, NullPointer) so intent is
clear; otherwise remove the enum and replace usages with a constant string
(e.g., pub const KIND: &str = "opaque_pointer") and change the struct field to a
String or &'static str with a serde default helper (add a default_kind() that
returns KIND.to_string() and annotate the field with #[serde(default =
"default_kind")]) so serialization and defaults behave the same.


#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum Algo {
Blake3,
}
Comment on lines +37 to +41
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Same consideration for Algo enum: future-proof or simplify.

The Algo enum has a single variant (Blake3). Similar to Kind, if no other algorithms are planned, consider whether an enum is necessary. However, cryptographic agility is often important, so the enum may be justified for future hash function migration.

If the enum is intentional for crypto-agility, add documentation explaining the design choice.

/// Hash algorithm for pointer digests.
/// 
/// Currently only BLAKE3 is supported. Future variants may include
/// SHA3-256 or other cryptographic hash functions as the ecosystem evolves.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum Algo {
    Blake3,
    // Future: Sha3, ...
}
🤖 Prompt for AI Agents
In crates/gatos-privacy/src/lib.rs around lines 37 to 41, the Algo enum
currently has a single variant (Blake3) which is fine but needs explicit
documentation for crypto-agility; add a doc comment above the enum explaining
that this represents the hash algorithm for pointer digests, that BLAKE3 is the
currently supported algorithm, and that additional algorithms (e.g., SHA3-256)
may be added in the future to support migration, and optionally include a
commented placeholder for future variants so intent is clear to readers and
maintainers.


50 changes: 27 additions & 23 deletions docs/FEATURES.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,29 +141,6 @@ Each feature includes user stories per relevant stakeholders (format requested),

---

## F6 — Opaque Pointers & CAS

### F6-US-DML

| | |
|--|--|
| **As a...** | Data/ML Engineer |
| **I want..** | encrypted artifacts with verifiable pointers |
| **So that...** | I can ship models across untrusted storage |

#### Acceptance Criteria

- [ ] Pointer includes plaintext hash, ciphertext hash, cipher meta
- [ ] Rekey operation available

#### Test Plan

- [ ] Golden: decrypt with correct key → match plaintext hash
- [ ] Edge: wrong bytes → hash mismatch
- [ ] Failure: rekey without authorization → deny

---

## F7 — Epochs & Compaction

### F7-US-PENG
Expand Down Expand Up @@ -208,3 +185,30 @@ Each feature includes user stories per relevant stakeholders (format requested),
- [ ] Golden: metrics show non-zero counters post workload
- [ ] Edge: cache stale → doctor recommends rebuild
- [ ] Failure: FF-only violation → doctor flags critical
---

## F9 — Hybrid Privacy Model

See also: [ADR-0004](./decisions/ADR-0004/DECISION.md).

### F9-US-DEV

Comment on lines +194 to +195
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

Missing F9-US-DEV user story content.

Line 194 declares ### F9-US-DEV but provides no table, acceptance criteria, or test plan. This appears to be a placeholder or incomplete. The ADR objectives mention "F9-US-DEV user story converted to BDD," yet no content is present.

Please provide the F9-US-DEV user story (table: As a... I want... So that...), acceptance criteria, and test plan to match the structure of F9-US-SEC (and other user stories in this file).

Alternatively, if F9-US-DEV is intentionally deferred, add a clarifying note (e.g., (deferred) or (see ADR-0004 for details)) to communicate the intent.


🏁 Script executed:

#!/bin/bash
# Extract lines 188-213 from docs/FEATURES.md to verify F9 section structure
sed -n '188,213p' docs/FEATURES.md

Length of output: 816


F9-US-DEV remains an empty stub—add user story table, acceptance criteria, and test plan.

The verification confirms line 194's ### F9-US-DEV heading is immediately followed by blank space, with ### F9-US-SEC taking over at line 196. F9-US-SEC has complete structure (user story table, acceptance criteria checklist, test plan), but F9-US-DEV has zero content.

Provide matching content for F9-US-DEV following the same structure as F9-US-SEC, or explicitly document why it's deferred (e.g., (deferred—see ADR-0004)).

🤖 Prompt for AI Agents
In docs/FEATURES.md around lines 194 to 195, the heading "### F9-US-DEV" is an
empty stub; populate it to match the structure used for F9-US-SEC by adding a
user story table (ID, Title, Description, Priority, Owner), an acceptance
criteria checklist (clear, testable bullet points), and a test plan (setup, test
steps, expected results, and rollback/edge cases), or if work is intentionally
postponed replace the body with a single-line note like "(deferred—see
ADR-0004)" plus a reference link; ensure formatting and headings follow the
exact pattern used by F9-US-SEC for consistency.

### F9-US-SEC

| | |
|--|--|
| **As a...** | Security/Compliance |
| **I want..** | to audit the separation of public and private data |
| **So that...** | I can verify that sensitive data is properly isolated and access is controlled |

#### Acceptance Criteria

- [ ] Opaque Pointer resolution fails without a valid capability.
- [ ] Private blob digest matches the digest in the public pointer.
- [ ] Commit trailers (`Privacy-Redactions`, `Privacy-Pointers`) accurately report the number of redactions/pointers.

#### Test Plan

- [ ] Golden: project a unified state, resolve pointer, and verify content matches original.
- [ ] Edge: attempt to resolve a pointer with an invalid capability URI → DENY.
- [ ] Failure: tamper with a private blob → digest mismatch on resolution.
91 changes: 73 additions & 18 deletions docs/SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ graph TD
end

subgraph "Job Plane"
Compute("gatos-compute");
Compute("gatos-compute (planned)");
end

subgraph "Ledger Plane"
Expand Down Expand Up @@ -128,6 +128,7 @@ graph TD
A1 --> B6(audit)
A1 --> B7(cache)
A1 --> B8(epoch)
A1 --> B9(private)
C(notes) --> C1(gatos)
end
subgraph Workspace
Expand All @@ -148,6 +149,8 @@ The normative layout is as follows:
│ └── gatos/
│ ├── journal/
│ ├── state/
│ ├── private/
│ │ └── <actor-id>/ # e.g., the actor's ed25519 public key
│ ├── mbus/
│ ├── mbus-ack/
│ ├── jobs/
Expand Down Expand Up @@ -282,28 +285,80 @@ On **DENY**, the gate **MUST** append an audit decision to `refs/gatos/audit/pol

---

## 7. Blob Pointers & Opaque Storage
## 7. Privacy and Opaque Pointers

Large or sensitive data is stored out-of-band in a content-addressed store and referenced via pointers.
See also: [ADR‑0004](./decisions/ADR-0004/DECISION.md).

GATOS supports a hybrid privacy model where state can be separated into a verifiable public projection and a confidential private overlay. This is achieved by applying a deterministic **Projection Functor** during the state fold process, which replaces sensitive or large data with **Opaque Pointers**.

### 7.1 Projection Model

The State Engine (`gatos-echo`) can be configured with privacy rules. When folding history, it first computes a `UnifiedState` containing all data. It then applies the privacy rules to produce a `PublicState` and a set of `PrivateBlobs`.

- **`PublicState`**: Contains only public data and Opaque Pointers. This is committed to the public `refs/gatos/state/public/...` namespace and is globally verifiable.
- **`PrivateBlobs`**: The raw data that was redacted or pointerized. This data is stored in a separate, private store (e.g., a local directory, a private object store) and is addressed by its content hash.

Any commit that is the result of a privacy projection **MUST** include trailers indicating the number of redactions and pointers created.

```text
Privacy-Redactions: 5
Privacy-Pointers: 2
```

### 7.2 Opaque Pointers

An Opaque Pointer is a canonical JSON object that acts as a verifiable, addressable link to a private blob. It replaces the sensitive data in the `PublicState`.

```mermaid
classDiagram
class BlobPointer {
+String kind: "blobptr"
+String algo
+String hash
+Number size
}
class OpaquePointer {
+String kind: "opaque"
+String algo
+String hash
+String ciphertext_hash
+Object cipher_meta
+string kind: "opaque_pointer"
+string algo: "blake3"
+string digest: "blake3:<hex>" // plaintext digest
+string ciphertext_digest: "blake3:<hex>" // optional
+int size // bytes; SHOULD be present
+string location
+string capability // MUST NOT embed secrets
+object extensions // forward-compatible
}
```

Pointers **MUST** refer to bytes in `gatos/objects/<algo>/<hash>`. For opaque objects, no plaintext **MAY** be stored in Git.
- `digest`: The **REQUIRED** `blake3` hash of the plaintext. For low‑entropy privacy classes, the public pointer MUST NOT expose this value.
- `ciphertext_digest`: The `blake3` hash of the stored ciphertext. For low‑entropy privacy classes, this field MUST be present in the public pointer.
- `size`: The size of the private blob in bytes (RECOMMENDED).
- `location`: A **REQUIRED** stable URI indicating where the blob can be fetched (e.g., `gatos-node://ed25519:<pubkey>`, `s3://bucket/key`). Do not embed pre‑signed tokens.
- `capability`: A **REQUIRED** reference to the authn/z + decryption mechanism (e.g., `gatos-key://...`, `kms://...`). It MUST NOT embed secrets; resolution occurs at the policy layer.

The pointer itself is canonicalized via RFC 8785 JCS and its `content_id` is `blake3(JCS(pointer_json))`.
Comment on lines 314 to +332
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Critical: OpaquePointer field documentation contradicts schema.

Lines 326-327 state digest is "REQUIRED", but the schema (schemas/v1/privacy/opaque_pointer.schema.json lines 17-30) makes it optional via anyOf, allowing ciphertext-only pointers.

This contradiction was flagged in the schema review. The SPEC must align with the schema's conditional requirements.

Update lines 326-332 to accurately reflect the anyOf semantics:

--   `digest`: The **REQUIRED** `blake3` hash of the plaintext. For low‑entropy privacy classes, the public pointer MUST NOT expose this value.
--   `ciphertext_digest`: The `blake3` hash of the stored ciphertext. For low‑entropy privacy classes, this field MUST be present in the public pointer.
+-   `digest`: The `blake3` hash of the plaintext. For high-entropy privacy classes, this field is **REQUIRED** and `ciphertext_digest` is **OPTIONAL**. For low-entropy privacy classes, this field **MUST NOT** be present in the public pointer.
+-   `ciphertext_digest`: The `blake3` hash of the stored ciphertext. For low-entropy privacy classes, this field is **REQUIRED**. For high-entropy classes, this field is **OPTIONAL**. At least one of `digest` or `ciphertext_digest` **MUST** be present.
     `size`: The size of the private blob in bytes (RECOMMENDED).
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
class OpaquePointer {
+String kind: "opaque"
+String algo
+String hash
+String ciphertext_hash
+Object cipher_meta
+string kind: "opaque_pointer"
+string algo: "blake3"
+string digest: "blake3:<hex>" // plaintext digest
+string ciphertext_digest: "blake3:<hex>" // optional
+int size // bytes; SHOULD be present
+string location
+string capability // MUST NOT embed secrets
+object extensions // forward-compatible
}
```
Pointers **MUST** refer to bytes in `gatos/objects/<algo>/<hash>`. For opaque objects, no plaintext **MAY** be stored in Git.
- `digest`: The **REQUIRED** `blake3` hash of the plaintext. For low‑entropy privacy classes, the public pointer MUST NOT expose this value.
- `ciphertext_digest`: The `blake3` hash of the stored ciphertext. For low‑entropy privacy classes, this field MUST be present in the public pointer.
- `size`: The size of the private blob in bytes (RECOMMENDED).
- `location`: A **REQUIRED** stable URI indicating where the blob can be fetched (e.g., `gatos-node://ed25519:<pubkey>`, `s3://bucket/key`). Do not embed pre‑signed tokens.
- `capability`: A **REQUIRED** reference to the authn/z + decryption mechanism (e.g., `gatos-key://...`, `kms://...`). It MUST NOT embed secrets; resolution occurs at the policy layer.
The pointer itself is canonicalized via RFC 8785 JCS and its `content_id` is `blake3(JCS(pointer_json))`.
class OpaquePointer {
string kind: "opaque_pointer"
string algo: "blake3"
string digest: "blake3:<hex>" // plaintext digest
string ciphertext_digest: "blake3:<hex>" // optional
int size // bytes; SHOULD be present
string location
string capability // MUST NOT embed secrets
object extensions // forward-compatible
}
- `digest`: The `blake3` hash of the plaintext. For high-entropy privacy classes, this field is **REQUIRED** and `ciphertext_digest` is **OPTIONAL**. For low-entropy privacy classes, this field **MUST NOT** be present in the public pointer.
- `ciphertext_digest`: The `blake3` hash of the stored ciphertext. For low-entropy privacy classes, this field is **REQUIRED**. For high-entropy classes, this field is **OPTIONAL**. At least one of `digest` or `ciphertext_digest` **MUST** be present.
- `size`: The size of the private blob in bytes (RECOMMENDED).
- `location`: A **REQUIRED** stable URI indicating where the blob can be fetched (e.g., `gatos-node://ed25519:<pubkey>`, `s3://bucket/key`). Do not embed pre‑signed tokens.
- `capability`: A **REQUIRED** reference to the authn/z + decryption mechanism (e.g., `gatos-key://...`, `kms://...`). It MUST NOT embed secrets; resolution occurs at the policy layer.
The pointer itself is canonicalized via RFC 8785 JCS and its `content_id` is `blake3(JCS(pointer_json))`.


### 7.3 Pointer Resolution

Endpoint and AuthN:
- Clients MUST resolve via `POST /gatos/private/blobs/resolve` with body `{ "digest": "blake3:<hex>", "want": "plaintext"|"ciphertext" }` and `Authorization: Bearer <JWT>`.
- Tokens MUST include standard claims (`sub`, `aud`, `method`, `path`, `exp`, `nbf`); skew tolerance ±300s. 401 for authn failures; 403 for policy denials.

Verification Steps:
1. Fetch the ciphertext blob from `location` via the node’s resolver endpoint.
2. Acquire the necessary keys via the `capability` reference (policy-driven; no secrets in the pointer).
3. Decrypt. Compute `blake3(ciphertext)` and compare with `ciphertext_digest` when present; compute `blake3(plaintext)` and compare with `digest` when exposed. Any mismatch MUST yield `DigestMismatch`.
4. Servers SHOULD return `X-BLAKE3-Digest` and `Digest: sha-256=…` headers for response integrity.

Error Taxonomy:
- `Unauthorized` (401), `Forbidden` (403), `NotFound` (404), `DigestMismatch` (422), `CapabilityUnavailable` (503), `PolicyDenied` (403).

Optional HTTP Message Signatures profile (RFC 9421):
- As an alternative to JWT, clients MAY sign `@method`, `@target-uri`, `date`, `host`, `content-digest` and send `Signature-Input`/`Signature` headers. Servers SHOULD still emit `Digest` and `X-BLAKE3-Digest` response headers.

Pointer Rotation (Rekey):
1) fetch ciphertext; 2) decrypt; 3) re‑encrypt per new capability; 4) store new ciphertext; 5) emit rotation event updating pointer fields (capability/location). `digest` (plaintext) MUST remain stable. Add trailer `Privacy-Pointer-Rotations: <n>`.

Namespacing:
- `refs/gatos/private/<actor-id>/…` holds private overlay indices/metadata only; workspace mirror is `gatos/private/<actor-id>/…`. Blobs live in external stores keyed by digest.

Canonicalization:
- All JSON labeled as canonical MUST use RFC 8785 JCS; non‑JSON maps MUST be ordered lexicographically by lowercase UTF‑8 keys.

This process guarantees that even though the data is stored privately, its integrity is verifiable against the public ledger.

---

Expand Down Expand Up @@ -624,7 +679,7 @@ Proposal → Approvals (N‑of‑M) → Grant. Quorum groups (e.g., `@leads`) MU
Proposal-Id: blake3:<hex>
Approval-Id: blake3:<hex>
Signer: ed25519:<pubkey>
Expires-At: <ISO8601> # OPTIONAL
Expires-At: <ISO8601> # OPTIONAL. If present, the approval is only valid until this time. It cannot extend the proposal's expiration.
```

- Grant (at `refs/gatos/grants/…`):
Expand All @@ -640,10 +695,10 @@ Proposal → Approvals (N‑of‑M) → Grant. Quorum groups (e.g., `@leads`) MU
`Proof-Of-Consensus` is the BLAKE3 of a canonical JSON envelope containing:

- The canonical proposal envelope (by value or `Proposal-Id`).
- A sorted list (by `Signer`) of all valid approvals used to reach quorum (by value or `Approval-Id`).
- A lexicographically sorted list of approvals ordered by the lowercase ASCII of each approval's `Signer` value (the `ed25519:<hex>` string). Each approval is included by value or via `Approval-Id`.
- The governance rule id (`Policy-Rule`) and effective quorum parameters.

PoC envelope SHOULD be stored canonically under `refs/gatos/audit/proofs/governance/<proposal-id>`; the Grant’s `Proof-Of-Consensus` trailer MUST equal `blake3(envelope_bytes)`.
PoC envelope MUST be stored canonically under `refs/gatos/audit/proofs/governance/<proposal-id>`; the Grant’s `Proof-Of-Consensus` trailer MUST equal `blake3(envelope_bytes)`.

### 20.4 Lifecycle States

Expand Down
Loading
Loading