Skip to content

update roadmap 2 #67

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
flyingrobots wants to merge 25 commits into
base: main
Choose a base branch
from
+3,509 −718
Open

update roadmap 2 #67

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
7cf53dc
feat(adr): add ADR-0004 for hybrid privacy model
flyingrobots Nov 16, 2025
c2e6527
docs(adr): enhance ADR-0004 with clarifications and alternatives cons…
flyingrobots Nov 16, 2025
6e74200
feat(adr): add ADR-0005 for shiplog event stream
flyingrobots Nov 17, 2025
e6b1fc2
docs(adr): enhance ADR-0005 with clarifications and alternatives
flyingrobots Nov 17, 2025
d4fcd88
docs ADR-0005 udpates
flyingrobots Nov 17, 2025
13b55cf
Add ADR-0006, sweep ADR-0005
flyingrobots Nov 17, 2025
0b44935
docs ADR-0006 example schemas
flyingrobots Nov 17, 2025
8004abc
ADR 0006-0017
flyingrobots Nov 17, 2025
6cb9a20
ADR-0007 fix some details that were missing
flyingrobots Nov 17, 2025
7ca80b0
docs: more ADR refinement
flyingrobots Nov 17, 2025
cd3da09
docs: ADR-0010 updates
flyingrobots Nov 17, 2025
f08c95c
docs: Updates based on ADRs
flyingrobots Nov 17, 2025
48f096b
docs: expand proof + schema coverage
flyingrobots Nov 18, 2025
9d53708
docs: clarify REST scopes and dlq ops
flyingrobots Nov 18, 2025
f74d9db
docs: document federated stream semantics
flyingrobots Nov 18, 2025
cb05c9d
docs: capture GH approval bridge + queue ops
flyingrobots Nov 18, 2025
d011f29
docs: define export envelope + samples
flyingrobots Nov 18, 2025
6bf73f8
docs: lock down federation discovery
flyingrobots Nov 18, 2025
888c43e
docs: define fold prewarm + cache policy
flyingrobots Nov 18, 2025
1dd22a1
chore: Cargo.lock update
flyingrobots Nov 18, 2025
4d285eb
docs: normalize ADR metadata and roadmap
flyingrobots Nov 18, 2025
2e0469b
docs: add structured task backlog
flyingrobots Nov 18, 2025
41262a6
docs: add ADRs for sessions and PoX
flyingrobots Nov 18, 2025
4970d12
docs: align spec and guide with sessions & PoX
flyingrobots Nov 18, 2025
b70539b
docs: add operations & observability chapter
flyingrobots Nov 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,14 @@ scripts/setup-hooks.sh

If the hook fails, fix the reported issues and retry the commit.

> **ADR-0006 preview:** Once `git gatos install-hooks` lands, use it instead of `scripts/setup-hooks.sh`. It installs the managed `pre-commit`, `pre-push`, and `post-checkout/merge` hooks referenced in the spec, plus records bypasses under `refs/gatos/audit/locks/*`.

### Watcher / Lock Testing

- `git gatos watch --once` — run a single enforcement pass to verify `.gatos/policy.yaml` `locks[]` before committing docs.
- `git gatos lock acquire path1 path2` — exercise the governance workflow locally; grants appear under `refs/gatos/grants/...` and the watcher should clear read-only bits automatically.
- Event logs live in `~/.config/gatos/watch/events.log`; attach them to issues when debugging local enforcement.
Comment on lines +83 to +89
Copy link

@coderabbitai coderabbitai bot Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

Verify CLI commands and event log paths are documented and implemented.

Lines 83-89 introduce new developer commands (git gatos watch --once, git gatos lock acquire) and a hardcoded event log path (~/.config/gatos/watch/events.log). These must be verified to exist in the implementation and be documented in the CLI help/man pages.

To verify these commands exist and are discoverable:

🏁 Script executed:

#!/bin/bash
# Search for CLI command definitions for watch and lock subcommands
rg -nP '(?:subcommand|command|Command)\s*.*(?:watch|lock)' -g '!*.md' --type rust | head -20

# Search for the hardcoded event log path
rg -n '\.config/gatos/watch/events\.log' -g '!*.md'

# Check if watch and lock are documented in help
rg -n 'fn.*watch|fn.*lock' --type rust | head -10

Length of output: 0

🏁 Script executed:

#!/bin/bash
# Discover repository structure and primary language
echo "=== Repository Structure ==="
find . -maxdepth 2 -type f \( -name "*.rs" -o -name "*.go" -o -name "*.py" -o -name "*.ts" -o -name "*.js" -o -name "Cargo.toml" -o -name "go.mod" -o -name "package.json" \) | head -20

echo -e "\n=== Top-level files ==="
ls -la | grep -E '\.(rs|go|py|toml|mod|json)$|Makefile|Dockerfile'

echo -e "\n=== Broad search: 'watch' command ==="
rg -n 'watch' -g '!*.md' -g '!*.lock' -g '!*.txt' | head -15

echo -e "\n=== Broad search: 'lock' command ==="
rg -n 'lock' -g '!*.md' -g '!*.lock' -g '!*.txt' | head -15

echo -e "\n=== Search for event log path ==="
rg -n 'events\.log|watch.*log|gatos.*log' -g '!*.md' | head -10

Length of output: 2642

🏁 Script executed:

#!/bin/bash
# Identify repository purpose
echo "=== README intro ==="
head -50 README.md 2>/dev/null | head -20

echo -e "\n=== Cargo.toml purpose ==="
head -20 Cargo.toml

echo -e "\n=== ADR-0006 references ==="
rg -n 'ADR-0006|adr-0006|adr0006' -g '!*.md' --no-heading

echo -e "\n=== Check for gatos implementation or references ==="
rg -n 'git.*gatos|gatos.*install|gatos.*lock|gatos.*watch' -g '*.md' | head -15

echo -e "\n=== Check CONTRIBUTING.md for context about these features ==="
rg -nB5 -A5 'gatos.*watch|gatos.*lock|gatos.*install-hooks' CONTRIBUTING.md | head -40

echo -e "\n=== Directory structure ==="
ls -la src/ 2>/dev/null || echo "[no src/ directory]"
ls -la .gatos/ 2>/dev/null || echo "[no .gatos/ directory]"

Length of output: 5279

🏁 Script executed:

#!/bin/bash
echo "=== Searching crates for 'watch' command handler ==="
find crates -type f -name '*.rs' | xargs grep -l 'watch' | head -10

echo -e "\n=== Searching crates for 'lock' command handler ==="
find crates -type f -name '*.rs' | xargs grep -l 'lock.*acquire\|lock.*command' | head -10

echo -e "\n=== Checking CLI crate structure ==="
ls -la crates/*/src/ 2>/dev/null | grep -E '^d|^-' | head -20

echo -e "\n=== Search for event log path configuration ==="
find crates -type f -name '*.rs' | xargs grep -n 'events\.log\|watch.*log\|config.*gatos' 2>/dev/null | head -15

echo -e "\n=== Check main CLI entry (gatosd or similar) ==="
find crates -name 'main.rs' -o -name 'cli.rs' | xargs ls -la 2>/dev/null

echo -e "\n=== ADR-0006 full search (including non-.md) ==="
find . -type f \( -name '*.rs' -o -name '*.toml' -o -name '*.yaml' -o -name '*.json' \) | xargs grep -l 'ADR-0006\|adr.0006' 2>/dev/null

Length of output: 1494

🏁 Script executed:

#!/bin/bash
echo "=== gatosd main.rs content ==="
cat crates/gatosd/src/main.rs

echo -e "\n=== All Rust files in crates (check sizes) ==="
find crates -type f -name '*.rs' -exec wc -l {} \; | sort -rn | head -20

Length of output: 1858

Remove or clarify aspirational commands in CONTRIBUTING.md lines 83-89; commands do not exist in implementation.

Verification confirms that git gatos watch --once, git gatos lock acquire, and the event log path ~/.config/gatos/watch/events.log are not implemented in the codebase. gatosd is a minimal stub (~45 lines) containing only a --stdio flag and an explicit TODO stating the JSONL RPC server "will be implemented in a subsequent iteration." No watch, lock, or install-hooks subcommands exist.

Lines 83-89 should either:

  • Be moved to a "Planned Features" section with explicit caveats that commands are not yet available
  • Be removed entirely until the commands are actually implemented and integrated into gatosd

The "ADR-0006 preview" prefix is insufficient to prevent developers from attempting to use these non-existent commands while following the contributing guide.

🤖 Prompt for AI Agents 
In CONTRIBUTING.md around lines 83-89, the listed commands and log path are
aspirational and not implemented; update these lines by either removing them or
moving them to a new "Planned Features" section and add an explicit caveat that
they are not available yet (e.g., reference that gatosd is currently a minimal
stub and link to the TODO/issue tracking implementation), and change the
"ADR-0006 preview" prefix to a clear note that the commands are planned but not
implemented to avoid misleading contributors.


### Docs Normalization (AST pipeline)

We run a deterministic Markdown normalizer (unified/remark) as a prebuild check. It parses Markdown to an AST, applies project transforms (link fixes, SPEC/TECH-SPEC linkification), and stringifies back. This keeps formatting/linting idempotent without touching anchors.
Expand Down
49 changes: 47 additions & 2 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ members = [
"crates/gatos-ledger-core",
"crates/gatos-ledger-git",
"crates/gatos-ledger",
"crates/gatos-mind",
"crates/gatos-message-plane",
"crates/gatos-graphql",
Comment on lines +8 to +9
Copy link

@coderabbitai coderabbitai bot Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

Verify that new crate members exist and all dependents are updated.

The workspace members have been updated correctly in structure. However, this change must be validated against the actual crate directories and all downstream dependencies.

Verify:

  1. Both new crate directories (crates/gatos-message-plane/ and crates/gatos-graphql/) exist with valid Cargo.toml files.
  2. All workspace members that previously depended on gatos-mind are now updated to depend on gatos-message-plane.

🏁 Script executed:

#!/bin/bash
# Verify new crate members exist and have valid manifests
for crate in crates/gatos-message-plane crates/gatos-graphql; do
  if [[ ! -f "$crate/Cargo.toml" ]]; then
    echo "ERROR: Missing $crate/Cargo.toml"
    exit 1
  fi
  echo "✓ $crate/Cargo.toml exists"
done

# Search for any remaining references to gatos-mind as a dependency
echo ""
echo "Searching for stray gatos-mind dependencies..."
rg -n 'gatos-mind' --type=toml crates/*/Cargo.toml

# Verify workspace members parse correctly
cd . 2>/dev/null && cargo metadata --format-version 1 >/dev/null 2>&1 && echo "✓ Workspace metadata valid" || echo "ERROR: Invalid workspace"

Length of output: 160

🏁 Script executed:

#!/bin/bash
# Inspect the new crate Cargo.toml files
echo "=== crates/gatos-message-plane/Cargo.toml ==="
cat crates/gatos-message-plane/Cargo.toml

echo ""
echo "=== crates/gatos-graphql/Cargo.toml ==="
cat crates/gatos-graphql/Cargo.toml

echo ""
echo "=== Attempting cargo check for detailed errors ==="
cargo check 2>&1 | head -50

echo ""
echo "=== Checking for circular dependency issues ==="
rg -n 'gatos-message-plane|gatos-graphql' crates/gatos-message-plane/Cargo.toml crates/gatos-graphql/Cargo.toml

Length of output: 1347

Fix inconsistent workspace metadata inheritance in crates/gatos-graphql/Cargo.toml.

The new crate directories exist and gatos-mind has been cleanly removed from all dependencies. However, crates/gatos-graphql/Cargo.toml violates workspace consistency standards:

  • crates/gatos-message-plane/Cargo.toml correctly uses workspace inheritance:

    version.workspace = true
edition.workspace = true
license.workspace = true

  • crates/gatos-graphql/Cargo.toml hardcodes these values instead:

    version = "0.1.0"
edition = "2021"
license = "Apache-2.0"

Update crates/gatos-graphql/Cargo.toml to inherit version, edition, and license from the workspace root to maintain consistency and reduce maintenance burden.

🤖 Prompt for AI Agents 
In Cargo.toml around lines 8 to 9 (crates/gatos-graphql/Cargo.toml), the crate
currently hardcodes version, edition, and license instead of inheriting
workspace metadata; change the crate's Cargo.toml to use workspace inheritance
by replacing the explicit version, edition, and license entries with the
corresponding workspace.workspace=true-style entries (i.e., set
version.workspace = true, edition.workspace = true, license.workspace = true) so
the crate picks these values from the workspace root and matches
crates/gatos-message-plane.

"crates/gatos-echo",
"crates/gatos-policy",
"crates/gatos-kv",
Expand Down
14 changes: 11 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,14 @@ git push

Store sensitive data (PII, large datasets) in private stores, but commit their **cryptographic commitments** to the public graph — public commitments; private bytes behind a policy-gated resolver. ***Verify the integrity of the computation without revealing the raw bytes***.

### 4. Local Guardrails (Watcher + Hooks)

Artists and infra engineers get Perforce-style safety without leaving Git. The `gatos watch` daemon keeps locked files read-only until a governance Grant exists, `gatos lock acquire/release` walks you through the approval flow, and managed Git hooks (`gatos install-hooks`) block bad pushes before they ever hit the remote—while logging any bypass under `refs/gatos/audit/locks/*`.

### 5. GraphQL Truth Service

Need a typed API for dashboards or custom UIs? The GraphQL endpoint (`POST /api/v1/graphql`) lets you query any state snapshot by commit (`stateRef`) or ref (`refPath`), with Relay pagination, rate limiting, and automatic policy filtering. Opaque pointers surface private blobs without leaking bytes, so you can build richly typed clients on top of verified state.

-----

## How it Works: The 5 Planes
Expand All @@ -117,7 +125,7 @@ GATOS organizes the repository into five distinct planes using standard Git refe
| **2. Policy/Trust** | `refs/gatos/policies/*` | Executable policy (Lua/WASM), capabilities, quorum; **deny-audit** on violations. |
| | `refs/gatos/trust/*` | Keys, groups, grants, revocations. |
| **3. State** | `refs/gatos/state/*` | Deterministic checkpoints derived from the ledger (**Proof-of-Fold**). |
| **4. Message** | `refs/gatos/mbus/*` | Commit-backed pub/sub (at-least-once + idempotency). |
| **4. Message** | `refs/gatos/messages/*` | Commit-backed message plane (topics served via `messages.read`). |
| **5. Job** | `refs/gatos/jobs/*` | Jobs and **Proofs-of-Execution (PoE)**; exclusive claim via CAS. |
Comment on lines +128 to 129
Copy link

@coderabbitai coderabbitai bot Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Align Message Plane terminology.

Line 128 changed the ref namespace from refs/gatos/mbus/* to refs/gatos/messages/*, but the Plane name is now just "Message" instead of "Message Plane." This is inconsistent with ADR-0005 and the broader PR terminology (see PR objectives referencing "Git-native, commit-backed Message Plane"). Update the table entry to read **4. Message Plane** for consistency.

🤖 Prompt for AI Agents 
In README.md around lines 128 to 129, the table entry for the ref namespace was
changed to use `refs/gatos/messages/*` but the Plane name was shortened to
"Message"; update the table row to use the consistent terminology "Message
Plane" per ADR-0005 and PR objectives. Edit that line so the left column reads
`**4. Message Plane**` while keeping the ref and description cells unchanged,
ensuring wording matches the rest of the document and other plane names.


-----
Expand Down Expand Up @@ -212,7 +220,7 @@ See also: Deterministic Lua profile for policies/folds: [docs/deterministic-lua.

## Contributing

🚧 GATOS is currently under construction, but you can check out the [ROADMAP](./ROADMAP.md). 🗺️
🚧 GATOS is currently under construction, but you can check out the [ROADMAP](./docs/ROADMAP.md). 🗺️

**Currently Working On:** Conceptualization & Planning Phase

Expand All @@ -238,7 +246,7 @@ See also: Deterministic Lua profile for policies/folds: [docs/deterministic-lua.
> We are looking for design partners in **scientific research**, **regulated fintech**, and **AI alignment**. If you're interested in GATOS, please get in touch. [james@flyingrobots.dev](mailto:james@flyingrobots.dev)

* [Read the Specification](./docs/SPEC.md)
* [View the Roadmap](./ROADMAP.md)
* [View the Roadmap](./docs/ROADMAP.md)
* [Join the Discussion](https://github.com/flyingrobots/gatos/discussions)

---
Expand Down
Loading
Loading