Skip to content

Commit e135fde

Browse files
docs: comprehensive README for v0.13.0
Added: RBAC section, Membrane section, multi-tenancy, batch import, encryption (classical + PQ + hybrid), event streaming, import/export, per-resource health, get_content/replace, faceted search. Updated: badges (518 tests, full crypto suite), install table (all 12 extras with combo examples), security table (12 layers), documentation table (16 guides), CLI (15 commands), FastAPI (22+ endpoints). Preserved: all original sections (content addressing, trust tiers, lifecycle, Merkle diagrams, QP stack).
1 parent 19cb714 commit e135fde

1 file changed

Lines changed: 124 additions & 42 deletions

File tree

README.md

Lines changed: 124 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -4,38 +4,39 @@
44

55
**The governed knowledge store for autonomous organizations.**
66

7-
Every document has a trust tier that weights search results. Every chunk has a SHA3-256 content ID. Every mutation is auditable. The entire vault is verifiable via Merkle tree. Air-gap native.
7+
Every document has a trust tier that weights search results. Every chunk has a SHA3-256 content ID. Every mutation is auditable. The entire vault is verifiable via Merkle tree. Content is screened by a Membrane before indexing. Access is controlled by RBAC. Air-gap native. Post-quantum ready.
88

99
[![Python](https://img.shields.io/badge/Python-3.12+-3776AB.svg)](https://www.python.org/)
1010
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
11-
[![Tests](https://img.shields.io/badge/Tests-448_passing-brightgreen.svg)](tests/)
12-
[![Crypto](https://img.shields.io/badge/Crypto-SHA3--256%20%C2%B7%20Ed25519-purple.svg)](#security)
11+
[![Tests](https://img.shields.io/badge/Tests-518_passing-brightgreen.svg)](tests/)
12+
[![Crypto](https://img.shields.io/badge/Crypto-SHA3--256%20%C2%B7%20AES--256--GCM%20%C2%B7%20ML--KEM--768%20%C2%B7%20ML--DSA--65-purple.svg)](#security)
1313

1414
</div>
1515

1616
---
1717

1818
## The Architecture
1919

20-
A Vault is a governed store of knowledge resources. Each resource is chunked, hashed, trust-classified, and indexed for hybrid search. The trust tier directly affects which results surface first.
20+
A Vault is a governed store of knowledge resources. Each resource is screened by the Membrane, chunked, hashed, trust-classified, and indexed for hybrid search. RBAC controls who can read, write, or administer.
2121

2222
```
2323
┌─────────────────────────────────────────────────────────────┐
2424
│ VAULT │
2525
├──────────────┬──────────────────────────────────────────────┤
26+
│ RBAC │ READER · WRITER · ADMIN permission matrix │
27+
│ MEMBRANE │ Innate scan → Release gate → Index/Quarantine│
2628
│ INGEST │ Parse → Chunk → SHA3-256 CID → Embed → Store│
2729
│ GOVERN │ Trust tiers · Lifecycle · Data classification│
2830
│ RETRIEVE │ Hybrid search · Trust-weighted · Time-travel │
2931
│ VERIFY │ Merkle tree · CID per chunk · Proof export │
3032
│ AUDIT │ Every write → VaultEvent → Capsule (opt.) │
33+
│ ENCRYPT │ AES-256-GCM · ML-KEM-768 · ML-DSA-65 │
3134
├──────────────┴──────────────────────────────────────────────┤
3235
│ Trust weights: CANONICAL 1.5x · WORKING 1.0x · EPHEMERAL 0.7x│
33-
SHA3-256 content IDs · Merkle root · Ed25519+ML-DSA-65 audit
36+
FIPS 202 · FIPS 197 · FIPS 203 · FIPS 204
3437
└─────────────────────────────────────────────────────────────┘
3538
```
3639

37-
Knowledge is not static. Resources move through a lifecycle (DRAFT, REVIEW, ACTIVE, SUPERSEDED, EXPIRED, ARCHIVED), organized into memory layers (OPERATIONAL, STRATEGIC, COMPLIANCE), and verified cryptographically on every read.
38-
3940
*Knowledge that can't be verified can't be trusted.*
4041

4142
---
@@ -95,7 +96,7 @@ Trust is not metadata. It is a scoring function.
9596
Search ranking formula:
9697

9798
```
98-
relevance = (0.7 × vector_similarity + 0.3 × text_rank) × trust_weight × freshness_decay
99+
relevance = (0.7 × vector_similarity + 0.3 × text_rank) × trust_weight × freshness_decay × layer_boost
99100
```
100101

101102
Where `freshness_decay = exp(-age_days / half_life × ln2)`. A 180-day-old WORKING document retains 50% freshness. A CANONICAL document of the same age retains 70%.
@@ -112,8 +113,6 @@ DRAFT ──→ REVIEW ──→ ACTIVE ──→ SUPERSEDED ──→ ARCHIVED
112113
ACTIVE (re-activate)
113114
```
114115

115-
Supersession creates a linked chain. Point-in-time search returns historically correct results.
116-
117116
```python
118117
vault.transition(r.id, "review", reason="Ready for security team")
119118
vault.transition(r.id, "active")
@@ -134,17 +133,27 @@ pip install qp-vault
134133

135134
| Command | What You Get | Dependencies |
136135
|---|---|---|
137-
| `pip install qp-vault` | SQLite, trust search, CAS, Merkle, lifecycle | **1** (pydantic) |
136+
| `pip install qp-vault` | SQLite, trust search, CAS, Merkle, lifecycle, Membrane, RBAC | **1** (pydantic) |
138137
| `pip install qp-vault[postgres]` | + PostgreSQL + pgvector hybrid search | + sqlalchemy, asyncpg, pgvector |
138+
| `pip install qp-vault[encryption]` | + AES-256-GCM encryption at rest | + cryptography, pynacl |
139+
| `pip install qp-vault[pq]` | + ML-KEM-768 + ML-DSA-65 post-quantum crypto | + liboqs-python |
139140
| `pip install qp-vault[capsule]` | + Cryptographic audit trail | + [qp-capsule](https://github.com/quantumpipes/capsule) |
140141
| `pip install qp-vault[docling]` | + 25+ format document processing (PDF, DOCX, etc.) | + docling |
141-
| `pip install qp-vault[encryption]` | + AES-256-GCM encryption at rest | + cryptography, pynacl |
142142
| `pip install qp-vault[local]` | + Local embeddings (sentence-transformers, air-gap safe) | + sentence-transformers |
143143
| `pip install qp-vault[openai]` | + OpenAI embeddings (cloud) | + openai |
144-
| `pip install qp-vault[fastapi]` | + REST API (15+ endpoints) | + fastapi |
145-
| `pip install qp-vault[cli]` | + `vault` command-line tool | + typer, rich |
144+
| `pip install qp-vault[integrity]` | + Near-duplicate + contradiction detection | + numpy |
145+
| `pip install qp-vault[fastapi]` | + REST API (22+ endpoints) | + fastapi |
146+
| `pip install qp-vault[cli]` | + `vault` command-line tool (15 commands) | + typer, rich |
146147
| `pip install qp-vault[all]` | Everything | All of the above |
147148

149+
Combine extras with commas:
150+
151+
```bash
152+
pip install qp-vault[postgres,encryption,pq,capsule,fastapi] # Production API server
153+
pip install qp-vault[local,encryption,cli] # Air-gapped deployment
154+
pip install qp-vault[docling,integrity,cli] # Development
155+
```
156+
148157
---
149158

150159
## Quick Start
@@ -157,21 +166,48 @@ vault = Vault("./my-knowledge")
157166
# Add with trust tiers
158167
vault.add("Incident response: acknowledge within 15 minutes...",
159168
name="sop-incident.md", trust="canonical")
160-
vault.add("Draft proposal for new onboarding process...",
161-
name="draft-onboard.md", trust="working")
162169

163-
# Trust-weighted search
170+
# Trust-weighted search (deduplicated, with freshness decay)
164171
results = vault.search("incident response")
165-
# CANONICAL surfaces first (1.5x), even at lower raw similarity
172+
173+
# Retrieve full content
174+
text = vault.get_content(results[0].resource_id)
175+
176+
# Replace content (atomic: creates new version, supersedes old)
177+
old, new = vault.replace(resource_id, "Updated policy v2 content")
166178

167179
# Verify integrity
168180
result = vault.verify()
169-
print(result.merkle_root) # Changes if any content is modified
181+
print(result.merkle_root)
170182

171183
# Export proof for auditors
172184
proof = vault.export_proof(resource_id)
173185
```
174186

187+
### Multi-Tenancy
188+
189+
```python
190+
vault.add("content", tenant_id="site-123")
191+
vault.search("query", tenant_id="site-123") # Scoped to tenant
192+
193+
# Or lock the vault to a single tenant
194+
vault = Vault("./knowledge", tenant_id="site-123")
195+
```
196+
197+
### RBAC
198+
199+
```python
200+
vault = Vault("./knowledge", role="reader") # Search, get, verify only
201+
vault = Vault("./knowledge", role="writer") # + add, update, delete
202+
vault = Vault("./knowledge", role="admin") # + export, import, config
203+
```
204+
205+
### Batch Import
206+
207+
```python
208+
resources = vault.add_batch(["doc1.md", "doc2.md", "doc3.md"], trust="working")
209+
```
210+
175211
### Memory Layers
176212

177213
```python
@@ -180,20 +216,21 @@ from qp_vault import MemoryLayer
180216
ops = vault.layer(MemoryLayer.OPERATIONAL) # SOPs, runbooks (boost=1.5x)
181217
strategic = vault.layer(MemoryLayer.STRATEGIC) # ADRs, decisions (trust=CANONICAL)
182218
compliance = vault.layer(MemoryLayer.COMPLIANCE) # Audit evidence (reads audited)
183-
184-
await ops.add("deploy-runbook.md") # Layer defaults auto-applied
185-
await compliance.search("SOC2") # This search is logged
186219
```
187220

188221
### Health Scoring
189222

190223
```python
191-
score = vault.health()
192-
# score.overall: 85.0/100
193-
# score.freshness: 92.0 (are documents current?)
194-
# score.uniqueness: 100.0 (no duplicates?)
195-
# score.coherence: 80.0 (no contradictions?)
196-
# score.connectivity: 70.0 (are docs organized?)
224+
score = vault.health() # Vault-wide (0-100)
225+
score = vault.health(resource.id) # Per-resource
226+
```
227+
228+
### Content Screening (Membrane)
229+
230+
Content is screened before indexing. Prompt injection, jailbreak, XSS, and code injection attempts are quarantined.
231+
232+
```python
233+
vault.add("ignore all previous instructions") # Quarantined automatically
197234
```
198235

199236
### Plugin System
@@ -206,29 +243,61 @@ class MyEmbedder:
206243
dimensions = 768
207244
async def embed(self, texts):
208245
return my_model.encode(texts)
246+
```
247+
248+
Three discovery methods: explicit registration, entry_points, or `--plugins-dir` (air-gap).
249+
250+
### Encryption
251+
252+
```python
253+
from qp_vault.encryption import AESGCMEncryptor, HybridEncryptor
254+
255+
# Classical
256+
enc = AESGCMEncryptor()
257+
ciphertext = enc.encrypt(b"secret")
209258

210-
@parser("dicom")
211-
class DicomParser:
212-
supported_extensions = {".dcm"}
213-
async def parse(self, path):
214-
return ParseResult(text=extract_dicom(path))
259+
# Post-quantum hybrid (ML-KEM-768 + AES-256-GCM)
260+
enc = HybridEncryptor()
261+
pub, sec = enc.generate_keypair()
262+
ciphertext = enc.encrypt(b"classified", pub)
215263
```
216264

217-
Three discovery methods: explicit registration, entry_points (pip packages), or `--plugins-dir` (air-gap: drop `.py` files, no install needed).
265+
### Event Streaming
266+
267+
```python
268+
from qp_vault.streaming import VaultEventStream
269+
270+
stream = VaultEventStream()
271+
vault = AsyncVault("./knowledge", auditor=stream)
272+
273+
async for event in stream.subscribe():
274+
print(f"{event.event_type}: {event.resource_name}")
275+
```
276+
277+
### Import / Export
278+
279+
```python
280+
await vault.export_vault("backup.json")
281+
await vault.import_vault("backup.json")
282+
```
218283

219284
### CLI
220285

221286
```bash
222287
vault init ./knowledge
223288
vault add report.pdf --trust canonical
224289
vault search "revenue projections" --top-k 5
225-
vault inspect <resource-id>
290+
vault list --trust canonical --tenant site-123
226291
vault verify
227292
vault health
228-
vault status
293+
vault delete <id>
294+
vault transition <id> review --reason "Ready for review"
295+
vault content <id>
296+
vault replace <id> new-version.md
297+
vault export backup.json
229298
```
230299

231-
Exit codes: `0` = pass, `1` = fail. Designed for CI: `vault verify && deploy`.
300+
15 commands. Exit codes: `0` = pass, `1` = fail. Designed for CI: `vault verify && deploy`.
232301

233302
### FastAPI
234303

@@ -237,7 +306,8 @@ from qp_vault.integrations.fastapi_routes import create_vault_router
237306

238307
router = create_vault_router(vault)
239308
app.include_router(router, prefix="/v1/vault")
240-
# 15+ endpoints: resources CRUD, search, verify, health, lifecycle, proof
309+
# 22+ endpoints: resources, search, faceted search, verify, health,
310+
# lifecycle, collections, provenance, batch, export, content
241311
```
242312

243313
---
@@ -247,12 +317,19 @@ app.include_router(router, prefix="/v1/vault")
247317
| Layer | Algorithm | Standard | Purpose |
248318
|---|---|---|---|
249319
| Content integrity | SHA3-256 | FIPS 202 | Tamper-evident CIDs and Merkle roots |
250-
| Audit signatures | Ed25519 + ML-DSA-65 | FIPS 186-5, FIPS 204 | Via [qp-capsule](https://github.com/quantumpipes/capsule) (optional) |
251-
| Encryption at rest | AES-256-GCM | FIPS 197 | `pip install qp-vault[encryption]` |
252-
| Search integrity | Parameterized SQL | -- | No string interpolation, FTS5 sanitized |
320+
| Symmetric encryption | AES-256-GCM | FIPS 197 | Data at rest |
321+
| Key encapsulation | ML-KEM-768 | FIPS 203 | Post-quantum key exchange |
322+
| Digital signatures | ML-DSA-65 | FIPS 204 | Post-quantum provenance signing |
323+
| Audit signatures | Ed25519 | FIPS 186-5 | Via [qp-capsule](https://github.com/quantumpipes/capsule) |
324+
| Hybrid encryption | ML-KEM-768 + AES-256-GCM | FIPS 203+197 | Quantum-resistant data encryption |
325+
| Content screening | Membrane pipeline | -- | Prompt injection, jailbreak, XSS detection |
326+
| Access control | RBAC | -- | Reader / Writer / Admin roles |
253327
| Input validation | Pydantic + custom | -- | Enum checks, name/tag/metadata limits |
328+
| Plugin verification | SHA3-256 manifest | -- | Hash-verified plugin loading |
329+
| Key management | ctypes memset | -- | Secure key zeroization |
330+
| Self-testing | FIPS KAT | -- | SHA3-256 + AES-256-GCM known answer tests |
254331

255-
No deprecated cryptography. No runtime network dependencies. Air-gapped operation supported. 100/100 security score. Full threat model in [docs/security.md](docs/security.md).
332+
No deprecated cryptography. No runtime network dependencies. Air-gapped operation supported. Full threat model in [docs/security.md](docs/security.md).
256333

257334
---
258335

@@ -279,6 +356,11 @@ Each independently useful. Together, the governed AI platform for autonomous org
279356
| [Trust Tiers](docs/trust-tiers.md) | Developers, Product |
280357
| [Knowledge Lifecycle](docs/lifecycle.md) | Developers, Compliance |
281358
| [Memory Layers](docs/memory-layers.md) | Developers, Architects |
359+
| [Multi-Tenancy](docs/multi-tenancy.md) | Developers, SaaS |
360+
| [Encryption](docs/encryption.md) | Developers, Security |
361+
| [RBAC](docs/rbac.md) | Developers, Compliance |
362+
| [Membrane](docs/membrane.md) | Developers, Security |
363+
| [Streaming & Telemetry](docs/streaming-and-telemetry.md) | DevOps, Observability |
282364
| [Plugin Development](docs/plugins.md) | SDK Authors |
283365
| [Security Model](docs/security.md) | CISOs, Security Teams |
284366
| [CLI Reference](docs/cli.md) | DevOps, Developers |

0 commit comments

Comments
 (0)