Epic: Trained DensePose Model with RuVector Signal Intelligence + RVF Container

[ruvnet]

Trained DensePose Model with RuVector Signal Intelligence Pipeline

What This Is

A complete pipeline for training, packaging, and deploying a real trained neural network that converts WiFi CSI signals into dense human body surface estimation (DensePose). This replaces the current heuristic pose derivation with a model trained on public research datasets, enhanced by RuVector signal processing algorithms, and packaged in the RVF binary container format for single-file deployment.

Why It Matters

The current system does WiFi sensing (presence, motion detection) and derives approximate pose keypoints from signal heuristics. That works, but it is not a trained model -- the keypoint positions are rule-based, not learned from data. The CMU "DensePose From WiFi" paper showed that a neural network trained on paired WiFi + camera data can produce accurate body surface UV coordinates from WiFi alone. This epic implements that capability using RuVector signal processing crates as the backbone.

What You Get

Capability	Current (Heuristic)	After Training
Keypoint count	17 (approximate)	17 (learned, accurate)
Body surface UV	None	24 body parts + UV coordinates
Accuracy (PCK@0.2)	Not measured	Target >70%
Environment adaptation	None	SONA LoRA in <5 seconds
Deployment	ONNX/binary	Single .rvf file
Browser inference	Not possible	WASM in .rvf container

---

Architecture

ESP32 CSI (UDP :5005)
    |
    v
RuVector Signal Intelligence (5 crates)
|- ruvector-attn-mincut    -> noise-suppressed spectrogram
|- ruvector-mincut         -> sensitive/insensitive subcarrier partition
|- ruvector-attention      -> body velocity profile extraction
|- ruvector-solver         -> Fresnel TX-body-RX distance estimation
'- ruvector-temporal-tensor -> compressed temporal buffering
    |
    v
Trained Neural Network (6 additional crates)
|- ruvector-graph-transformer -> CSI-to-pose cross-attention
|- ruvector-gnn               -> body skeleton graph reasoning
|- ruvector-sparse-inference  -> PowerInfer edge deployment
|- ruvector-sona              -> LoRA + EWC++ online adaptation
|- ruvector-math              -> optimal transport loss
'- ruvector-fpga-transformer  -> hardware acceleration path
    |
    v
Output: 17 keypoints + 25 body parts + 48 UV channels + confidence
    |
    v
Packaged as wifi-densepose-v1.rvf (single deployable file)

---

RVF Container Format

The trained model ships as a single .rvf file containing everything needed for inference:

Segment	Type	Contents	Purpose
Manifest	0x05	Model ID, dataset provenance, segment directory	Container metadata
Vec	0x01	~5M model weight parameters	Neural network weights
Index	0x02	HNSW layers A/B/C over weight partitions	Sparse inference routing
Overlay	0x03	Subcarrier, antenna topology, body skeleton graphs	Pre-computed min-cut structures
Quant	0x06	INT8/FP16 codebooks, calibration stats	Weight quantization
Witness	0x0A	Training proof, Ed25519 signature, metrics	Verifiable provenance
Meta	0x07	COCO keypoints, body part labels, normalization	Model metadata
AggregateWeights	0x36	Per-environment SONA LoRA deltas	Adaptation profiles
Profile	0x0B	Input/output specs, hardware requirements	Domain declaration
Crypto	0x0C	Ed25519 public key, signature chain	Integrity verification
Wasm	0x10	WASM inference engine	Browser deployment
Dashboard	0x11	Three.js visualization UI	Embedded web UI

Progressive loading: Layer A loads in <5ms (instant startup), full accuracy in ~500ms.

---

Training Data Strategy

Source	Subcarriers	Labels	Volume	When
MM-Fi (NeurIPS 2023)	114 -> 56	17 COCO + DensePose UV	40 subjects, 320K frames	Phase 1 (bootstrap)
Wi-Pose (NjtechCVLab)	30 -> 56	18 keypoints	12 subjects, 166K packets	Phase 1 (diversity)
ESP32 self-collected	56 (native)	Camera teacher labels	Unlimited	Phase 5+ (fine-tuning)

Pre-train on public data -> fine-tune on your own ESP32 data -> continuous SONA adaptation at runtime.

---

RuVector Crates Used (11 total)

Signal Processing (5, already integrated)

Crate	Version	Algorithm	Pipeline Role
ruvector-attn-mincut	2.0.4	Attention-gated min-cut	Noise-suppressed spectrogram
ruvector-mincut	2.0.4	Subpolynomial dynamic min-cut	Subcarrier partitioning
ruvector-attention	2.0.4	Scaled dot-product attention	Body velocity profile
ruvector-solver	2.0.4	Sparse Neumann solver O(sqrt n)	Fresnel geometry + subcarrier resampling
ruvector-temporal-tensor	2.0.4	Tiered temporal compression	CSI frame buffering

Neural Network (6, newly integrated)

Crate	Version	Algorithm	Pipeline Role
ruvector-graph-transformer	2.0.4	Proof-gated graph transformer	CSI->pose cross-attention bottleneck
ruvector-gnn	2.0.4	GNN on HNSW topology	Body skeleton constraint enforcement
ruvector-sparse-inference	2.0.4	PowerInfer-style sparsity	Edge deployment (<50ms on ARM)
ruvector-sona	0.1.6	LoRA + EWC++	Online environment adaptation
ruvector-math	2.0.4	Optimal transport	Wasserstein regularization loss
ruvector-fpga-transformer	2.0.4	FPGA-optimized transformer	Hardware acceleration path

RVF Container (8 subcrates)

Crate	Purpose
rvf-types	Segment headers, type discriminators, flags
rvf-wire	Binary serialization/deserialization
rvf-manifest	Level-1 manifest and segment directory
rvf-index	HNSW progressive layers A/B/C
rvf-quant	Temperature-tiered quantization (f32/f16/u8/binary)
rvf-crypto	Ed25519 signatures, attestation
rvf-runtime	Container loading and progressive startup
rvf-adapter-sona	SONA LoRA delta serialization

---

Performance Targets

Metric	Target
PCK@0.2 (keypoint accuracy)	>70% on MM-Fi validation
OKS mAP (COCO standard)	>0.50
DensePose GPS (UV accuracy)	>0.30
Inference latency (x86)	<10ms per frame
Inference latency (ARM)	<50ms per frame
RVF container startup (Layer A)	<5ms
RVF full load	<500ms
SONA adaptation	<50 gradient steps, <5 seconds
Model size (FP16)	<25MB
Model size (INT8)	<12MB
RVF container size (with WASM+UI)	<30MB

---

Implementation Phases

Phase 1: Dataset Loaders (2 weeks)

• MmFiDataset loader -- read .npy, resample 114->56 via ruvector-solver
• WiPoseDataset loader -- read .mat, zero-pad 30->56
• Phase sanitization via wifi-densepose-signal
• Temporal windowing with ruvector-temporal-tensor
• Unit tests for data loading pipeline

Phase 2: Graph Transformer Integration (2 weeks)

• Add ruvector-graph-transformer to ModalityTranslator bottleneck
• Build antenna topology graph (nodes = antenna pairs, edges = spatial proximity)
• Add ruvector-gnn body graph reasoning (17 nodes, 16 anatomical edges)
• GNN message passing with anatomical constraint enforcement
• Forward pass shape validation tests

Phase 3: Teacher-Student Label Generation (1 week)

• Python script: Detectron2 DensePose -> UV pseudo-labels from MM-Fi RGB
• Cache labels as .npy for Rust loader
• Visual validation on random subset

Phase 4: Training Loop (3 weeks)

• WiFiDensePoseTrainer with 6-term loss function
• Optimal transport loss via ruvector-math
• GNN edge consistency loss
• Cosine LR schedule, early stopping, checkpointing
• PCK@0.2, OKS mAP, DensePose GPS validation metrics
• Deterministic proof verification with weight hash
• Achieve PCK@0.2 >70% on MM-Fi validation

Phase 5: SONA Online Adaptation (2 weeks)

• Integrate ruvector-sona LoRA injection
• EWC++ Fisher information regularization
• Self-supervised temporal consistency loss
• Camera calibration mode (5-minute supervised session)
• CSI baseline drift detection
• Convergence in <50 gradient steps

Phase 6: Sparse Inference + Edge Deployment (2 weeks)

• Profile neuron activation frequencies
• ruvector-sparse-inference hot/cold partitioning
• INT8 backbone, FP16 heads quantization
• WASM export via ruvector-sparse-inference-wasm
• Benchmark: <10ms x86, <50ms ARM

Phase 7: RVF Container Build Pipeline (2 weeks)

• build-rvf binary -- serialize all segments
• Vec segment: model weight embeddings
• Index segment: HNSW layers A/B/C for sparse routing
• Overlay segment: min-cut graphs (subcarrier, antenna, body)
• Quant segment: quantization codebooks
• Witness segment: training proof + Ed25519 signature
• AggregateWeights segment: SONA LoRA deltas
• Wasm segment: embedded inference runtime
• Dashboard segment: Three.js UI
• verify-rvf binary -- validate container integrity

Phase 8: Sensing Server Integration (1 week)

• Load .rvf container via rvf-runtime
• Progressive loading (Layer A -> instant startup)
• Replace heuristic pose with trained model inference
• --model wifi-densepose-v1.rvf CLI flag
• --ui-from-rvf serve embedded Dashboard segment
• Apply SONA profile via --env office-3f
• Fallback to heuristic when no model present
• DensePose UV data in WebSocket protocol

---

ADR Reference

Full technical details: docs/adr/ADR-023-trained-densepose-model-ruvector-pipeline.md

Related Issues

• Release: Rust Sensing Server v0.1.0 — Full DensePose-Compatible API #41 -- Rust Sensing Server v0.1.0 Release
• ⭐ Tutorial: ESP32-S3 CSI Pipeline End-to-End Setup (ADR-018) #34 -- ESP32-S3 CSI Pipeline Tutorial
• ⭐ Tutorial: Windows WiFi Sensing Quick Start (ADR-013) #36 -- Windows WiFi Sensing Tutorial

Related ADRs

• ADR-005: SONA Self-Learning for Pose Estimation
• ADR-015: Public Dataset Strategy (MM-Fi + Wi-Pose)
• ADR-016: RuVector Integration (5 crates)
• ADR-020: Rust AI/Model Migration
• ADR-021: Vital Sign Detection (RVDNA pipeline)