Implement Valkey/Redis exporter and tests for graph nodes & Appearance-Based ReID Tracking#5
Merged
bimantoromaesa merged 2 commits intomainfrom Mar 9, 2026
Merged
Conversation
…ID Tracking. - Implement comprehensive tests for the Valkey/Redis exporter, covering export and load functionalities for various result types including VisionResult, ClassifyResult, DepthResult, and OCRResult. - Introduce tests for ValkeyStore and ValkeyLoad graph nodes, ensuring correct artifact handling, key templating, and integration within graph execution. - Validate serialization and deserialization processes, including support for custom serializers and TTL parameters. - Ensure robust error handling for missing keys and unsupported result types.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
v1.9.2 Beta — Valkey/Redis Graph Pipeline Storage + Appearance-Based ReID Tracking
Valkey / Redis Graph Pipeline Storage
export_valkey(result, url, key, ttl, serializer)— serializes any MATA result type to a Valkey or Redis key with optional TTL; supportsjson(default) andmsgpackserializersload_valkey(url, key, result_type="auto")— deserializes stored results back to their original type; auto-detectsVisionResult,ClassifyResult,DepthResult, andOCRResultpublish_valkey(result, url, channel, serializer)— fire-and-forget Pub/Sub broadcast_parse_valkey_uri()helper supportingvalkey://,valkey://<db>/, andredis://user:pass@host:port/db/keyURI formatsValkeyStoregraph sink node — pass-through sink with{node}/{timestamp}key template placeholders, TTL, and serializer selectionValkeyLoadgraph source node —inputs={}source that loads a stored result from Valkey and injects it as a typed artifactvalkey://andredis://URI scheme dispatch to all sixresult.save()methods (VisionResult,DetectResult,SegmentResult,ClassifyResult,DepthResult,OCRResult) — existing file-based paths fully unaffectedModelRegistry.get_valkey_connection(name="default")— reads named connection profiles fromstorage.valkeyYAML section; resolvespassword_envfrom environment variablesmata[valkey]→valkey>=6.0.0;mata[redis]→redis>=5.0.0export_valkey,load_valkey,publish_valkeyfrommata.core.exportersValkeyStore,ValkeyLoadfrommata.nodestest_valkey_exporter.py(42),test_valkey_nodes.py(33),test_valkey_config.py(14)Appearance-Based ReID Tracking
mata.track(..., reid_model="org/model")— activates BotSort appearance re-identification via any HuggingFace encoder ID or local.onnxpathReIDAdapterabstract base class — L2-normalised embedding output; lazy-loaded (zero startup overhead when unused)HuggingFaceReIDAdapter— auto-detects CLIP / ViT-family / generic AutoModel architectures; alltransformersimports lazyONNXReIDAdapter— auto-detects NCHW/NHWC input layout from model metadata; supports CPU and CUDA execution providersTrackingAdapter.update()to extract crops, batch-encode through the ReID encoder, and inject embeddings into BOTSORT;Instance.embeddingpopulated in outputVisionResultmata.track()withreid_model: str | None,with_reid: bool = False, andreid_bridgekwargs (backward-compatible defaults)ReIDBridge— cross-camera appearance store backed by Valkey/Redis; publishes L2-normalised embeddings keyed byreid:{camera_id}:{track_id};query()returns nearest cosine-similarity matches from other cameras; TTL-based auto-eviction;msgpackbinary serialisation;scan_iter(non-blocking)TrackingAdapter.__init__()withreid_encoderandreid_bridgekwargs; confirmed-track embeddings published automatically after eachupdate();ConnectionErrorcaught and logged, never raisedReIDAdapter,HuggingFaceReIDAdapter,ONNXReIDAdapterfrommata.adaptersReIDBridgefrommata.trackerstest_reid_adapter.py,test_tracking_reid.py,test_reid_bridge.pyexamples/track/reid_tracking.pyandexamples/track/cross_camera_reid.pyDocumentation
docs/VALKEY_GUIDE.md— full integration guide: installation, basic usage, graph pipeline, YAML config, streaming, Pub/Sub, security (TLS,password_env, SSRF prevention, key sanitization), performance tuning, and troubleshootingdocs/GRAPH_API_REFERENCE.md— new "Storage Nodes" section with parameter tables forValkeyStoreandValkeyLoaddocs/VALIDATION_GUIDE.md— ReID tracking validation notes addedREADME.md— Valkey added to Key Features and Optional Dependencies; ReID tracking section with single-camera and cross-camera examplesQUICKSTART.md— new "Valkey / Redis Result Storage" sectionQUICK_REFERENCE.md— new "Valkey/Redis Storage Quick Reference (v1.9)" cheatsheetChanged
mata.nodes.__all__extended withValkeyStoreandValkeyLoadmata.core.exporters.__init__extended withexport_valkey,load_valkey,publish_valkeymata.track()signature extended withreid_model,with_reid,reid_bridgekwargsTrackingAdapter.__init__()extended withreid_encoder,reid_bridgekwargs (both defaultNone; zero overhead when unused)BOTSORT.get_dists()appearance-distance branch now reachable whenreid_encoderis setsrc/mata/trackers/configs/botsort.yaml—reid_model/with_reiddocumentation block addedWhy
Closes two major feature tracks planned for v1.9.x:
Valkey/Redis storage — enables MATA graph pipelines to persist and share results between nodes, services, and processes without writing intermediate files; supports both single-process caching and cross-service Pub/Sub streaming patterns.
Appearance-Based ReID — extends BotSort tracking with embedding-based identity matching, dramatically reducing ID switches in occlusion-heavy scenes;
ReIDBridgeenables cross-camera identity correlation backed by the same Valkey infrastructure added in this release.Checklist
pytest tests/ -v)black --check src/ tests/ && ruff check src/)mypy src/mata/)