feat: add context and task documentation for hackathon participation

Author: mnm-matin

.gitignore

@@ -63,7 +63,6 @@ scripts_ignored/
 
 # AI Context (Agent files)
 .claude/
-context/
 CLAUDE.md
 TASKS.md
 TESTS.md

context/BRIEF.md

@@ -0,0 +1,3 @@
+- AGENT Convos are in: .specstory
+- nuscens_fixture md file has the 
+- 

context/TASK.md

@@ -0,0 +1,36 @@
+We want to participate in the hackathon (we are already registered) by nvidia: https://luma.com/nvidia-cosmos-cookoff
+
+
+- in that page they mention the following:
+​Build something great with Cosmos Reason 2—post-train for a specialized reasoning model, a data curation tool, a robot brain that understands a new domain, or a video analytics agent. 
+
+- in the huggingface of cosmos reason they mention the following:
+Data curation and annotation — Enable developers to automate high-quality curation and annotation of massive, diverse training datasets. Experience NVIDIA Cosmos Curator, powered by Cosmos Reason, a framework that enables developers to quickly filter, annotate, and deduplicate large amounts of sensor data necessary for physical AI development.
+
+- they also mention their own data curation pipeline
+this is the link: https://github.com/nvidia-cosmos/cosmos-curate (git cloned in ./cosmos-curate)
+
+- another interesting thing is the video-search and summarization: https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization
+
+- we build an opensource project called hyperview which is a dataset curation tool
+you can find it in ../HyperView (or https://github.com/Hyper3Labs/HyperView)
+- in thi repo nvidia-hyper, we have hard forked HyperView to align it with the hackathon.
+
+- if necessary we have access to a slurm cluster with some H100s, if needed, let me know
+
+
+- I had a chance to ask them some questions:
+
+Hi, thanks for offering this opportunity. Just had a 2 questions:
+ 
+i) In the original description of the cookoff it was mentioned that a data curation tool could be something cool to build. can you elaborate on the data curation tool and what you had in mind regarding that. is it okay if we use cosmos-curate to do some parts of the curation?
+
+ii) we would like to visualize the embeddings, is there a Cosmos 2 Embedding model? we are currently using Cosmos-Emebd1
+
+thanks again!
+
+
+- this was their response:
+
+1) You can use Cosmos Reason 2 as a component in your data processing pipeline. Cosmos reason is a VLM that can help you filter, label or annotate data. https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/reason2/video_caption_vqa/post_training.html
+2)On top of it then, you can use Cosmos Curate and/or Cosmos Embed.

context/nuscenes_fixture_from_fresh_clone.md

@@ -0,0 +1,231 @@
+# nuScenes Fixture Setup (Fresh Clone, Local Testing Only)
+
+## Important: this is a fixture workflow
+
+This guide creates a **local testing fixture** for HyperView.
+
+- It is for **UI/API smoke testing and iteration**.
+- It is **not** a real Cosmos-Curate production run.
+- Captions/embeddings in the fixture are synthetic (generated by `scripts/create_nuscenes_fixture.py`).
+- Videos are built from local nuScenes camera frames to avoid solid-color test clips.
+
+## Prerequisites
+
+- macOS/Linux shell
+- `uv` installed
+- `node` + `npm` installed
+- `ffmpeg` installed (for frame->mp4 clip generation)
+- nuScenes mini data available under `~/nuscenes`
+  - Must include:
+    - `~/nuscenes/v1.0-mini`
+    - `~/nuscenes/samples`
+    - `~/nuscenes/sweeps`
+
+Check quickly:
+
+```bash
+ls -d ~/nuscenes/v1.0-mini ~/nuscenes/samples ~/nuscenes/sweeps
+```
+
+If `ffmpeg` is missing on macOS:
+
+```bash
+brew install ffmpeg
+```
+
+## 1) Fresh clone + dependencies
+
+```bash
+git clone <YOUR_FORK_OR_REPO_URL> nvidia-hyper
+cd nvidia-hyper
+
+uv sync
+cd frontend
+npm install
+cd ..
+```
+
+## 2) Build nuScenes source clips (real image frames -> mp4)
+
+This creates `48` short clips from `~/nuscenes/sweeps/CAM_FRONT`:
+
+```bash
+uv run python - <<'PY'
+import glob
+import os
+import shutil
+import subprocess
+from pathlib import Path
+
+src_dir = Path.home() / 'nuscenes' / 'sweeps' / 'CAM_FRONT'
+out_dir = Path('/private/tmp/nuscenes_source_clips')
+work_root = Path('/private/tmp/nuscenes_frames_work')
+
+num_clips = 48
+frames_per_clip = 20
+stride = 12
+fps = 10
+
+if not src_dir.exists():
+    raise SystemExit(f'Missing source dir: {src_dir}')
+
+frames = sorted(glob.glob(str(src_dir / '*.jpg')))
+if len(frames) < frames_per_clip:
+    raise SystemExit(f'Not enough frames: {len(frames)}')
+
+out_dir.mkdir(parents=True, exist_ok=True)
+work_root.mkdir(parents=True, exist_ok=True)
+
+for old in out_dir.glob('*.mp4'):
+    old.unlink()
+
+max_possible = 1 + (len(frames) - frames_per_clip) // stride
+clips_to_make = min(num_clips, max_possible)
+
+for clip_idx in range(clips_to_make):
+    start = clip_idx * stride
+    chunk = frames[start:start + frames_per_clip]
+    if len(chunk) < frames_per_clip:
+        break
+
+    clip_work = work_root / f'clip_{clip_idx + 1:03d}'
+    if clip_work.exists():
+        shutil.rmtree(clip_work)
+    clip_work.mkdir(parents=True, exist_ok=True)
+
+    for i, src in enumerate(chunk, start=1):
+        dst = clip_work / f'{i:04d}.jpg'
+        os.symlink(src, dst)
+
+    out_mp4 = out_dir / f'clip_{clip_idx + 1:03d}.mp4'
+    subprocess.run(
+        [
+            'ffmpeg', '-y', '-v', 'error',
+            '-framerate', str(fps),
+            '-i', str(clip_work / '%04d.jpg'),
+            '-vf', 'scale=640:360:force_original_aspect_ratio=decrease,pad=640:360:(ow-iw)/2:(oh-ih)/2',
+            '-c:v', 'libx264', '-pix_fmt', 'yuv420p',
+            str(out_mp4),
+        ],
+        check=True,
+    )
+
+print(f'Created {len(list(out_dir.glob("*.mp4")))} clips in {out_dir}')
+PY
+```
+
+## 3) Create the full fixture artifacts
+
+This creates:
+
+- `/private/tmp/nuscenes_fixture_real/split_output`
+- `/private/tmp/nuscenes_fixture_real/dedup_output`
+
+```bash
+uv run python scripts/create_nuscenes_fixture.py \
+  --source-clips-path /private/tmp/nuscenes_source_clips \
+  --output-root /private/tmp/nuscenes_fixture_real \
+  --num-clips 48 \
+  --embedding-dim 256
+```
+
+## 4) Run backend (fixed port `6263`)
+
+```bash
+lsof -ti tcp:6263 | xargs -r kill -9
+
+uv run python scripts/load_cosmos_curate.py \
+  --split-output-path /private/tmp/nuscenes_fixture_real/split_output \
+  --dataset-name nuscenes_fixture_real_live \
+  --no-persist \
+  --no-browser \
+  --port 6263
+```
+
+## 5) Run frontend (fixed port `3001`)
+
+Open a second terminal:
+
+```bash
+cd frontend
+PORT=3001 npm run dev -- --webpack
+```
+
+Open:
+
+- `http://127.0.0.1:3001`
+
+## 6) Smoke-check that video serving works
+
+In another terminal:
+
+```bash
+uv run python - <<'PY'
+import json
+import os
+import urllib.request
+
+base = 'http://127.0.0.1:6263'
+ds = json.load(urllib.request.urlopen(base + '/api/dataset'))
+print('dataset=', ds.get('name'))
+
+samples = json.load(urllib.request.urlopen(base + '/api/samples?offset=0&limit=1')).get('samples', [])
+if not samples:
+    raise SystemExit('No samples loaded')
+
+sid = samples[0]['id']
+video = urllib.request.urlopen(base + f'/api/video/{sid}')
+print('video_status=', video.status, 'content_type=', video.headers.get('Content-Type'))
+
+all_samples = json.load(urllib.request.urlopen(base + '/api/samples?offset=0&limit=500')).get('samples', [])
+missing = 0
+for sample in all_samples:
+    metadata = sample.get('metadata') or {}
+    path = metadata.get('video_path') or sample.get('filepath') or ''
+    ok = isinstance(path, str) and path.endswith('.mp4') and os.path.exists(path)
+    if not ok:
+        missing += 1
+print('total=', len(all_samples), 'missing_video_paths=', missing)
+PY
+```
+
+Expected:
+
+- `dataset= nuscenes_fixture_real_live`
+- `video_status= 200`
+- `content_type= video/mp4`
+- `missing_video_paths= 0`
+
+## Troubleshooting
+
+### I still see solid-color videos
+
+Most common causes:
+
+1. Backend still points to old synthetic fixture (example: `/private/tmp/cookoff_fixture_256`).
+2. Browser has stale frontend state/cache.
+3. Another backend process is running on `6263`.
+
+Fix:
+
+```bash
+lsof -ti tcp:6263 | xargs -r kill -9
+```
+
+Then restart backend with:
+
+```bash
+uv run python scripts/load_cosmos_curate.py \
+  --split-output-path /private/tmp/nuscenes_fixture_real/split_output \
+  --dataset-name nuscenes_fixture_real_live \
+  --no-persist \
+  --no-browser \
+  --port 6263
+```
+
+Then hard refresh frontend (`Cmd+Shift+R`).
+
+### I only have `v1.0-mini` JSON metadata, no images
+
+You need the nuScenes media folders (`samples/`, `sweeps/`) under `~/nuscenes`.
+Without those, clip generation cannot run.

scripts/demo.py

@@ -0,0 +1,84 @@
+#!/usr/bin/env python3
+"""Run HyperView demo with CIFAR-10 dataset."""
+
+import argparse
+import os
+import sys
+from pathlib import Path
+
+# Add src to path for development
+sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Run HyperView demo")
+    parser.add_argument(
+        "--dataset",
+        type=str,
+        default="cifar10_demo",
+        help="Dataset name to use for persistence (default: cifar10_demo)",
+    )
+    parser.add_argument(
+        "--samples", type=int, default=50000, help="Number of samples to load (default: 50000)"
+    )
+    parser.add_argument(
+        "--port", type=int, default=6263, help="Port to run server on (default: 6263)"
+    )
+    parser.add_argument(
+        "--no-browser", action="store_true", help="Don't open browser automatically"
+    )
+    parser.add_argument(
+        "--no-persist", action="store_true", help="Don't persist to database (use in-memory)"
+    )
+    parser.add_argument(
+        "--model",
+        type=str,
+        default="openai/clip-vit-base-patch32",
+        help=(
+            "Embedding model_id to use (default: openai/clip-vit-base-patch32). "
+            "This is passed to Dataset.compute_embeddings(model=...)."
+        ),
+    )
+    parser.add_argument(
+        "--datasets-dir",
+        "--database-dir",
+        type=str,
+        default=None,
+        help="Override persistence directory (sets HYPERVIEW_DATASETS_DIR)",
+    )
+    parser.add_argument(
+        "--no-server",
+        action="store_true",
+        help="Don't start the web server (useful for CI / DB checks)",
+    )
+    args = parser.parse_args()
+
+    if args.datasets_dir:
+        os.environ["HYPERVIEW_DATASETS_DIR"] = args.datasets_dir
+
+    import hyperview as hv
+
+    dataset = hv.Dataset(args.dataset, persist=not args.no_persist)
+
+    dataset.add_from_huggingface(
+        "uoft-cs/cifar10",
+        split="train",
+        image_key="img",
+        label_key="label",
+        max_samples=args.samples,
+    )
+
+    space_key = dataset.compute_embeddings(model=args.model, show_progress=True)
+
+    # Compute a single layout for the UI to display by default.
+    # Switch to geometry="euclidean" for standard 2D UMAP.
+    dataset.compute_visualization(space_key=space_key, geometry="poincare")
+
+    if args.no_server:
+        return
+
+    hv.launch(dataset, port=args.port, open_browser=not args.no_browser)
+
+
+if __name__ == "__main__":
+    main()