NeuroReach VR is a pass-through mixed-reality (MR) prototype built in Unity (C#) for Meta Quest 3S with Logitech MX Ink stylus support. It provides two short, repeatable upper-limb exercises designed around clinically meaningful primitives (reaching/grasping and tracing) and prioritizes simple, clinician-readable metrics exported as CSV for spreadsheet review.
Scope note: This project is a proof-of-concept prototype. It is intended to support structured practice and logging, not to make diagnostic or clinical-efficacy claims.
- System overview
- Developer setup & Quest build
- Tasks
- Difficulty modes (Easy vs Hard)
- Scoring
- What gets logged
- Metrics glossary
- How to use the application
- Data export (CSV)
- Implementation notes (high level)
- Known limitations
Hardware / runtime
- Headset: Meta Quest 3S (pass-through MR)
- Input devices:
- Logitech MX Ink stylus (6DoF) for PathTrace, including vibration (haptics)
- Runtime: Unity on Android (Quest), using OpenXR device access.
Design goals
- Keep users grounded in the real environment (pass-through MR), reduce disorientation and discomfort.
- Provide tasks that are short, repeatable, and easy to administer.
- Provide immediate feedback (visual urgency + haptics) and auditable outputs (CSV logs with interpretable metrics).
Goal: Train repeated reaching and grasping in peri-personal space under time pressure.
Interaction
- Balloons spawn in a forward “reach dome” in front of the user.
- A balloon is popped via a stylus-tracked motion, in which you are required to prick the balloon with the stylus.
Urgency cue
- Each balloon has a lifetime T.
- Balloon color shifts blue → red from spawn to expiry, so urgency is visible without needing text UI.
What this task trains (practical interpretation)
- Repetition count (number of successful reaches)
- Movement initiation under time pressure (misses / slower reaction times)
- Throughput over a block (pops per minute)
Goal: Train fine motor control, steady stylus motion, and visuomotor error correction.
Interaction
- A target silhouette (e.g., line/polygon/circle/spiral) is shown on a stable drawing plane anchored in front of the user (aligned to a table surface when available).
- The user traces the target using the MX Ink stylus.
Tolerance corridor + haptic error feedback
- The target is represented as a polyline.
- A tolerance corridor of half-width τ (in mm) is defined around the target.
- At runtime we compute the shortest distance d from the stylus tip to the target polyline:
- If d > τ, the stylus vibrates in pulses until the trace returns inside the corridor (d ≤ τ).
- This provides a binary, easy-to-interpret error signal without requiring reading text.
What this task trains (practical interpretation)
- Accuracy (fraction of trace inside corridor / overlap)
- Speed–accuracy trade-offs (time vs overlap)
- Error frequency and stability (out-of-bounds events and time)
The prototype exposes two clinician-selectable profiles per task. Parameters are intentionally visible and adjustable rather than “opaque” adaptive difficulty.
- Easy: lifetime L = 10 s, spawn radius r ∈ [0.35, 0.55] m
- Hard: lifetime L = 5 s, spawn radius r ∈ [0.45, 0.75] m
- Primary knob: corridor half-width τ
- Easy: τ = 20 mm
- Hard: τ = 8 mm
- Secondary knobs:
- optional time limit T for the trial (selectable before start)
- path complexity:
- Easy: simpler shapes (e.g., lines, polygons)
- Hard: more complex curves (e.g., circles, spirals)
Scores are designed to be simple, interpretable, and aligned with the feedback mechanisms.
Let:
- T = balloon lifetime (seconds)
- t_p = time since spawn when the balloon is popped (seconds)
- S_max = max score per balloon (default: 10)
Normalized score:
s = max(0, 1 - t_p / T)
Points added for that balloon:
ΔScore = S_max * s
- Fast pops → higher score
- Expired balloons → add 0
Let:
- L = target path length
- l_trace = traced length credited as “covered” (progress along the path)
- S_max = max score per stage (default: 10)
Normalized score:
s = min(1, l_trace / L)
Points added:
ΔScore = S_max * s
- Full completion / overlap → higher score
- Traces outside the corridor reduce credited progress, and haptics highlight errors.
NeuroReach VR is designed around clinician-readable outputs:
- No specialized signal processing required
- Metrics emphasize counts, time, and accuracy/overlap
- Balloons spawned
- Balloons popped
- Balloons expired (missed)
- Mean reaction time (t_p)
- Pop rate (pops per minute)
- Total score
- Completion time (with flags for completed vs timed out)
- Overlap (% samples within corridor)
- Total out-of-corridor duration
- Out-of-bounds events (debounced count)
- Total score (progress/completion)
The table below mirrors the intended “how a clinician can use it” interpretation.
| Task | Metric | What it measures (computed) | Practical interpretation |
|---|---|---|---|
| BalloonPop | Balloons popped | Count of successful pops in a block | Repetition count (completed reaches) |
| Balloons missed | Count of expired balloons | Time-pressure sensitivity / slower initiation | |
| Pop rate | Pops per minute over block duration | Throughput/efficiency over time | |
| Score | Σ Smax * max(0, 1 - tp / T) | Rewards faster responses under time pressure | |
| PathTrace | Overlap (%) | Fraction of samples within corridor | Accuracy / fine motor control summary |
| Trace time | Time to completion or timeout | Speed/efficiency; compare against overlap | |
| Out-of-bounds events | Debounced corridor-exit count | Error frequency / subtle instability | |
| Score | Σ Smax * min(1, ltrace / L) | Completion/progress incentive |
A typical session flow (therapist/clinician administered):
- Start NeuroReach VR (pass-through MR).
- In the clinician menu, select:
- Patient ID
- Task order (BalloonPop or PathTrace)
- Difficulty (Easy / Hard)
- Duration / time limit (block duration for BalloonPop; optional time limit for PathTrace)
- Patient completes the selected task(s).
- After each block/trial, the application provides:
- Score
- Core performance indicators (counts/time/overlap) in the saved logs
- Repeat blocks as needed for practice.
- Export/retrieve CSV logs for review.
The system exports per-trial summary metrics in CSV format for direct spreadsheet review.
Export contents:
- One row per trial including:
- timestamp
- Patient ID
- task type
- difficulty profile
- duration / time limit
- task-specific metrics (listed above)
Additionally, an event-level log may also be produced, capturing events such as:
- BalloonPop: spawn, pop, expiry
- PathTrace: trace start, trace end, timeout
Retrieval: Logs are stored in the application’s persistent storage on the Quest device. Use Quest Developer Hub or ADB to pull the CSV files from the headset for analysis.
-
Anchoring / coordinate frames
- A local task frame is defined relative to the user’s head pose at trial start.
- Balloon positions are sampled within this frame to keep reach distances consistent.
- PathTrace spawns a stable drawing plane at a fixed offset in front of the user to reduce drift effects during a trial.
-
Haptics
- Stylus vibration is triggered on corridor exit (d > τ).
- Pulses are debounced to avoid constant buzzing and to keep error feedback perceivable.
-
Parameterization
- Key parameters (lifetimes, spawn radii, corridor widths, time limits) are stored in a JSON config file loaded at startup.
- The initial evaluation was a non-patient feasibility pilot (healthy participants); no clinical-efficacy claims are made.
- Participants reported occasional spatial drift of virtual elements (a priority for future iterations).
- Some users requested clearer onboarding and stronger completion feedback (sound/visual confirmations).