Unity • Meta Quest 3 • Inverse3 Haptic Device
A portfolio showcase of my Multimodal VR Simulation with Inverse3 Haptic Device, a medical injection training simulation built with Unity, Meta Quest 3, and the Inverse3 haptic device for the FMH Media Lab at the University of Sydney. Includes documentation, screenshots, and demo videos (no proprietary code).
Training in medical procedures such as needle insertions often relies on limited physical resources and direct supervision, which can restrict practice opportunities and increase the risk of errors during real clinical settings. To address this challenge, the Multimodal VR Simulation with Inverse3 Haptic Device project seeks to create a virtual reality-based training platform that realistically simulates the process of inserting a needle into a patient’s arm. The project leverages immersive technologies and haptic feedback to enhance procedural learning within a safe, controlled, and repeatable environment. By integrating the Inverse3 haptic device with the Meta Quest 3 VR headset, this project aims to bridge the gap between theoretical instruction and hands-on clinical experience, offering a scalable and accessible training tool for medical students and healthcare professionals.
Goals
- Develop a fully integrated desktop simulation system combining the Inverse3 haptic device and Meta Quest 3 VR headset.
- Provide tactile and visual realism that accurately mimics human anatomy and the sensation of syringe insertion.
- Produce comprehensive technical and user documentation to support training and future system development.
Project Aims
- Create a realistic, interactive training tool for medical students to practise needle insertion techniques safely.
- Enhance clinical education and confidence by simulating realistic tactile feedback and environmental immersion.
- Provide a foundation for future extensions to cover different medical specialisations and procedures.
University of Sydney Faculty of Medicine and Health Media Lab: The client team providing anatomical and procedural expertise, ensuring educational and technical accuracy.
Medical students and healthcare professionals: The primary users who gain hands-on experience and improved confidence through realistic simulations.
Educators and trainers: End users embedding the simulation into teaching programs and assessing training outcomes.
Development team: Responsible for software development, 3D modelling, device integration, testing, and documentation.
- Rotating XP roles (weekly): Manager, Tracker, Doomsayer, Client Liaison, and Tester.
- 3D modelling of the human patient in Blender, including mesh creation and armature rigging.
- 3D modification and armature rigging of key assets in Blender, including the tourniquet, syringe, and arm rest.
- Performed texture painting of vein lines on the patient model in Blender.
- Animation of the human patient and medical assets in Unity, including:
- Patient: blinking, breathing, looking around, hand pumping, and pain reactions.
- Assets: blood-filling syringe animation and tourniquet falling off.
- Created vein and muscle hit zones in Unity to support accurate anatomical needle injection detection.
- Created custom materials for key medical assets in Unity, including the syringe, arm rest, and tourniquet.
- Integrated all major components into a unified Unity scene, including syringe–haptic interactions, haptic resistance materials, UI player boundaries, the 3D doctor’s office environment, and the patient model.
- Set up and configured the Meta Quest 3 for Unity development and testing.
- Regularly performed hardware troubleshooting for the Meta Quest 3 headset and the Inverse3 haptic device.
- Created project setup documentation and repository usage guides for team onboarding and consistency.
- Performed usability testing with FMH staff and iterated based on feedback.
- Performed regular testing on the simulation to fine-tune injection functionality, anatomical accuracy, and scene scaling for accurate spatial proportions.
- Recorded simulation demonstrations using the Unity Recorder package.
- Unity 2021 LTS
- XR Interaction Toolkit
- Meta Quest 3 SDK
- Haply Inverse SDK
- C#
- Blender
- VS Code
- GitHub
- Meta Quest 3 - VR headset used for immersive interaction and visualisation.
- Inverse3 haptic device - provides physical resistance and haptic feedback mapped to virtual needle depth.
 |
 |
 |
 |