OnBook is a mixed reality tool designed to revolutionize the way live performers collaborate and rehearse. Utilizing multi-player mixed reality, OnBook streamlines the rehearsal process by helping directors give more nuanced feedback and by enabling actors to practice more efficiently. Bridging the gap between creative intent and execution, OnBook fosters a more supportive and collaborative rehearsal environment for everyone on stage and on set.
Our team started from our desired tech stack. We met during team formation because we all wanted to build a multiplayer, mixed reality app on either Quest 3 or Apple Vision Pro.
Thematically, we decided to use mixed reality technology to create a tool for theatre makers and live performers. Modern day theatre productions are notoriously under-resourced and operate on a very strict time schedule; performers usually only have one chance to work on the staging of a scene with their director and fellow actors before they are expected to have every detail memorized. We wanted to make it easy for performers to record the ideal version of a scene's staging immediately after working with their director and to be able to go back and review this staging individually throughout the production.
"On-Book" is a term borrowed from live theatre production. It refers to the practice of having someone, usually the assistant to the stage manager, following along the rehearsal while keeping track of the scripted lines and blocking notes.
OnBook allows theatre makers to record and play-back their scenes in mixed reality. When recording, an actor wears a Quest 3 headset in passthrough mode and runs through a scene with their scene partner. On playback, the actor's recorded movement and audio data are replayed on a 3D avatar along with an avatar captured from their scene partner. For this iteration of OnBook, we focused on this core functionality along with basic playback controls.
In addition to the core features we fully developed, we designed features for OnBook which would allow theatre makers to annotate their scene recordings, integrate digital props, re-stage their blocking based on the dimensions of a new venue, and capture their realistic expressions.
- Colocation setup
- Avatar recording and playback
- Networking large data files
- Networking small data files
- Experimental Movement SDK capture and playback of data
- Multiplayer networking for colocation
- Putting it all together in such a short amount of time
- Concept design process
- Probably used the most Meta SDKs of any other hacking team
- Time management for hackathons, especially this one. We were all first time hackers.