The goals of this Notch project was to track the rotation of a physical object and replicate it digitally and to interact with the scene through spoken input. To this end, we constructed a "guess the number" game, where the user moves a physical box of cereal that is replicated digitally and can guess the number of digital pieces of cereal in the box by speaking their guess out loud.
In this project, I was responsible for setting up the Notch scene, its physics system, and the logic to govern scene transitions. A gyroscope embedded in the physical box passes its rotation data to TouchDesigner, which is then sent via OSC to the Notch scene. My scene setup applies that rotation to the digital version of the box and Notch's physics system to jostle around the several hundred cereal piece clones.
When the user guesses, a microphone in the physical box similarly passes the spoken data to Touchdesigner, which is constantly searching input words for numbers. These numbers are then separated from the rest of the data and sent over OSC to the Notch scene. My scene logic then displays a screen showing if the guess is too low, or too high or correct.
The largest challenge in this scene by far was optimizing the physics and rendering to account for the several hundred cereal pieces and unpredictable user movement. By adjusting scale, polygon count, physics shape complexity, physics update periods, movement smoothing, and delay, I was able to achieve a very smooth result for the volume of physics interactions being processed.
Thank you to team that worked on this project with me - Shelly Boehm, who led the project and provided 3D models, and Nathan Mysona and Angel Hardy, who assisted in setting up the gyroscope tracking and properly passing the data into Notch.