Red Light, Green Light

The problem

Our high school had a day where Grade 7s would come and learn about STEM. We needed an activity where students would have fun but also be interested in learning about AI and the cool technology we have in the world. The idea was to recreate red light, green light from Squid Game, where we would make the doll, and it would use computer vision to detect if a player has moved when it's red.

The approach

The game was built end to end on a Jetson Orin Nano, and Cindy X painted the 3D-printed doll to make it look realistic. For tracking, I used YOLOv8-pose for real-time multi-player movement, while also keeping each player's identity remembered. The projector screen showed the UI, the camera view, and what state it was in. It drew outlines of every player and waited for everyone to step behind the start line before starting. A servo-driven head turns between green and red-light phases, and caught players are identified by their clothes and called out over a PA speaker via ElevenLabs text-to-speech.

The result

The game ran smoothly but there was still some challeneges. Getting there took a handful of optimizations: pre-resizing camera frames before feeding the model instead of leaving that to the model itself, running shirt-colour detection only at catch/win moments instead of every frame, and locking the Jetson at max clocks. I also built three dedicated calibration GUIs which made it much easier to get the whole system dialed in before event day.