Space Invaders TUI · 2026
Designing a tangible controller for Space Invaders using computer vision
How do we build a controller that feels like flying a ship?
The goal was to design a Tangible User Interface (TUI) for Space Invaders, one where the physical interactions mapped naturally to what was happening on screen. Rather than simply remapping keyboard keys to buttons, I wanted users to feel like they were piloting the ship.
From arcade cabinets to fighter jets
I started by researching vintage game controls, since Space Invaders is an arcade game. The space invaders arcade console is a simple layout of buttons, so it served little inspiration for my controller. But, in the process I came across race car arcade games that used a wheel as their controller, allowing the user to turn left and right.
Racing themed arcade games became the foundation of my controller because they make the user feel like they are actually steering the car instead of simply pushing buttons. But, users would be flying a ship in Space Invaders, not a car. So I examined existing flight controls found in ships and planes, such as Star Wars’ X-Wing and real life jets. Most jets, both fictional and real, seem to use a stick to control the plane.
My first sketches mirrored these controllers. It was a stick with a simple trigger at the top. The user could steer by shifting the stick around, and could shoot by pressing the trigger. But, there were two factors I didn’t love about this design.
My first issue was that the control stick felt like it should move in all four directions, and restricting that motion was going to require additional work. Second, I needed to place the CV markers somewhere on the controller, and having two hands on the single stick removed a lot of spare room. So I went back to looking at planes and found a jet controller with two handles that rotated around a center point. This design instantly felt more intuitive for my game. So, I sketched out a design that could be held by two hands.
Design Recipe
The final controller uses tilt to steer and a physical trigger to shoot. A folded cardboard accordion acts as a spring inside the trigger, and a CV marker hidden beneath it disappears when pressed, serving as a Boolean input in Beholder.
Bringing the TUI to life
Once my design was complete, I built the controller using cardboard, foam, hot glue, and Beholder. Beholder is a computer vision software that uses markers to assign computer keys to actions.
A controller people actually wanted to use
During in-class testing, users found the controller intuitive and satisfying to use. Many successfully played and won rounds of Space Invaders without any instruction. The trigger received particular praise for its tactile feel.
What I'd do differently
The CV marker delay made rapid firing difficult, and the controller couldn't navigate the start menu. If I revisited this project, I'd refine the input response speed, improve the physical craftsmanship, and scale up the form factor to accommodate a wider range of hand sizes.