Geoffrey Guterl created a hack that allows him to control an XBox 360 with a MIDI keyboard. In the video above, he demonstrates, playing Tekken — and handily defeating his online opponent! — while playing piano. As you might expect, the music produced isn’t exactly Chopin; the limited controls and repetitive nature of gameplay make for a small palette of sounds. I’d love to see what the gameplay looks like when someone’s playing from sheet music. Guterl explains the nuts and bolts:
How it works: The piano sends a MIDI signal, which is transferred to an Arduino. According to the signals, the Arduino triggers transistors, which then trigger inputs on a paewang PCB (this is the PCB of an arcade stick). The paewang is connected to an Xbox360 (you can also use it on PS3).
There are ways to extract MIDI data from just about any instrument, so there’s no reason this couldn’t be rigged to work with a guitar, clarinet, saxophone, etc with relatively minor modifications. What about playing Zelda with a MIDI Ocarina?
