From high-tech spectrographs to old-fashioned sheet music, there are plenty of ways to visually encode sound, but none quite like PhonoPaper. The app, free on iOS and Android, analyzes anything you can record on your phone, then saves it as an image filled with black-and-white blobs. Print that image out, and you — or anyone else with PhonoPaper installed — can read it with a smartphone camera and play the sound back.
PhonoPaper’s interface isn’t always intuitive and its sound is sometimes unintelligible, but once you get the hang of the app, using it feels magical. The first thing I tried was the intro to The Knife’s “Heartbeats,” which has a repetitive rhythm and an easily recognizable melody. That didn’t turn out so well: scanning my camera like a barcode reader across the image I printed out, I heard a thick blanket of static, with the song’s signature bass line just barely peeking out.
A second go, recording myself talking, worked much better. The voice that came out of the app is muffled and robotic, but it’s recognizable as my own, and you can even make out a few of the words. Voila!
PhonoPaper encodes and decodes audio using a two-pronged process called spectral analysis and resynthesis, which operates on the principle that every sound is composed entirely of simple vibrations called sine waves. The app breaks the sound down to these sonic building blocks, then represents each as a squiggly line on an image. When PhonoPaper “reads” the image back, it uses those squiggles to control lots of digital sine wave generators, rebuilding the sound from the ground up.
Getting a good spectral analysis usually takes careful tweaking of parameters with esoteric names like “frequency resolution” and “amplitude threshold,” so considering that PhonoPaper takes a more user-friendly one-size-fits-all approach, the low fidelity is understandable.
Experimentation is where the app truly shines. As the video above demonstrates, you can print out a blank template and draw your own sound, or set the app to “free mode” and have it try to make sense of anything in your camera’s field of vision — usually resulting in dense, ever-shifting waves of noise.
It’s not hard to imagine using PhonoPaper as a simple instrument for making experimental electronic music, or to secretly pass messages to fellow audio geeks. If you’re interested in perfectly reproducing digital sound, there are many much easier, more effective ways to do that.
