Spectral Tablature is a series of collaborative installations that explore sound generated through visual processes. Sound is recorded or synthesized using common techniques then converted into images called spectral analysis. These forms are re-interpreted as a visual artifact then converted back into sound. For each pair, or “duet,” the similarities and differences in tone and texture can be heard as well as seen in the work. This series, along with two more of my installations, is currently on display for my thesis exhibition at the Northrup King Building in Minneapolis. Please read on for images and descriptions of each pair of prints along with the audio.

The first pair in the series are titled Vocal Exercises and Vocal Exercises Reprise. In this pair I collaborated with my father (also named John Keston) who was the subject for the vocal recording. The image above represents the spectral analysis of his recorded vocalizations. Frequency is the y-axis and time is the x-axis. The darker lines are the fundamental frequecies. The faded areas show overtones and undertones.

To re-interpret this piece I regenerated the recorded vocalizations using formant sequencing, a technique that synthesizes human vowels sounds and sibilance similar to speech synthesis. Instead of sampling the audio, a collection of parameters are extracted from the signal that control filters over time in a formant sequence that is then programmed into the Yamaha FS1R synthesizer.

The second pair are titled Synthetic Arches and Synthetic Arches Reprise. In print the spectral analysis of the original synthesized, arpeggiated sequence generates a precise, geometric pattern that is rarely seen in acoustic signals.

Collaborator Jasio Stefanski’s interpretation of the piece was deconstructed using a system of shapes and colors that simplify and emphasize the original geometric forms. Ironically the texture of the paper and screen printed inks create a more complex sonic texture than the original synthesized oscillations.

The final pair in the series are titled Rubato Etude #9 and Rubato Etude #9 Reprise. This sparse piano composition illustrates the dense and organized frequency spectrum of the instrument while the sustain pedal is held.

In this interpretation by Jon Davis the spectral analysis of the piano recording has been composited with a processed photograph of rail cars producing a glitchy, organic noise along with the piano textures.

Note: all of the spectral analysis files were generated using Michel Rouzic’s powerful application, Photosounder. The Yamaha FS1R system exclusive files that contain the formant sequence in Vocal Exercises Reprise were created using Zach Archer‘s online tool FSeq-Flash. You can also view the iPad optimized web application that I designed for the exhibit’s gallery installation.