(link to video)

Speak and Play! is a speech synthesizer for pianists. It has two parts: a score generator that turns written English text into musical scores, and a sample set (hurriedly constructed from my own voice) which, when used to play the scores, approximately recreates the original text.

Thanks to pianist Margaret Leng Tan for making Speak and Play! speak and play, and for coming up with the name!

More details below…

The synthesizer is a soundfont made in a hurry from samples of my own voice, and the score generator uses the CMU Pronouncing Dictionary to guess the pronunciation of written texts, and Lilypond to engrave the score. It took a few hours and a chicken parmigiana sandwich to get it working. It’s not ready for the public yet, but if you’d like to try it, get in touch.

I counted the number of uses of each phoneme in the dictionary’s 133,000 or so words with 850,000 phonemes, so I could put the most popular sounds close together to reduce the amount of octave hopping. Here’s the top ten phonemes in English:

AH 70922 8%
N 60394 7%
IH 49759 5%
S 49639 5%
L 49333 5%
T 48459 5%
R 45884 5%
K 42471 4%
IY 34461 4%
D 32352 3%


Here’s the score for “Hello! We love toy pianos!” as used in the video:

Once you have an instrument that plays your own voice, you can mess around with it like this:

(link)