As a reader of this blog, you may be familiar with the online Museum of Imaginary Musical Instruments – a site Thomas Patteson and I launched in 2013 to showcase imaginary musical instruments we came across in our research.
Now, it’s becoming a book! And it turns out the world of imaginary musical instruments is even bigger, weirder and more wonderful than we knew. With over 80 images, the book includes some classics from the online museum (like the cat piano and steam concert) and many new gems. We organized these into a guided tour that offers both a cultural history and a study of creative thought, and reveals changing ideas about sound, invention, and the limits of the possible.
The Museum of Imaginary Musical Instrumentswill be published by Reaktion in 2026, available in April in the UK and July in the US. We hope it sparks something of the joy and wonder we experienced in gathering these remarkable artifacts.
No humans here. Just AI, in robotic form, ready to take full control of the creative tasks of producer and composer. In fact, images intimating such a future have been circulating since the early 20th century. Compare the following image from 1930, from a campaign by a musicians’ union against the substitution of recorded music for live musicians in theaters:
It’s easy retrospectively to laugh at worries that mechanical reproduction would bring the destruction of art. But we should not be so blasé as to think that it all turned out ok. Yes, the substitution of recorded music for live musicians in theaters threatened the livelihood of countless instrumentalists across the nation. But the representation of recorded music as a robot wresting control of musical culture ignored the many people and new kinds of creative labor involved in producing recorded music. We are left to wonder “what if”: what if the energy poured into a simplified, sensationalist image of mechanical substitution had instead been directed to fueling a more robust public discourse about recording’s new configurations of people and tools, and their implications for musical authorship, ownership, and compensation? Perhaps what some now regard as among the greatest injustices in the history of recorded music – the shadow status of session musicians whose creative contributions were integral to the iconic sound of hit songs, for instance, or the infamous Bridgeport Music, Inc. v. Dimension Films 2005 blanket ruling against unlicensed sampling, which failed to fathom the creative labor of hip hop musicians – might have played out differently.
Similarly, today’s robot representations do not capture the realities of AI. As Mary Gray and Siddharth Suri demonstrate in their new book Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass, the AI systems in our lives depend on many largely hidden human workers. Think of search engine results, social media feeds, recommender systems – these “rely on a shared pool of on-demand workers amassed by on-demand platforms” (xvii), and these on-demand platforms “allow humans to power many of the websites, apps, online services, and algorithms most consumers think are automated” (170). By selling their services as powered by the magic of AI, companies (and the media hype surrounding them) conceal an underlying dynamic: that as we ask AI to do more, we are generating “new needs and different types of human labor to fill those needs” (xviii).
Recognizing the human labor involved in AI systems, the “humans in the loop” as Gray and Suri put it, calls for fundamentally reimagining what AI is – conjuring not autonomous robots that eliminate humans from the picture, but rather networked systems in which people play a constant, dynamic, essential part. It calls, in other words, for changing the image of AI in popular consciousness.
Re-enter music. Although music has long been shadowed by fears of robot take-over, musicians are also adept at taking in the world around them and reflecting it back in ways that can wake us up to new (and old) realities, helping us apprehend our world differently not just at intellectual but also at emotional, visceral levels.
This past summer, Holly Herndon released PROTO, an album made in collaboration with other human musicians and Spawn, her name for a collection of vocal models created using machine learning techniques. Herndon also calls Spawn her “AI baby,” and the use of AI in creating the album has driven its media coverage. But Herndon’s work resists reduction to “now AI can compose music too!”-style headlines. When asked by Jezebel’s Hazel Cills how to think about the people behind AI, Herndon responded:
That’s one of the biggest problems of AI; it’s this kind of opaque, black box technology, and when we have this glossy press release where it’s like “the machine just wrote this song” you’re totally discounting all the human labor that went into the training set that the thing learns on. That was a really important part of how we set up the project and the way that we did. We wanted the people training Spawn to be visible, to be audible, to be named, to be compensated, because I think that’s a huge part of what we’re facing with this thing today.
The track “Evening Shades (Live Training)” illustrates this desire for public recognition of the people and labor required to make Spawn work. Through the alternation of a full-voiced human choir with Spawn’s oddly filtered and stuttering efforts to repeat back the same phrase, the track makes audible some of the process – some of the human labor – involved in training Spawn to “sing”:
Rather than picture AI as a robot, then, listening to PROTO can help us imagine AI as a gathering of people and machines – a collective endeavor in which many humans work together with algorithmic systems. What if we were to start our discussions of AI from such an understanding – might we better grapple with the implications for authorship, ownership, compensation, with how to recognize and value the many “humans in the loop?” What if…?
For most of his early career, Beethoven played on German and Viennese pianos. With the light action, clear attack, and rapid decay of these instruments, he composed themes such as this, from the concluding movement of Op. 49, No. 1:
Op. 49, No. 1 – Rondo: Allegro; Zvi Meniker playing a reproduction of an Anton Walter piano, c. 1790
Beethoven, Op. 49, No. 1, third movement (first edition)
In 1803, Beethoven received a new, French piano from the maker Sébastian Érard. This piano responded differently to his touch, used a foot pedal rather than knee lever to lift the dampers, and produced different sonorities. It was – as Tilman Skowroneck has discussed – a new tool with which to conceive musical ideas. With the less clear attack and longer decay of its tones, Beethoven explored new possibilities and abandoned old ones. One result was themes such as this, from the last movement of Op. 53 (the Waldstein Sonata):
Sonata No. 21, Op. 53 – Rondo: Allegretto moderato – Prestissimo, Bart van Oort playing a c. 1815 Salvatore Lagrassa piano
Beethoven, Op. 53, third movement (first edition)
The long slurs, the slow pace of harmonic change, the rippling accompaniment in the right hand while the left hand alternates between resounding bass notes and treble-register theme, the bell-like quality of this theme – all are products of Beethoven’s interaction with his new Érard piano, the medium of his creative thought.
In “Personal Dynamic Media” (1977), Alan Kay and Adele Goldberg heralded a new “dynamic medium for creative thought” in the form of the Dynabook, a predecessor to the notebook computer.
Kay and Goldberg described the Dynabook as an active medium (or really, “metamedium” that can be all other media), which they saw as basically unprecedented. “For most of recorded history,” they wrote, “the interactions of humans with their media have been primarily nonconversational and passive in the sense that marks on paper, paint on walls, even ‘motion’ pictures and television, do not change in response to the viewer’s wishes.”
Yet the piano, and indeed all musical instruments, are responsive media. Some are more responsive than others – in 1796, Beethoven was dissatisfied with a Streicher-made piano because it “deprived him of the freedom to create my own tone.” But all musical instruments respond to the “queries and experiments” (to use Kay and Goldberg’s language) of their users.
Why did Kay and Goldberg exclude musical instruments from the prehistory of the Dynabook? Not out of neglect. As Kay and Goldberg state, “one of the metaphors we used when designing such a system was that of a musical instrument, such as a flute, which is owned by its user and responds instantly and consistently to its owner’s wishes.” Here, the reason for the exclusion becomes clear: Kay and Goldberg conceived musical instruments as interfaces, not as media.
Recently, Kay has suggested that musical instruments and computers belong to the same category. In a 2003 interview, he remarked, “the computer is simply an instrument whose music is ideas.” This sounds like a statement from a culture in which musical instruments are primarily vehicles for already composed music. It is as if music exists prior to instruments, simply waiting to be accessed. That musical instruments and computers are now the same for Kay may reflect the failure of one the dreams behind the Dynabook: the dream that everyone would become computer “literate.” Discussing the thinking behind the programming language he developed for the Dynabook, Kay explained, “the ability to ‘read’ a medium means you can access materials and tools generated by others. The ability to ‘write’ in a medium means you can generate materials and tools for others. You must have both to be literate.” The early environments developed using Kay’s language emphasized the “writing” side of literacy: they were for such activities as painting, animation, and composing. On the Dynabook, kids wouldn’t learn how to play a musical instrument – they would create their own musical instruments, and write music with them. With the Dynabook, Kay and Goldberg hoped, “acts of composition and self-evaluation could be learned without having to wait for technical skill in playing.”
Music on the Dynabook prototype. On the right, “a musical instrument is created.”
But lets look at how Kay and Goldberg conceptualized media in 1977: “external media serve to materialize thoughts and, through feedback, to augment the actual paths the thinking follows.” That, to me, sounds like a good description of media. And it sounds like an excellent description of Beethoven’s Érard piano. Which should teach us that no technology can be a short-cut to our ideas; but any can be a medium for creative thought.
Everything is an idea
Technology drives creativity
Pushing us
And the bounds of our
Imagination
Technology inspires composers
Technology inspires children
Technology inspires closet rock stars
And even nations
Opening up broader, richer frontiers
Challenging what can be
This is the beauty of technology
Evolving how we create
One idea at a time
Where will we go next?
Technology transforming creativity
Technology is a big destroyer of emotion and truth. Technology doesn’t do anything for creativity. Yeah, it makes things easier and you can get home sooner. But it doesn’t make you a more creative person. That’s the disease we have to fight in any creative field – ease of use.
Deirdre Loughridge 
