\n<\/p>\n
March 3, 2026<\/p>\n
4 min read<\/p>\n
Add Us On Google<\/span>Add SciAm<\/span><\/span><\/p>\n This musician built an AI clone of her voice so anyone can sing as her<\/p>\n Experimental composer Holly Herndon says this technology isn\u2019t here to replace artists\u2014and that the future of creativity belongs to collective intelligence<\/p>\n By Deni Ellis B\u00e9chard edited by Eric Sullivan<\/span><\/p>\n Holly Herndon at the Serpentine North Gallery in London, October 2024.<\/p>\n Matthew Chattle\/Future Publishing via Getty Images<\/p>\n Holly Herndon hears the future of music in data. Herndon came to electronic music after singing in church and choirs in East Tennessee. She earned a master\u2019s degree at Mills College and a doctorate at Stanford University\u2019s Center for Computer Research in Music and Acoustics.<\/p>\n When she began experimenting with machine learning in 2015, the outputs sounded \u201cscratchy,\u201d but she recalls seeing \u201cthe diamond in the rough.\u201d Today those experiments have evolved into custom models that allow anyone to perform as her.<\/p>\n Scientific American spoke to Herndon about training her AI models and her belief that creativity has always been collective\u2014AI just makes it visible.<\/p>\n If you’re enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.<\/p>\n [An edited transcript of the interview follows.]<\/p>\n You describe your work as \u201cprotocol art.\u201d What does that mean?<\/p>\n In the 20th century, the site of media generation\u2014the paper and pen where music was written\u2014was the artistic act. With protocol art, the creative act happens upstream of media generation. It\u2019s creating the rule set and conditions in which art is made.<\/p>\n We\u2019re really interested in training our own models. I always say \u201cwe\u201d because I work with my partner, Mat Dryhurst. We treat each step in the model-making process as a creative intervention moment. The making of the dataset is part of the artwork. I often write music for training\u2014music not necessarily for human ears but for a computer to learn something.<\/p>\n Can you give me an example of what that looks like in practice?<\/p>\n We have an exhibition in Berlin right now. We were inspired by Hildegard von Bingen, a medieval composer. We wanted to pretend as if polyphony had existed when she was alive. We started with a model of her compositions and added rule sets so it could generate polyphony in her style. We took those outputs, rearranged them and gave them to human singers to interpret. Then we created a huge installation where performers sing and invite the public to train with us.<\/p>\n It\u2019s not about putting in \u201cwrite me a pop song with a guitar.\u201d It\u2019s about using this technology to bring humans together to make art in real space.<\/p>\n Most commercial AI models are trained on data scraped from the Internet. Why do you insist on building your own models?<\/p>\n As an electronic musician, I was never one to sample\u2014I always made my own sound palettes. When we started, pre-Suno and pre-all-this-stuff, we had to make our own dataset. It just felt natural, like making my own samples or digital instruments.<\/p>\n One criticism of products [like Suno] is that they\u2019re very \u201cmid\u201d sounding\u2014trained on everything or the most average. My models sound unique because I\u2019m making the training data myself. I also think there\u2019s prompting under the hood in Suno limiting it to three-minute songs with verse-chorus structure. There are guardrails making it boring. I\u2019d love for them to release some constraints.<\/p>\n Has a model ever surprised you?<\/p>\n We did a project called Holly+ around 2021\u2014a voice clone of my particular voice. We worked with Voctro Labs to train a voice model that works in real time so people can sing using my voice. That was game-changing.<\/p>\n If this works in real time, other people can perform each other\u2019s identity in real time. When we were testing it, my partner, who\u2019s British, was singing into it. I heard my voice with a British accent. It was so uncanny, I had to leave the room\u2014he was singing as me. That was one of the biggest mental unlocks of how weird and cool this stuff can get.<\/p>\n I think it\u2019ll take five to 10 years to be seamless. But once we\u2019re body morphing in real time\u2014imagine you could create a model of a whale voice, then do a hybrid soprano whale. When you sing high, it goes operatic; when you sing low, you\u2019re more whale or Barry White. We\u2019re no longer tied to my larynx.<\/p>\n Where do you think we\u2019ll be in 10 years?<\/p>\n A lot of fears around this technology are actually fears of how the current Internet works\u2014the attention economy, how difficult it is as a creator. My partner always says, \u201cScrolling is for bots, and strolling is for humans.\u201d<\/p>\n Our more optimistic vision is using agents to deal with all the crap and filter through stuff, actually bringing us together in the real world. That\u2019s why our projects involve people meeting IRL and doing things together. Some of my smartest developer friends are vibe coding with multiple agents while cooking or hiking with their toddler. Things could be really beautiful if we imagine and build it that way.<\/p>\n Does this technology change your definition of creativity?<\/p>\n This whole AI thing might force us to see ourselves as maybe not the only creative actors in the universe. That needn\u2019t be scary\u2014it could be beautiful and liberating.<\/p>\n Creativity happens in swarm, in community. AI is just collective intelligence\u2014aggregated human intelligence. The 20th-century art model is tied to an individual genius who touches an object and imbues it with value. That\u2019s being thrown on its head. I\u2019m all team collective intelligence.<\/p>\n If you enjoyed this article, I\u2019d like to ask for your support. Scientific American<\/span> has served as an advocate for science and industry for 180 years, and right now may be the most critical moment in that two-century history.<\/p>\n I\u2019ve been a Scientific American<\/span> subscriber since I was 12 years old, and it helped shape the way I look at the world. SciAm <\/span>always educates and delights me, and inspires a sense of awe for our vast, beautiful universe. I hope it does that for you, too.<\/p>\n If you subscribe to Scientific American<\/span>, you help ensure that our coverage is centered on meaningful research and discovery; that we have the resources to report on the decisions that threaten labs across the U.S.; and that we support both budding and working scientists at a time when the value of science itself too often goes unrecognized.<\/p>\n In return, you get essential news, captivating podcasts, brilliant infographics, can’t-miss newsletters, must-watch videos, challenging games, and the science world’s best writing and reporting. You can even gift someone a subscription.<\/p>\n There has never been a more important time for us to stand up and show why science matters. I hope you\u2019ll support us in that mission.<\/p>\n","protected":false},"excerpt":{"rendered":" March 3, 2026 4 min read Add Us On GoogleAdd SciAm This musician built an AI clone of her voice so anyone can sing as her Experimental composer Holly Herndon says this technology isn\u2019t here to replace artists\u2014and that the future of creativity belongs to collective intelligence By Deni Ellis B\u00e9chard edited by Eric Sullivan<\/p>\n","protected":false},"author":1,"featured_media":45714,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[50],"tags":[4932,7172,1280,9441,23491,23490,8229],"class_list":{"0":"post-45713","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-environment","8":"tag-built","9":"tag-clone","10":"tag-composer","11":"tag-experimental","12":"tag-herndon","13":"tag-holly","14":"tag-voice"},"_links":{"self":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts\/45713","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=45713"}],"version-history":[{"count":0,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts\/45713\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/media\/45714"}],"wp:attachment":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=45713"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=45713"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=45713"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}On supporting science journalism<\/h2>\n
It\u2019s Time to Stand Up for Science<\/h2>\n