Today, in partnership with YouTube, Google DeepMind announced Lyria, “our most advanced AI music generation model to date,” and two AI experiments designed to open a new playground for creativity:

• Dream Track – an experiment in YouTube Shorts designed to help deepen connections between artists, creators, and fans through music creation.
• Music AI tools – a set of tools that Google Deep Mind is designing with artists, songwriters, and producers to help bolster their creative processes.

To develop these projects, Google Deepmind brought together Google technical experts with “a diverse range of world-renowned artists and songwriters to explore how generative music technologies can responsibly shape the future of music creation, enhance the work of professional musicians and the artist community, and deliver a positive contribution to the future of music.”

The Lyria model

Built by Google DeepMind, the newe Lyria model “excels at generating high-quality music with instrumentals and vocals, performing transformation and continuation tasks, and giving users more nuanced control of the output’s style and performance.”

“Music contains huge amounts of information — consider every beat, note, and vocal harmony in every second. When generating long sequences of sound, it’s difficult for AI models to maintain musical continuity across phrases, verses, or extended passages. Since music often includes multiple voices and instruments at the same time, it’s much harder to create than speech.”

Dream Track

The team is trialing Lyria in an experiment called Dream Track, which is designed to test new ways for artists to connect with their fans and developed in collaboration with YouTube. “A limited set of creators will be able to use Dream Track for producing a unique soundtrack with the AI-generated voice and musical style of artists including Alec Benjamin, Charlie Puth, Charli XCX, Demi Lovato, John Legend, Sia, T-Pain, Troye Sivan, and Papoose. Each participating artist has partnered with us and will have a hand in helping us test and learn to shape the future of AI in music.

“Dream Track users can simply enter a topic and choose an artist from the carousel to generate a 30-second soundtrack for their Short. Using our Lyria model, Dream Track simultaneously generates the lyrics, backing track, and AI-generated voice in the style of the participating artist selected.

Here are a couple of samples generated in the styles of Charlie Puth or T-Pain:

Exploring music AI tools with the industry

“Our researchers have been exploring with artists, songwriters, and producers in YouTube’s Music AI Incubator how generative AI can best support the creative process, and working together to responsibly design a suite of music AI tools.

“Imagine singing a melody to create a horn line, transforming chords from a MIDI keyboard into a realistic vocal choir, or adding an instrumental accompaniment to a vocal track.

“With our music AI tools, users can create new music or instrumental sections from scratch, transform audio from one music style or instrument to another, and create instrumental and vocal accompaniments. This work draws on our history of research and experimentation with AI and music, and we’ll continue testing our music AI tools with incubator participants throughout their development.”

Watermarking AI-generated audio with SynthID

The team is also “pioneering responsible deployment of our technologies with best-in-class tools for watermarking and identifying synthetically generated content. Any content published by our Lyria model will be watermarked with SynthID, the same technology toolkit we’re using for identifying images generated by Imagen on Google Cloud’s Vertex AI.”

“SynthID embeds a watermark into AI-generated audio content that’s inaudible to the human ear and doesn’t compromise the listening experience. It does this by converting the audio wave into a two-dimensional visualization that shows how the spectrum of frequencies in a sound evolves over time.

“The watermark is designed to maintain detectability even when the audio content undergoes many common modifications such as noise additions, MP3 compression, or speeding up and slowing down the track. SynthID can also detect the presence of a watermark throughout a track to help determine if parts of a song were generated by Lyria.”

The future of generative music tools

“Generative music technologies could transform the future of music creation and use, the team continues. “Our cutting-edge work in this space will unlock an exciting new wave of artist tools that can inspire creativity for artists, songwriters, producers, and fans everywhere.

“We’ve only just begun to explore how AI can bolster people’s musical creativity and we can’t wait to see what we can accomplish next in partnership with artists, the music industry, and wider creative community.”

Read more on YouTube’s blog

Let us know your thoughts! Sign up for a Mindplex account now, join our Telegram, or follow us on Twitter.