This AI tool generates music from text prompts
Riffusion is fun to explore, while learning how it works
If you are not a subscriber of The Fanbase Builder, join the dozens of artists, creators and music industry executives who receive it for free.
Let’s dive into to today’s topic:
Riffusion: An AI tool for generating music from text prompts
There are many tools that use Artificial Intelligence (AI) to create music. Take a few minutes to explore AI tool Riffusion , and learn how it works.
Why it matters
AI music tools are developing rapidly. Not everyone can keep up with the latest technological innovations, especially those involving phrases like Machine Learning, Stable Diffusion, or Text Prompts.
However, the development of creative AI is undeniable. To maintain a sustainable career in music, it is valuable to explore new tools and gain an understanding of how they work.
How it works
Riffusion is built on Stable Diffusion; an AI model capable of generating photo-realistic images from text input.
Riffusion altered the model to generate images of spectograms - a visual display of a sound clip’s frequencies and amplitudes.
These spectrograms are then transposed to audio with several algorithms and audio processors.
Based on the primary spectogram, more spectograms are diffused to create longer clips
We can tell Riffusion to imagine music in certain situations:
Christmas with the family (takes some effort to hear Christmas in it)
When we give Riffusion more musically orientated prompts, we learn it’s far from perfect:
It does work with emoji prompts 👍
Yes, but..
Somehow I find Riffusion very good at generating loops that help me concentrate on my work. Unfortunately, the music stops after a few minutes, so I still get distracted easily.
Things are moving fast: Riffusion was released in December, and now already there are new AI models with much better audio quality. However, most of them lack a web-app to play around with.
No, this Riffusion robot won't take over a musician's job. It's not a tool that is practically applicable in a creative process. However, it is cleverly developed and has educational value. It also offers a glimpse into the future, where music AI is more developed and is certainly a competitor in the sync market.
Take action now
Use this tool to learn some AI basics. It’ll only take five minutes of your time.
First, play around with Riffusion, just enter a text prompt and press play.
Then, take some time to learn more about how it works by reading this excellent about page.
Go deeper
Check out Water & Music’s Season 3 report about Creative AI, which was released last Friday. The report offers a comprehensive guide on AI’s emerging opportunities and challenges for music.
