Skip to main content

How AI is solving one of music’s most expensive problems

How AI is solving one of music’s most expensive problems

/

The last step in audio production is being taken over by machines

Share this story

If you buy something from a Verge link, Vox Media may earn a commission. See our ethics statement.

Photo by Amelia Holowaty Krales / The verge

Making music is one of the most human things we do, but in recent years, AI has stepped in to lend a helping hand. Algorithms have been infiltrating nearly every part of music making, from generating original drum loops to writing melodies, producing parts that are increasingly hard to distinguish from human players. Now, AI is reaching the mastering process, raising hard questions about the need for human experts in the most specialized areas of music production.

Mastering is the final step in audio post production, and balances out all of a song’s elements so it will sound consistent no matter how you’re listening to it — on Spotify, in iTunes, or on a CD. The goal of mastering is to make the listening experience balanced and cohesive from song to song. The process is a blend of science and personal taste. With a good mixdown, a mastering engineer will make sure they understand the sound you’re going for and help you get there. Without mastering, the song will be quieter and less punchy. As mastering engineer Ian Cooper says, mastering is “a bit like photography — you can make the sky bluer, the greens greener.”

“You can make the sky bluer, the greens greener”

Mastering can also be expensive. Depending on the engineer’s experience, it can cost anywhere from hundreds to tens of thousands of dollars for a single track, thanks to the critical listening skills involved. Those prices can break the bank for indie artists and bedroom producers.

But over the past few years, automated options have popped up that promise artists access to professional-sounding mastering without the costs of human engineers. Some use deep learning networks, which analyze the data fed to it over time, while others use a carefully crafted signal chain designed by a human and deployed as software. But no matter how they operate, the goal is the same: mastering audio with a couple clicks.

Landr is one of the most popular such services, hosted as a web service. You can upload the song you want mastered, let Landr’s algorithm analyze it, choose between three options for how strongly you want effects applied, and then export the result. It’s a catchall approach, and it’s not exactly flexible. If you’re not happy with what Landr’s output gives you, you can’t ask it to finesse the sound the way you would with a human engineer. ArsTechnica published a scathing review in 2016, calling Landr’s auto mastering an “auto turd,” but others say it does the job. And in theory, Landr’s algorithm is improving with every song uploaded to the platform. CEO Pascal Pilon told The Verge that “in 2017, we ran a series of blind tests with major labels and professional mastering engineers and LANDR was actually picked over some of the world’s best mastering houses.”

“a traditional mastering engineer will always be the ultimate option.”

Some worry that AI mastering services will eliminate the need for human engineers, but London engineer Streaky compares it to buying an off-the-rack suit. Someone who cares a lot about tailoring or the quality of fabric will still get a bespoke suit made especially for them, but for plenty of people, the cheaper option makes more sense.

Software company iZotope approached AI with an educational lens. The company already makes a popular suite of plug-ins called Ozone, and added in an intelligent “Master Assistant” in 2017. The assistant doesn’t do all the work for you. Instead, it gives you a starting point you can tweak to your liking. That way, producers can make informed decisions based upon the choices the AI has made. “It has nothing to do with competing with humans,” an iZotope representative tells The Verge. “For the fearful professionals out there, assistive technology minimizes time-consuming cleanup work so that they can hone in on the creative side of things.”

MajorDecibel founder Adam Love agreed, saying, “It is not a replacement for mastering done by a mastering engineer. Mastering engineers can provide feedback to the artist about their mix, hone in on a particular style, and make more deliberate corrections and enhancements. A human is slow and methodical but unconstrained. Automation is fast but significantly more limited in what it can do.”

What’s left is an affordable alternative to make music sound better. “Rather than replacing jobs or disrupting an industry we see ourselves as creating a new market, allowing people who currently can’t receive quality mastering to finally have an opportunity to do so,” Collin McLoughlin from eMastered told The Verge. “For the absolute best mastering however, a traditional mastering engineer will always be the ultimate option.”

It’s hard to say if AI can ever learn to listen with nuance the way a person does, but it may not need to. AI mastering is already sophisticated enough to be a viable option for many musicians. “To the people who don’t believe that AI can make competitive sound, I’d say the proof is in the ten million tracks we’ve mastered for millions of artists around the world,” Pilon says. “I’m sure that when the automatic camera was introduced, people had their doubts, but no one can argue that it hasn’t earned its place in the creative field.”