Humans undertake many creative activities. Artificial intelligence has now succeeded in replicating these to a great extent. Among the human activities that AI reproduces, music has been one of the toughest. But not any more it seems. A bouquet of apps and AI-based programs are out there producing AI music. Jukebox, from the US-based OpenAI research laboratory, is the latest and arguably the savviest of the lot to join the club. 

Using music with AI is a tough task. And that’s precisely why the researchers decided to give it a try, says OpenAI. The results are not supremely impressive if you go by human standards, but as they say, this small step by AI into music could be the beginning of a giant leap in the history of machine learning. 

The ‘song’ produced by Jukebox is more like super sweet copies of many songs we have already heard. But they are technically sound; the chords, melodies and often words are in sync with the rules and people can figure out their order and placement properly. 

But how did OpenAI do it?

The team firsts tutored the program using some raw audio, instead of using “symbolic music,” like player pianos. The researchers first used convolutional neural networks. Then they encoded and compressed raw audio and sent to a transformer. This generated new compressed audio that was then upsampled and transformed back into raw audio. 

OpenAI had earlier developed a music-making AI, MuseNet. Jukebox is a little more advanced as it can generate lyrics in joining with OpenAI researchers. MuseNet, used MIDI data, while JukeBox was trained on a raw dataset of 1.2 million songs and lyrics scoured from LyricWiki. 

That said, Jukebox cannot compose new music yet. The AI application now takes about nine hours to render a single minute of music. Thus to make it useful for large scale interactive purpose is out of scope currently, say the researchers.

“While Jukebox represents a step forward in musical quality, coherence, length of the audio sample, and ability to condition on artist, genre, and lyrics, there is a significant gap between these generations and human-created music,” the team of researchers said.  

The applications for Jukebox are mostly academic right now. But, the idea is fascinating states the team.