The Google researchers present MusicLM, a model that generates high-fidelity music from textual descriptions such as "a soothing violin melody accompanied by a distorted guitar riff." It makes music at 24 kHz that is consistent across several minutes. 

Experiments demonstrate that MusicLM beats earlier methods in terms of audio quality and fidelity to the text description. In addition, the researchers show that MusicLM can be conditioned on both text and melody since it can change whistled and hummed melodies to match the style defined in a written caption. Finally, to facilitate future research, the researchers offer MusicCaps, a dataset consisting of 5,500 music-text pairs with extensive text descriptions written by human specialists.

MusicLM

MusicLM is a text-conditioned generative model that consistently generates high-quality music at 24 kHz for several minutes while remaining true to the text-conditioning signal. In addition, they demonstrate that our strategy outperforms baselines on MusicCaps, a manually curated, high-quality dataset containing 5,500 music-text combinations created by musicians.

Some of their method's weaknesses are inherited from MuLan, as their model needs to understand negations and adhere to the text's proper temporal ordering. Moreover, their quantitative evaluations require further improvement. In particular, because MCC also uses MuLan, the MCC scores favour their method. Furthermore, future research may concentrate on producing lyrics and enhancing text conditioning and vocal quality. Another part is modelling high-level song structure, such as the introduction, verse, and chorus. An additional objective is to model the music with a greater sampling rate.

Conclusion

MusicLM adds to the collection of technologies that help people with creative activities by producing high-quality music from a written description. But their model and the use-case it addresses come with several hazards. The biases present in the training data will be reflected in the generated samples, presenting the issue of whether it is appropriate to create music for cultures that are underrepresented in the training data while also raising issues about cultural appropriation.