MusicGen

Meta AI · 2023

A simple and controllable music generation model, which is able to generate high-quality music given textual description.

Application types: Text-to-musicMelody-to-music
Architecture:Autoregressive transformer

Essential categories

CategoryStatusNotes
Source code✔︎ OpenCode is available.
Training data~ PartialThey describe the datasets used to train their model (licensed data). However, they rely on 10k music tracks that are not described (proprietary data). They do not provide any of the datasets used for training MusicGen. Only a dummy dataset containing just a few examples for illustrative purposes is provided.
Model weights✔︎ OpenModels are available in the GitHub repository and in Hugging Face. Note that the weights are released under the CC-BY-NC 4.0 license as found in the https://github.com/facebookresearch/audiocraft/blob/main/LICENSE_weights.
Code documentation✔︎ OpenCodebase is documented in the GitHub repository, including installation requirements.
Training procedure✔︎ OpenTraining procedure is well documented in the paper, including hardware requirements and model configurations. Additional implementation details are available in the GitHub repository.
Evaluation procedure✔︎ OpenEvaluation is documented in the paper and the code provides detailed explanation about the implementation of evaluation metrics. Evaluating the model to reproduce the results still may require some effort. Dataset used for evaluation is MusicCaps benchmark.
Research paper✔︎ OpenAccepted at NeurIPS 2023.
Licensing✔︎ OpenThe code in this repository is released under the MIT license as found in the https://github.com/facebookresearch/audiocraft/blob/main/LICENSE. The models weights in this repository are released under the CC-BY-NC 4.0 license as found in the https://github.com/facebookresearch/audiocraft/blob/main/LICENSE_weights.

Desirable categories

CategoryStatusNotes
Model card⭐ IncludedModel card discusses intended use, limitations and biases.
Datasheet∅ Not includedData is copyrighted and not available.
Package⭐ IncludedHas two packages: audiocraft package from Meta, and transformers package from Hugging Face.
User-oriented application⭐ IncludedDemo in Hugging Face spaces, user-oriented gradio demo, and jupyter notebooks.
Supplementary material page⭐ IncludedSupplementary material web page with sound examples.

Raw YAML file with complete evaluation.