Magenta RealTime

Google Magenta · 2025

High-quality real-time music synthesis with low-latency real-time controls via MIDI, text, and audio.

Application types: MIDI-to-audioText-to-music
Architecture:Autoregressive transformer

Essential categories

CategoryStatusNotes
Source code~ PartialThe codebase is available on GitHub, including model architecture, and inference code. However, the training code is not publicly available.
Training data✘ ClosedTechnical report indicates the model has been trained on ~190,000 hours of instrumental stock music. But there is no clear description of the datasets or specifc sources used for training.
Model weights✔︎ OpenModel weights are available on Hugging Face.
Code documentation✔︎ OpenModel code is well documented in the GitHub repository, including installation requirements, usage instructions, and examples.
Training procedure✔︎ OpenTraining procedure is documented in the technical report, including hardware requirements and model configurations.
Evaluation procedure✔︎ OpenEvaluation procedure is documented in the technical report, including evaluation metrics and results. Magenta Realtime is compared against two other music genrative models (Stable Audio Open and MusicGen) using the Song Describer dataset.
Research paper~ PartialA comprehensive technical report is available. No peer-reviewed publication has been published yet.
Licensing✔︎ OpenThe code in this repository is released under Apache 2.0 License. The models weights in this repository are released under the CC-BY-NC 4.0 license.

Desirable categories

CategoryStatusNotes
Model card⭐ IncludedModel card is available on Hugging Face.
Datasheet∅ Not included
Package⭐ IncludedMagenta Realtime is available as a Python package, installable via PyPI (`magenta-rt`).
User-oriented application⭐ IncludedThere is a demo notebook available on Google Colab, which allows users to interact with the model in real-time.
Supplementary material page⭐ IncludedThere is a dedicated blog and supplementary page featuring listening examples, demonstrations, and explanations of the core architecture.

Raw YAML file with complete evaluation.