DDSP

Google Magenta · 2020

A library that enables direct integration of classic signal processing elements with DL models.

Application types: MIDI-to-audioAudio synthesisStyle transfer
Architecture:Differentiable DSP

Essential categories

CategoryStatusNotes
Source code✔︎ OpenThe complete source code is publicly available on GitHub, including core differentiable modules, training pipeline and inference code.
Training data✔︎ OpenThe research paper indicates that the autoencoder is trained on public, fully open-source datasets including NSynth and individual instrument stems from the URMP dataset (see Section 4.2). The repository includes open scripts (`ddsp_prepare_tfrecord`) to easily format custom training data.
Model weights~ PartialPre-trained model weights are available in demos and tutorial Colab notebooks, and can be downloaded from there. However, no independent downloadable weights are provided.
Code documentation✔︎ OpenThe codebase is well documented, with a detailed README file and additional documentation for the core differentiable modules. The repository also includes example notebooks for training and inference, providing clear guidance on how to use the code.
Training procedure✔︎ OpenThe training procedure is described in the research paper. It is also well-documented in the GitHub repository, with clear instructions and scripts provided for training the model.
Evaluation procedure✔︎ OpenThe evaluation procedure is described in the research paper (see Appendix C, for further details).
Research paper✔︎ OpenAccepted at ICLR 2020.
Licensing✔︎ OpenThe project is licensed under the Apache License 2.0.

Desirable categories

CategoryStatusNotes
Model card∅ Not included
Datasheet∅ Not included
Package⭐ IncludedIt is distributed as an indexed, versioned Python software package that can be installed via pip (`pip install ddsp`).
User-oriented application⭐ IncludedDSSP is a highly accessible model to end-users and creators through user-friendly interactive Google Colab notebooks, web-based demos, and an official real-time VST3/AU audio plugin (ddsp-vst) for digital audio workstations (DAWs).
Supplementary material page⭐ IncludedThere is a dedicated blog and supplementary page featuring interactive listening examples, sonified timbre transfers, and explanations of the core architecture.

Raw YAML file with complete evaluation.