AFTER

IRCAM · 2024

Generative model that creates new audio by blending two sources: one audio stream to set the style or timbre, and another input (either audio or MIDI) to shape the structure over time.

Application types: MIDI-to-audioAudio synthesisStyle transfer
Architecture:Latent diffusionRectified flow

Essential categories

CategoryStatusNotes
Source code✔︎ OpenSource code is available in the GitHub repository, including data processing, training and inference pipelines, as well as model architecture.
Training data~ PartialDatasets used for training the original model (SLAKH, MaestroV2, GuitarSet and URMP) are publicly available and described in the paper, but the datasets used for training the current model are unknown.
Model weights✔︎ OpenWeights are available to download from the link provided in the GitHub repository.
Code documentation✔︎ OpenDocumentation is available in the GitHub repository, including installation requirements, examples of scripts usage and configuration files. The codebase could still benefit from more detailed documentation.
Training procedure~ PartialTraining procedure of the original model is fully described in the paper, but relevant changes in the model architecture and training procedure are not properly documented.
Evaluation procedure~ PartialEvaluation procedure of the original model is fully described in the paper, including used metrics and implementation details. However, no evaluation is provided for the current model.
Research paper~ PartialOriginal paper is publicly available and peer-reviewed, but no publication is available for the current model.
Licensing~ PartialCreative Commons Attribution-NonCommercial 4.0 International license.

Desirable categories

CategoryStatusNotes
Model card∅ Not included
Datasheet∅ Not included
Package∅ Not included
User-oriented application⭐ IncludedMax/MSP patches with real-time implementation of the model are provided.
Supplementary material page⭐ IncludedThere is a supplementary page for the original model.

Raw YAML file with complete evaluation.