MusGO Framework: Detailed Criteria

We here describe with complete detail the Music-Generative Open AI (MusGO) framework: a community-driven framework for assessing openness in music-generative AI models. This detailed criteria is linked to the research article “MusGO: A Community-Driven Framework for Assessing Openness in Music-Generative AI”, authored by Roser Batlle-Roca, Laura Ibáñez-Martínez, Xavier Serra, Emilia Gómez, and Martín Rocamora.

MusGO framework consists of 13 dimensions of openness, distinguishing between essential (1–8) and desirable (9–13) categories. Essential categories follow a three-level scale: (✔︎ open, ~ partial, or ✘ closed). Desirable categories are binary, indicating whether an element exists (⭐) or not.

Essential

[1] Open source code.

Is the system source code, including the data processing, model architecture, training pipeline, and inference, openly available for inspection and use?

✗ System source code is closed. The system's source code, including the data processing, model architecture, training pipeline and inference, is not available for inspection or use.
≃ Some source code is provided. Part of the source code is available, but significant gaps do not enable full reproducibility or understanding.
✔ System source code is fully available. The complete source code, including the data processing, model architecture, training pipeline, and inference, is accessible for inspection and use.

[2] Training data.

Is the training data of the model fully described, including sources, acquisition methods, and access conditions? Is training data available for inspection?

✗ Training data of the model is not available nor described. The training data is not disclosed, and no information about the data sources, acquisition methods, or access conditions is provided.
≃ Some of the training data or information about the data is available. Some of the training data of the model is available, or information about the data, such as its sources or a general description, is provided. However, the complete dataset is not provided, or information about the data is not complete.
✔ The training data of the model is fully available and described. The training data used for the model is described in detail, including its sources, acquisition methods, and any permissions or licenses associated with its use.

[3] Model weights.

Are the complete model weights (of the production-ready model) fully shared and accessible for inspection and use?

✗ Model weights are not shared. Weights of the production-ready model are not publicly available.
≃ Model weights are partially shared. Some or all model weights are publicly available, but there are limitations or restrictions on their accessibility.
✔ Model weights are fully shared. The complete model weights are publicly available for inspection and use.

[4] Code documentation.

Is the codebase accompanied by sufficient and complete documentation to allow for its replication, extension, or modification?

✗ Code documentation is not available. No documentation for the code is provided, making it difficult or impossible to replicate, extend, or modify the codebase.
≃ Some components of the codebase are documented. Some parts of the code are documented, but the documentation is incomplete or lacks sufficient detail.
✔ The codebase is fully documented. All components of the codebase are accompanied by comprehensive documentation that includes clear instructions, explanations, and details.

[5] Training procedure.

Is the training procedure of the system fully documented to allow for replication and understanding of the system?

✗ Training procedure is not documented. No documentation is provided for the system's training procedure.
≃ Training procedure is partially documented. Some aspects of the system's training procedure are documented, but the details are incomplete or unclear.
✔ Training procedure is fully documented. The system's training procedure is thoroughly documented, including details such as hardware requirements and model configuration.

[6] Evaluation procedure.

Is the evaluation procedure of the system fully documented to support reproducibility of evaluation results and performance?

✗ Evaluation procedure is not documented. No information is provided about the evaluation procedure of the model.
≃ Evaluation procedure is partially documented. Some aspects of the system’s evaluation procedure are documented, but the details are incomplete or unclear.
✔ Evaluation procedure is fully documented. The system’s evaluation procedure is comprehensively documented, including details on evaluation data, evaluation metrics, and the complete evaluation process.

[7] Research paper.

Is there a publicly available and accessible research paper, or alternative technical report, that provides an overview of the introduced model? Is it peer-reviewed by an external group of reviewers?

✘ No peer-reviewed research paper or alternative technical report is available. There is no publicly available publication or report that provides an overview of the introduced model.
≃ A research paper or alternative technical report exists, but it is either not peer-reviewed or not accessible. Any available research paper or technical report may describe an overview of the introduced model, but the document is not peer-reviewed and/or not accessible.
✓ A peer-reviewed and accessible research paper is available. An accessible peer-reviewed research paper exists and provides an overview of the introduced model.

[8] Licensing.

Is the system and its components licensed under a clear and adequate open framework, such as an OSI, RAIL license, or other context-appropriate license?

✘ No license available. The system lacks context-appropriate licensing.
≃ System is only partially covered by an adequate license. Some components of the system are covered by clear and adequate open license, such as OSI or RAIL license, but not all distributed materials.
✓ System is covered by a clear and adequate open license. The system and all distributed materials are fully covered by a clear and adequate open license, such as OSI or RAIL license.

Desirable

[9] Model card.

Is a model card or equivalent documentation provided?

⭐ Model card(s) available. Model card(s) or equivalent documentation is available, providing insights into architecture, training, tuning, evaluation, or intended use cases and limitations.

[10] Datasheet.

Does the model include datasheets or equivalent documentation that provide a systematic and standardized account of the data used for training?

⭐ Datasheet(s) available. Datasheets or equivalent documentation are available, offering an overview of data collection, curation, and considerations like consent, limitations, and selection strategies.

[11] Package.

Is the model released as an indexed software package or provided through an equivalent developer-oriented solution?

⭐ Packaged release available. A versioned software package release (e.g., PyPI, Conda, Homebrew) or equivalent solution is provided, ensuring easy installation, reproducibility, and usability.

[12] User-oriented application.

Is the model accessible via a user-oriented interface, such as an API or a UX tool for creative contexts?

⭐ User-oriented application available. An accessible and user-oriented interface of the model exists, such as an API, a real-time model implementation, or a UX tool designed for creative processes.

[13] Supplementary material page.

Is there a demo page that showcases its capabilities by providing sonified generated examples of the model?

⭐ Supplementary material page available. An accompanying website showcasing diverse examples, including sonified outputs and/or detailed usage demonstrations.