MusGO Framework: Evaluation Checklist

We here provide an evaluation checklist based on the criteria of the Music-Generative Open AI (MusGO) framework. This evaluation checklist is linked to the research article “MusGO: A Community-Driven Framework for Assessing Openness in Music-Generative AI”, authored by Roser Batlle-Roca, Laura Ibáñez-Martínez, Xavier Serra, Emilia Gómez, and Martín Rocamora.

Essential

1. Source code:

Source code is fully open.

It includes:

Data processing
Model architecture
Training pipeline
Inference code

The code is shared in a well-organised and usable format, allowing detailed analysis.

2. Training data:

Training data is fully described.

It includes:

Data sources
Data acquisition methods
Data permissions or licenses associated with its use

Training data is accessible.

If direct access to the data is restricted due to copyright or other legal concerns, all sources and relevant access conditions are fully disclosed to allow for transparency and informed inspection.

3. Model weights:

All model weights are publicly available and usable.

4. Code documentation:

The codebase is fully documented.

It includes:

Clear instructions for use and replication
Environment setup and requirements
Configuration guides
Well-commented source code
(Optional) Guidance on how to extend or modify the codebase.

5. Training procedure:

Training procedure is fully described.

It includes:

Hardware requirements
Model configuration
Configuration guides

It allows others to replicate the model and understand the system.

6. Evaluation procedure:

Evaluation procedure is fully documented.

It includes:

Evaluation data
Evaluation metrics
Evaluation process, with necessary code.

7. Research paper:

A research paper or technical report is available.

A peer-reviewed research paper is available.
Note that if the peer-reviewed research paper is not available through open access publication, we consider the category to be “fully open” when a preprint of such research paper is available.

8. Licensing:

Clear and context-appropriate open license, such as OSI or RAIL, applies to all components.

Desirable

9. Model card:

Model card or equivalent documentation is available.

10. Datasheet:

Datasheet or data documentation is available.

11. Package:

Model is released as an installable or versioned package (e.g. PyPI, Conda).

12. User-oriented application:

Interface for interaction exists (e.g., API, demo app, UX tool).

13. Supplementary material page:

A page or demo exists showcasing generated examples.