-
Notifications
You must be signed in to change notification settings - Fork 23
Description
The repository contains an empty /ckpt directory but no actual model checkpoint files are provided. This prevents users from running the evaluation toolkit as the pretrained models mentioned in the README are not accessible.
Steps to Reproduce
- Clone the repository:
git clone https://github.com/ASLP-lab/SongEval.git - Navigate to the project directory:
cd SongEval - Install dependencies:
pip install -r requirements.txt - Try to run evaluation:
python eval.py -i /path/to/audio.mp3 -o /path/to/output
Expected Behavior
The toolkit should load pretrained models and successfully evaluate audio files across the five aesthetic dimensions:
- Overall Coherence
- Memorability
- Naturalness of Vocal Breathing and Phrasing
- Clarity of Song Structure
- Overall Musicality
Actual Behavior
The evaluation fails because no model checkpoint files are found in the /ckpt directory.
Environment
- Python version: [Please specify]
- Operating System: [Please specify]
- Hardware: [CPU/GPU details]
Possible Solutions
- Add the missing checkpoint files to the
/ckptdirectory - Provide download links or instructions for obtaining the pretrained models
- Update the README with clear instructions on how to acquire the model weights
- Consider hosting the models on Hugging Face Hub (similar to the dataset at
huggingface.co/datasets/ASLP-lab/SongEval)
Additional Context
The repository README mentions "pretrained neural models" but doesn't provide information on how to obtain or download these models. This is a critical blocker for users wanting to use the aesthetic evaluation toolkit.
Suggested Documentation Updates
Please consider adding a section to the README that includes:
- Model download instructions
- Model file requirements and expected locations
- File size and storage requirements
- Alternative hosting locations if models are too large for Git LFS
Thank you for developing this valuable tool for song aesthetic evaluation!