Skip to content

[Bug Report] Missing model checkpoint files in /ckpt directory #4

@DearCora

Description

@DearCora

The repository contains an empty /ckpt directory but no actual model checkpoint files are provided. This prevents users from running the evaluation toolkit as the pretrained models mentioned in the README are not accessible.

Steps to Reproduce

  1. Clone the repository: git clone https://github.com/ASLP-lab/SongEval.git
  2. Navigate to the project directory: cd SongEval
  3. Install dependencies: pip install -r requirements.txt
  4. Try to run evaluation: python eval.py -i /path/to/audio.mp3 -o /path/to/output

Expected Behavior

The toolkit should load pretrained models and successfully evaluate audio files across the five aesthetic dimensions:

  • Overall Coherence
  • Memorability
  • Naturalness of Vocal Breathing and Phrasing
  • Clarity of Song Structure
  • Overall Musicality

Actual Behavior

The evaluation fails because no model checkpoint files are found in the /ckpt directory.

Environment

  • Python version: [Please specify]
  • Operating System: [Please specify]
  • Hardware: [CPU/GPU details]

Possible Solutions

  1. Add the missing checkpoint files to the /ckpt directory
  2. Provide download links or instructions for obtaining the pretrained models
  3. Update the README with clear instructions on how to acquire the model weights
  4. Consider hosting the models on Hugging Face Hub (similar to the dataset at huggingface.co/datasets/ASLP-lab/SongEval)

Additional Context

The repository README mentions "pretrained neural models" but doesn't provide information on how to obtain or download these models. This is a critical blocker for users wanting to use the aesthetic evaluation toolkit.

Suggested Documentation Updates

Please consider adding a section to the README that includes:

  • Model download instructions
  • Model file requirements and expected locations
  • File size and storage requirements
  • Alternative hosting locations if models are too large for Git LFS

Thank you for developing this valuable tool for song aesthetic evaluation!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions