Skip to content

Course Lab Benchmark: General Improvements #20

@tareknaser

Description

@tareknaser

The plan is for me to get more familiar with the Course Lab Benchmark while taking notes on areas that could be improved. After that, I’ll submit a PR with the proposed updates.

The end goal isn’t super structured yet but a few things I have in mind are:

  • Add tests
  • Add support for including additional courses (right now, we only include mit_65840_2024). We should have a more generic data schema so that labs from other classes can be included as well.
    • Then also update the instructions on how to extend the benchmark accordingly
  • Add templates for pre-training and fine-tuning

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions