Skip to content

.circleci: Set up CircleCI#69

Merged
tomtseng merged 45 commits intomainfrom
tomtseng/add-cheap-test
Feb 7, 2026
Merged

.circleci: Set up CircleCI#69
tomtseng merged 45 commits intomainfrom
tomtseng/add-cheap-test

Conversation

@tomtseng
Copy link
Collaborator

@tomtseng tomtseng commented Feb 5, 2026

Changes

infra:

  • Replaces Github Actions CI with CircleCI
  • Makes CircleCI work: runs lint, runs smoke tests
    • And on every commit to main, runs multi-GPU tests through Runpod.
      • failed multi-GPU tests are supposed to notify to Slack, I haven't set that up yet though, I'll deal with it in a separate PR
  • Move dev dependencies from [project.optional-dependencies] to [dependency-groups], seems to be best practices for uv

tests:

  • Turns most tests into pytest tests, but adds the marker expensive and doesn't run them in CI because they're too expensive for smoke tests
  • Adds one CPU integration test test test_lora_finetune_cheap (uses pythia-14m)
    • needed to refactor LoRA target_modules to not be hardcoded
  • Adds one multi-GPU test
  • changes the strongREJECT test to be cheaper so that it can be a CircleCI evaluation integration test

fixes:

  • need to use_fast=True, the slow tokenizer had an import error. not sure if this breaks other stuff
  • was having an annoyance where run_in_isolation() hangs if ctrl-C'ed, so I adjusted it to stop hanging
  • fix a circular import in GCG by moving implementation from attacks/ to evals/

Testing

CircleCI passes, including multi-GPU tests

tomtseng and others added 30 commits January 15, 2026 20:36
I think [dependency-groups] is preferred for uv. And this changes the
`sync` commands since the dev group is synced by default, so no need for
an extra flag.
@tomtseng tomtseng force-pushed the tomtseng/add-cheap-test branch from 4b0f345 to c455edd Compare February 7, 2026 04:06
@tomtseng tomtseng marked this pull request as ready for review February 7, 2026 09:07
@tomtseng
Copy link
Collaborator Author

tomtseng commented Feb 7, 2026

This probably deserves review but I'm going to merge to avoid adding to the backlog of PR review, and to get CircleCI passing

@tomtseng tomtseng merged commit 5d0fbb6 into main Feb 7, 2026
3 checks passed
@tomtseng tomtseng deleted the tomtseng/add-cheap-test branch February 7, 2026 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant