Skip to content

Fix vocabulary merge count accumulation#40

Merged
SYSTEMS-OPERATOR merged 1 commit intomainfrom
codex/find-and-fix-bug
Jun 27, 2025
Merged

Fix vocabulary merge count accumulation#40
SYSTEMS-OPERATOR merged 1 commit intomainfrom
codex/find-and-fix-bug

Conversation

@SYSTEMS-OPERATOR
Copy link
Owner

Summary

  • fix merge_vocab so frequencies are combined when words merge into the same token
  • test that merge_vocab aggregates counts when tokens collide

Testing

  • pytest -q

https://chatgpt.com/codex/tasks/task_e_685e526131e0832484f5a7a517bf0078

@SYSTEMS-OPERATOR SYSTEMS-OPERATOR merged commit ceb163c into main Jun 27, 2025
1 check passed
@SYSTEMS-OPERATOR SYSTEMS-OPERATOR deleted the codex/find-and-fix-bug branch June 27, 2025 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant