Conversation
Update existing call site usage
|
Currently one failing test:
|
There was a problem hiding this comment.
Pull request overview
This PR adds guided/structured JSON generation support for the CoreML backend by wiring CoreML into the existing ConstrainedJSONGenerator infrastructure (token-level constrained sampling), and updates structured generation to be async-compatible across backends.
Changes:
- Made
TokenBackend.decode/sampleandConstrainedJSONGenerator.generate()async, updating all call sites and tests accordingly. - Implemented CoreML structured generation via a new CoreML token backend that applies logits processors and masks logits to the schema-allowed token set.
- Added CoreML structured generation tests and adjusted CoreML test suite execution to run serially.
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| Tests/AnyLanguageModelTests/StructuredGenerationTests.swift | Updates structured generation tests to use async generate(). |
| Tests/AnyLanguageModelTests/Shared/MockTokenBackend.swift | Updates mock backend to conform to async TokenBackend API. |
| Tests/AnyLanguageModelTests/CoreMLLanguageModelTests.swift | Serializes CoreML suite, caches model download/compile in a task, and adds structured generation tests. |
| Sources/AnyLanguageModel/StructuredGeneration.swift | Converts constrained generation to async and skips special tokens for numeric token sets. |
| Sources/AnyLanguageModel/Models/MLXLanguageModel.swift | Adapts structured generation path to async generate(), updates backend methods. |
| Sources/AnyLanguageModel/Models/LlamaLanguageModel.swift | Adapts structured generation to async generate() and updates backend methods (currently broken due to pointer lifetime issue). |
| Sources/AnyLanguageModel/Models/CoreMLLanguageModel.swift | Adds structured generation for CoreML (schema prompting + constrained sampling backend) and adjusts string generation output handling. |
| Package.resolved | Adds/updates Swift package dependencies used by the CoreML implementation. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Related to #27