Use containment for speaker sample text checks #4342
Conversation
- Add _normalize_text() for text normalization (lowercase + collapse whitespace) - Add _get_trigrams() to generate character trigrams from text - Simplify compute_text_similarity() to use shared helpers - Simplify compute_text_containment() to use shared helpers Removes code duplication where both functions had their own nested get_trigrams and normalize implementations. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request effectively switches from text similarity to a containment check for speaker sample validation, which is a more appropriate metric for the described use case. The changes are well-tested, with new unit tests for the containment logic and updates to existing tests. I've identified a couple of areas for improvement in the new compute_text_containment function to remove redundant logic, which will improve both clarity and efficiency.
backend/utils/text_utils.py
Outdated
| return ' '.join(text.lower().split()) | ||
|
|
||
| def get_trigrams(text: str) -> set: | ||
| text = normalize(text) |
There was a problem hiding this comment.
The text passed to this get_trigrams function has already been normalized on lines 54-55. This second normalization is redundant and can be removed for efficiency.
| if not trigrams_transcript: | ||
| return 0.0 |
There was a problem hiding this comment.
This check is unreachable. Given the preceding checks on lines 57 and 59, transcript will always have a length of at least 3 when get_trigrams is called. For such strings, get_trigrams will always return a non-empty set. This block can be safely removed.
|
Refactored
Ready for review. by AI for @beastoin |
|
@beastoin Ran by AI for @beastoin |
Add edge-case containment tests and verify the real containment path in verify_and_transcribe_sample to cover the new containment behavior. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Required fixes before merge:
Ready for final review. by AI for @beastoin |
|
@beastoin Re-ran by AI for @beastoin |
thanks |
Fixes #4340.
Switch speaker sample validation to a language-agnostic containment check so trimmed samples pass when their transcript is included in expanded/concatenated expected text, and add tests plus update the backend test script.
deploy steps
by AI for @beastoin