fix: route compress_prompt to llmlingua2 path when use_slingua=True (#235)#242
Open
ousamabenyounes wants to merge 1 commit intomicrosoft:mainfrom
Open
fix: route compress_prompt to llmlingua2 path when use_slingua=True (#235)#242ousamabenyounes wants to merge 1 commit intomicrosoft:mainfrom
ousamabenyounes wants to merge 1 commit intomicrosoft:mainfrom
Conversation
…icrosoft#235) When use_slingua=True, __init__ already called init_llmlingua2 correctly, but compress_prompt only checked self.use_llmlingua2 and fell through to the LLMLingua-1 causal-LM path, which passes past_key_values to XLMRobertaForTokenClassification and crashes. Generated by Claude Code Vibe coded by ousamabenyounes Co-Authored-By: Claude <noreply@anthropic.com>
Author
|
@microsoft-github-policy-service agree |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes #235
When
PromptCompressoris created withuse_slingua=True, callingcompress_prompt()raised:Root cause:
compress_prompt()only routed tocompress_prompt_llmlingua2()whenself.use_llmlingua2wasTrue.__init__already had the correctif use_llmlingua2 or use_slingua:guard (line 92) to callinit_llmlingua2, but the dispatch incompress_promptwas missing theor self.use_slinguahalf.Fix: one-line change at line 536 of
llmlingua/prompt_compressor.py:Before submitting
Issue: [Bug]: When calling slingua through PromptCompressor, use_llmlingua2 should also be set to true. #235
Added
tests/test_slingua_routing.pywith 3 unit tests covering the fixed routing, existing llmlingua2 routing, and the unchanged neither-flag path. Tests usesys.modulesstubs so they run without ML dependencies.Who can review?
@iofu728 @QianhuiWu @XufangLuo @mydmdm
Generated by Claude Code
Vibe coded by ousamabenyounes