Skip to content

Conversation

@Techbert08
Copy link
Contributor

This does three things (I love threes):

  1. Update default config to explicitly use 4.1 mini for scoring. This makes it more clear what's in use.
  2. Extending the LLM interface to have a "get_completions" in parallel to "get_completion". The default providers just route "get_completions" to parallel-call "get_completion". But the Pi one instead uses first-class support for parallel calls. There's some yuckiness around lists of kwargs vs kwargs.
  3. I added "ask_llm_parallel" to expose this functionality to the ranker.

Now it works with a local Pi modelserver and with 4.1 mini. My plan is to merge this (assuming it looks good), then finish deploying the updated Pi modelserver to Azure.

@chelseacarter29 chelseacarter29 merged commit 62de835 into nlweb-ai:main Jan 5, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants