Skip to content

fix: vLLM health() and rerank() now honour TLS and auth credentials#5340

Open
gyliu513 wants to merge 1 commit intollamastack:mainfrom
gyliu513:tls
Open

fix: vLLM health() and rerank() now honour TLS and auth credentials#5340
gyliu513 wants to merge 1 commit intollamastack:mainfrom
gyliu513:tls

Conversation

@gyliu513
Copy link
Copy Markdown
Contributor

What does this PR do?

Fix health() and rerank() ignoring TLS/SSL and auth configuration — health() was creating a bare httpx.AsyncClient() without applying network.tls or shared_ssl_context, causing health checks to fail against vLLM servers with self-signed certificates or enterprise CAs, and rerank() was using a bare aiohttp.ClientSession() with empty headers, failing against authenticated or custom-TLS servers.

Both methods now use a shared _build_httpx_client_kwargs() helper that mirrors the TLS resolution strategy in OpenAIMixin.client (prefer network config, fall back to shared_ssl_context), rerank() also sends the Authorization header when api_token is configured, aiohttp is replaced with httpx for consistency with the rest of the codebase, and the redundant manual tls_verifyshared_ssl_context logic in initialize() is removed since the config model_validator already migrates tls_verify to network.tls.verify.

Test Plan

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Mar 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant