Working with Claude Code and non-Anthropic models?

[elifarley]

I patched poe-code to allow the selection of other models, but once ~/.claude/settings.json:model was set to glm-4.6, it just hangs a bit and shows an error:

claude -p hi
API Error: 500 {"type":"error","error":{"type":"internal_server_error","message":"An internal error occurred while processing your request"}}

Howerver, claude-code was working kinda ok when using LiteLLM proxy (ANTHROPIC_BASE_URL=http://localhost:4000) pointing both Sonnet and Haiku to glm-4.6...

model_list:
  - model_name: "claude-haiku-4-5-20251001"
    litellm_params:
      model: "openai/glm-4.6"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "claude-sonnet-4-5-20250929"
    litellm_params:
      model: "openai/glm-4.6"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

litellm_settings:
  drop_params: True

litellm --config litellm-config.yaml --port 4000

Is there a way to make it work with poe-code, so that we don't need to install LiteLLM?

[kamilio]

Hey @elifarley, currently the anthropic API is only for anthropic. That's a solid finding, we could maybe just add LiteLLM proxy, so users don't have to fiddle with it.
You mentioned that it is working ok, what's working and what is not working?

[elifarley]

GLM is able to see the files, but once it tells me it's done with the modifications I requested, I can see no new commits and no changes in my working directory. GLM even tries to fool me, showing a git log with the new commit, but later on it admits it is a fake commit.

[kamilio]

A colleague of mine just submitted a PR to models.dev https://github.com/sst/models.dev/pull/374/files

You can see which models support tooling and which do not. Could you try with a model that supports tool calling?

I sent you a linkedin request, I wanted to grant you some compute point as thank you.

[elifarley]

Hmmm.. just tested it with Grok fast-non-reasoning and Grok-fast-reasoning. Still no file changes...

I'll test further.

{
  "apiKeyHelper": "$HOME/.claude/anthropic_key.sh",
  "env": {
    "ANTHROPIC_BASE_URL": "http://localhost:4000",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "Claude-Haiku-4.5",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "Claude-Sonnet-4.5",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "Claude-Opus-4.1"
  },
  "alwaysThinkingEnabled": false,
  "model": "Claude-Haiku-4.5"
}

odel_list:
  - model_name: "Claude-Haiku-4.5"
    litellm_params:
      model: "openai/grok-4-fast-non-reasoning"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "Claude-Sonnet-4.5"
    litellm_params:
      model: "openai/glm-4.6"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "Claude-Opus-4.1"
    litellm_params:
      model: "openai/grok-4-fast-reasoning"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "grok-fnr"
    litellm_params:
      model: "openai/grok-4-fast-non-reasoning"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "grok-fr"
    litellm_params:
      model: "openai/grok-4-fast-reasoning"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

litellm_settings:
  drop_params: True

litellm --config poe-config.yaml --port 4000

LiteLLM: Proxy initialized with Config, Set models:
    Claude-Haiku-4.5
    Claude-Sonnet-4.5
    Claude-Opus-4.1
    grok-fnr
    grok-fr
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:4000 (Press CTRL+C to quit)
INFO:     127.0.0.1:58046 - "POST /v1/messages/count_tokens?beta=true HTTP/1.1" 200 OK
INFO:     127.0.0.1:37772 - "POST /v1/messages?beta=true HTTP/1.1" 200 OK
INFO:     127.0.0.1:58046 - "POST /v1/messages?beta=true HTTP/1.1" 200 OK
INFO:     127.0.0.1:37786 - "POST /v1/messages?beta=true HTTP/1.1" 200 OK
INFO:     127.0.0.1:37772 - "POST /v1/messages?beta=true HTTP/1.1" 200 OK
INFO:     127.0.0.1:56146 - "POST /v1/messages/count_tokens?beta=true HTTP/1.1" 200 OK

> Who are you, exactly?
● I'm Claude Code, Anthropic's official CLI tool powered by the Claude AI model. I assist with software engineering tasks like coding, debugging, testing,
  and codebase exploration in your local environment (e.g., this Hug SCM project). I use specialized tools to read/edit files, run commands, search code,
  and manage tasks without leaving the terminal. For more details, try /help or check the docs at https://code.claude.com/docs.

[kamilio]

Cool, let me know!

[elifarley]

@kamilio

I installed a fresh python venv (3.12.12), installed LiteLLM (1.75.7), and tested again with this configuration:

Details

~/.claude/anthropic_key.sh

#!/bin/env bash
. ~/.env && printf "$LITELLM_VIRTUAL_KEY"

~/.claude/settings.json

{
  "apiKeyHelper": "/home/ecc/.claude/anthropic_key.sh",
  "env": {
    "ANTHROPIC_BASE_URL": "http://localhost:4000",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "grok-fr",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "glm-4.6",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "gpt-codex"
  },
  "model": "glm-4.6",
  "alwaysThinkingEnabled": false,
}

litellm-config.yaml

model_list:
  - model_name: "Claude-Haiku-4.5"
    litellm_params:
      model: "openai/claude-haiku-4.5"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "Claude-Sonnet-4.5"
    litellm_params:
      model: "openai/claude-sonnet-4.5"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "Claude-Opus-4.1"
    litellm_params:
      model: "openai/gpt-5.1-codex"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "kimi-k2-t"
    litellm_params:
      model: "openai/kimi-k2-thinking"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "glm-4.6"
    litellm_params:
      model: "openai/glm-4.6"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "grok-fnr"
    litellm_params:
      model: "openai/grok-4-fast-non-reasoning"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "grok-fr"
    litellm_params:
      model: "openai/grok-4-fast-reasoning"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "gemini-pro"
    litellm_params:
      model: "openai/gemini-2.5-pro"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "gpt-codex"
    litellm_params:
      model: "openai/gpt-5.1-codex"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

litellm_settings:
  drop_params: True

general_settings:
  store_model_in_db: true
  store_prompts_in_spend_logs: true

litellm.sh

#!/usr/bin/env bash
CMD_BASE="$(readlink -f "$0" 2>/dev/null || greadlink-f "$0")" || CMD_BASE="$0"; CMD_BASE="$(dirname "$CMD_BASE")"

export LITELLM_MASTER_KEY="sk-9999999"
export LITELLM_SALT_KEY="sk-XxXxX"
export DATABASE_URL="postgresql://litellm:litellm-password@localhost:5432/litellm"

# Download [schema.prisma](https://github.com/BerriAI/litellm/blob/main/litellm/proxy/schema.prisma) into prisma/schema.prisma
# nvm install 18
# python -m prisma generate

litellm --config "$CMD_BASE"/litellm-config.yaml --port 4000

Tested Models

• Gemini-2.5-pro
• GLM-4.6
• Kimi-K2-thinking (even though it has tool_call = false)
• Grok-4-fast-reasoning

They all worked 🎉

Maybe there was something wrong in my previous LiteLLM install.

Session transcript

> What time is it? 

● Thinking...

● Bash(date)
  ⎿  Mon Nov 17 11:54:50 AM -03 2025

● Thinking...

  It's 11:54:50 AM on Monday, November 17, 2025 (local time, -03 timezone).

> @hello.sh check the TODO 
  ⎿  Read hello.sh (5 lines)

● Thinking...

  I can see the hello.sh file has a TODO comment on line 3 that says "show Hello, world!". The script currently doesn't implement this functionality - it
  just has the shebang line and the TODO comment.

  Would you like me to implement the TODO by adding the code to display "Hello, world!"?

> yes 

● Thinking...

● Update(hello.sh)
  ⎿  Updated hello.sh with 1 addition and 1 removal
       1    #!/bin/env bash
       2    
       3 -  # TODO show "Hello, world!"
       3 +  echo "Hello, world!"
       4    

● Thinking...

  Done! I've implemented the TODO by replacing the comment with echo "Hello, world!" on line 3.


Total cost:            $0.2428 (costs may be inaccurate due to usage of unknown models)
Total duration (API):  57s
Total duration (wall): 1m 49s
Total code changes:    1 line added, 1 line removed
Usage by model:
             glm-4.6:  72.4k input, 1.7k output, 0 cache read, 0 cache write ($0.2428)

[elifarley]

The test above was too simple. So I teamed up Grok and Gemini 2.5 PRO to create something more realistic.

Total cost:            $2.14 (costs may be inaccurate due to usage of unknown models)
Total duration (API):  5m 13s
Total duration (wall): 6h 8m 58s
Total code changes:    1389 lines added, 0 lines removed
Usage by model:
             grok-fr:  14.9k input, 1.6k output, 0 cache read, 0 cache write ($0.0686)
          gemini-pro:  567.5k input, 24.3k output, 0 cache read, 0 cache write ($2.07)

[elifarley]

I think we could provide an option to use LiteLLM via its docker image. We would setup an alias to claude that would call claude-poe.sh, which then would check if the LiteLLM docker container is running. if not, it would launch it with the necessary LiteLLM config. then it would run claude itself.
The downside is that we would add a dependency to Docker, but only if the user wants to.

[kamilio]

Yeah, that's a good plan!

Let me get the agent work on it and see what happens.

[elifarley]

Weird, all models I tried show up as healthy in the LiteLLM dashboard, except for GPT 5.1:

model_list:
  - model_name: "chaiku"
    litellm_params:
      model: "openai/claude-haiku-4.5"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "csonnet"
    litellm_params:
      model: "openai/claude-sonnet-4.5"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "copus"
    litellm_params:
      model: "openai/claude-opus-4.1"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "kimi-k2-t"
    litellm_params:
      model: "openai/kimi-k2-thinking"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "glm"
    litellm_params:
      model: "openai/glm-4.6"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"
      default_params:
        strategy: "deep"

  - model_name: "grok-fnr"
    litellm_params:
      model: "openai/grok-4-fast-non-reasoning"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"

  - model_name: "grok-fr"
    litellm_params:
      model: "openai/grok-4-fast-reasoning"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"
      default_params:
        reasoning_effort: "high"

  - model_name: "gpt"
    litellm_params:
      model: "openai/gpt-5.1"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"
      default_params:
        reasoning_effort: "high"

  - model_name: "gpt-codex"
    litellm_params:
      model: "openai/gpt-5.1-codex"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"
      default_params:
        reasoning_effort: "high"

  - model_name: "gemini-pro"
    litellm_params:
      model: "openai/gemini-3.0-pro"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"
      default_params:
        thinking_config:
          thinking_level: "high"
          include_thoughts: true

litellm_settings:
  drop_params: True
general_settings:
  store_model_in_db: true
  store_prompts_in_spend_logs: true

litellm.NotFoundError: NotFoundError: OpenAIException - 404: Not Found
stack trace: Traceback (most recent call last):
  File "/home/user/IdeaProjects/claude-code-helper/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/llm_http_handler.py", line 2209, in async_response_api_handler
    response = await async_httpx_client.post(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/IdeaProjects/claude-code-helper/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/log

It's probably not Poe's fault, as the curl below works fine:

curl "https://api.poe.com/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $POE_API_KEY" \
  -d '{
    "model": "gpt-5.1-codex",
    "messages": [
      {"role": "user", "content": "Hello world"}
    ]
  }'

I'll try to find out what's going on on LiteLLM's side.

Raw JSON for the model

{
  "model_name": "gpt-codex",
  "litellm_params": {
    "api_base": "https://api.poe.com/v1",
    "use_in_pass_through": false,
    "use_litellm_proxy": false,
    "merge_reasoning_content_in_choices": false,
    "model": "openai/gpt-5.1-codex",
    "default_params": {
      "reasoning_effort": "high"
    }
  },
  "model_info": {
    "id": "c0b6b5ce6960af384625485ce5435260d381cd4849d16ff42ac9aeeeee0209c2",
    "db_model": false,
    "access_via_team_ids": [
      "7389b839-b90e-4215-99d0-07d11e30ff62"
    ],
    "direct_access": true,
    "key": "gpt-5.1-codex",
    "max_tokens": 128000,
    "max_input_tokens": 272000,
    "max_output_tokens": 128000,
    "input_cost_per_token": 0.00000125,
    "input_cost_per_token_flex": null,
    "input_cost_per_token_priority": 0.0000025,
    "cache_creation_input_token_cost": null,
    "cache_creation_input_token_cost_above_200k_tokens": null,
    "cache_read_input_token_cost": 1.25e-7,
    "cache_read_input_token_cost_above_200k_tokens": null,
    "cache_read_input_token_cost_flex": null,
    "cache_read_input_token_cost_priority": 2.5e-7,
    "cache_creation_input_token_cost_above_1hr": null,
    "input_cost_per_character": null,
    "input_cost_per_token_above_128k_tokens": null,
    "input_cost_per_token_above_200k_tokens": null,
    "input_cost_per_query": null,
    "input_cost_per_second": null,
    "input_cost_per_audio_token": null,
    "input_cost_per_token_batches": null,
    "output_cost_per_token_batches": null,
    "output_cost_per_token": 0.00001,
    "output_cost_per_token_flex": null,
    "output_cost_per_token_priority": 0.00002,
    "output_cost_per_audio_token": null,
    "output_cost_per_character": null,
    "output_cost_per_reasoning_token": null,
    "output_cost_per_token_above_128k_tokens": null,
    "output_cost_per_character_above_128k_tokens": null,
    "output_cost_per_token_above_200k_tokens": null,
    "output_cost_per_second": null,
    "output_cost_per_video_per_second": null,
    "output_cost_per_image": null,
    "output_vector_size": null,
    "citation_cost_per_token": null,
    "tiered_pricing": null,
    "litellm_provider": "openai",
    "mode": "responses",
    "supports_system_messages": false,
    "supports_response_schema": true,
    "supports_vision": true,
    "supports_function_calling": true,
    "supports_tool_choice": true,
    "supports_assistant_prefill": null,
    "supports_prompt_caching": true,
    "supports_audio_input": null,
    "supports_audio_output": null,
    "supports_pdf_input": true,
    "supports_embedding_image_input": null,
    "supports_native_streaming": true,
    "supports_web_search": null,
    "supports_url_context": null,
    "supports_reasoning": true,
    "supports_computer_use": null,
    "search_context_cost_per_query": null,
    "tpm": null,
    "rpm": null,
    "ocr_cost_per_page": null,
    "annotation_cost_per_page": null,
    "supported_openai_params": [
      "logit_bias",
      "max_tokens",
      "max_completion_tokens",
      "modalities",
      "prediction",
      "n",
      "seed",
      "stream",
      "stream_options",
      "temperature",
      "tools",
      "tool_choice",
      "function_call",
      "functions",
      "max_retries",
      "extra_headers",
      "parallel_tool_calls",
      "audio",
      "web_search_options",
      "service_tier",
      "safety_identifier",
      "response_format",
      "reasoning_effort",
      "verbosity"
    ]
  },
  "provider": "openai",
  "input_cost": "1.25",
  "output_cost": "10.00",
  "litellm_model_name": "openai/gpt-5.1-codex",
  "max_tokens": 128000,
  "max_input_tokens": 272000,
  "api_base": "https://api.poe.com/v1",
  "cleanedLitellmParams": {
    "use_in_pass_through": false,
    "use_litellm_proxy": false,
    "merge_reasoning_content_in_choices": false,
    "default_params": {
      "reasoning_effort": "high"
    }
  }
}

[elifarley]

I spent too many Poe compute points with Sonnet, so I am trying to continue my development with cheapoer alternatives.

Tried different LLMs on my open source project:

• GPT-5.1, GPT-5.1-codex simply give me a 404 not found (LiteLLM bug perhaps);
• Gemini-3.0-Pro cause internal Python errors in LiteLLM;
• Grok4-fast-reasoning sends git commands to commit the changes but in a different format than what Claude code expects, so nothing is committed.

So, now I'm using Grok4-Fast-non-reasoning as a Haiku replacement, and GLM-4.6 with strategy: "deep" as a Sonnet replacement.

[elifarley]

Now, GPT-5.1 is showing up as healthy in LiteLLM dashboard, but when I try to use it, I get:

  - model_name: "gpt"
    litellm_params:
      model: "openai/gpt-5.1"
      api_base: "https://api.poe.com/v1"
      api_key: "os.environ/POE_API_KEY"
      default_params:
        reasoning_effort: "high"

/model gpt 
  ⎿ API error: 400 {"error":{"message":"litellm.BadRequestError: 
    OpenAIException - Invalid 'max_output_tokens': integer below minimum 
    value. Expected a value >= 16, but got 1 instead.. Received Model 
    Group=gpt\nAvailable Model Group Fallbacks=None","type":"invalid_request_e
    rror","param":"max_output_tokens","code":"400"}}

GPT-5.1-codex still shows up as unhealthy.

[Zugunruhekami]

I spent too many Poe compute points with Sonnet, so I am trying to continue my development with cheapoer alternatives.

Tried different LLMs on my open source project:

• GPT-5.1, GPT-5.1-codex simply give me a 404 not found (LiteLLM bug perhaps);
• Gemini-3.0-Pro cause internal Python errors in LiteLLM;
• Grok4-fast-reasoning sends git commands to commit the changes but in a different format than what Claude code expects, so nothing is committed.

So, now I'm using Grok4-Fast-non-reasoning as a Haiku replacement, and GLM-4.6 with strategy: "deep" as a Sonnet replacement.

Haven't done the real testing but I fear your errors might stem from LiteLLM version being too low with all these "new" models you were testing? And for gemini, my experience is that you have to install a google-genai package.

Btw thanks for testing, both your mcp and poe-code looks like a dream-come-true to poe users.