Getting 400 Client Error: Bad Request when running sweep.py

[Filip-Anicic]

running python sweep.py --config ./configs/my_sweep_14B.json, I constantly get 400 Bad request from llama-server

[1/12] 14B Q8_0 (baseline)
  [llama.cpp] Starting server on 127.0.0.1:8080
  Command: /home/filip/Development/Builds/llama.cpp/build/bin/llama-server -m /home/filip/.lmstudio/models/lmstudio-community/Qwen2.5-14B-Instruct-GGUF/Qwen2.5-14B-Instruct-Q8_0.gguf --host 127.0.0.1 --port 8080 -ngl 99 -c 4096
  [llama.cpp] Server pid=46706
  Waiting for server ...
  Server ready. Benchmarking ...
  [14B Q8_0 baseline] request 1/3  ERROR: 400 Client Error: Bad Request for url: http://127.0.0.1:8080/v1/chat/completions
  [14B Q8_0 baseline] request 2/3  ERROR: 400 Client Error: Bad Request for url: http://127.0.0.1:8080/v1/chat/completions
  [14B Q8_0 baseline] request 3/3  ERROR: 400 Client Error: Bad Request for url: http://127.0.0.1:8080/v1/chat/completions
  Server stopped (pid 46706)
  Result: 0 tok/s

my_sweep_14B.json

{
  "name": "qwen2.5-14B",
  "hardware": "9070 XT",
  "backend": "llamacpp",
  "model_family": "Qwen2.5",

  "targets": [
    {"label": "14B Q8_0",     "path": "/home/filip/.lmstudio/models/lmstudio-community/Qwen2.5-14B-Instruct-GGUF/Qwen2.5-14B-Instruct-Q8_0.gguf"},
    {"label": "14B Q4_K_M",   "path": "/home/filip/.lmstudio/models/lmstudio-community/Qwen2.5-14B-Instruct-GGUF/Qwen2.5-14B-Instruct-Q4_K_M.gguf"}
  ],
  "drafts": [
    {"label": "0.5B Q8_0",    "path": "/home/filip/.lmstudio/models/lmstudio-community/Qwen2.5-Coder-0.5B-Instruct-GGUF/Qwen2.5-Coder-0.5B-Instruct-Q8_0.gguf"},
    {"label": "1.5B Q8_0",    "path": "/home/filip/.lmstudio/models/lmstudio-community/Qwen2.5-1.5B-Instruct-GGUF/Qwen2.5-1.5B-Instruct-Q8_0.gguf"},
    {"label": "1.5B Q4_K_M",  "path": "/home/filip/.lmstudio/models/lmstudio-community/Qwen2.5-1.5B-Instruct-GGUF/Qwen2.5-1.5B-Instruct-Q4_K_M.gguf"},
    {"label": "3B Q8_0",      "path": "/home/filip/.lmstudio/models/lmstudio-community/Qwen2.5-3B-Instruct-GGUF/Qwen2.5-3B-Instruct-Q8_0.gguf"},
    {"label": "3B Q4_K_M",    "path": "/home/filip/.lmstudio/models/lmstudio-community/Qwen2.5-3B-Instruct-GGUF/Qwen2.5-3B-Instruct-Q4_K_M.gguf"}
  ],
  "settings": {
    "llama_bin": "/home/filip/Development/Builds/llama.cpp/build/bin/llama-server",
    "runs": 1,
    "max_tokens": 1024,
    "temperature": 0.0,
    "gpu_layers": 99,
    "ctx_size": 4096,
    "port": 8080
  }
}

I've double checked my config paths, but they seem correct. Any help would be appreciated

[Filip-Anicic]

I don't know what changed, but now I sometimes get 1/3 or 2/3 running benchmarks. For now I'll let it run and hopefully get some useful data.

[1/12] 14B Q8_0 (baseline)
  [llama.cpp] Starting server on 127.0.0.1:8080
  Command: /home/filip/Development/Builds/llama.cpp/build/bin/llama-server -m /home/filip/.lmstudio/models/lmstudio-community/Qwen2.5-14B-Instruct-GGUF/Qwen2.5-14B-Instruct-Q8_0.gguf --host 127.0.0.1 --port 8080 -ngl 99 -c 4096
  [llama.cpp] Server pid=55858
  Waiting for server ...
  Server ready. Benchmarking ...
  [14B Q8_0 baseline] request 1/3  ttft=0.937s  tps=9.9  tokens=1024
  [14B Q8_0 baseline] request 2/3  ERROR: 400 Client Error: Bad Request for url: http://127.0.0.1:8080/v1/chat/completions
  [14B Q8_0 baseline] request 3/3  ERROR: 400 Client Error: Bad Request for url: http://127.0.0.1:8080/v1/chat/completions
  Server stopped (pid 55858)
  Result: 9.95 tok/s

[2/12] 14B Q8_0 + 0.5B Q8_0
  [llama.cpp (speculative)] Starting server on 127.0.0.1:8080
  Command: /home/filip/Development/Builds/llama.cpp/build/bin/llama-server -m /home/filip/.lmstudio/models/lmstudio-community/Qwen2.5-14B-Instruct-GGUF/Qwen2.5-14B-Instruct-Q8_0.gguf --host 127.0.0.1 --port 8080 -ngl 99 -c 4096 --model-draft /home/filip/.lmstudio/models/lmstudio-community/Qwen2.5-Coder-0.5B-Instruct-GGUF/Qwen2.5-Coder-0.5B-Instruct-Q8_0.gguf
  [llama.cpp (speculative)] Server pid=56091
  Waiting for server ...
  Server ready. Benchmarking ...
  [14B Q8_0 + 0.5B Q8_0] request 1/3  ttft=0.273s  tps=31.0  tokens=1024
  [14B Q8_0 + 0.5B Q8_0] request 2/3  ttft=1.106s  tps=9.6  tokens=1024
  [14B Q8_0 + 0.5B Q8_0] request 3/3  ERROR: 400 Client Error: Bad Request for url: http://127.0.0.1:8080/v1/chat/completions
  Server stopped (pid 56091)
  Result: 20.3 tok/s

[3/12] 14B Q8_0 + 1.5B Q8_0
  [llama.cpp (speculative)] Starting server on 127.0.0.1:8080
  Command: /home/filip/Development/Builds/llama.cpp/build/bin/llama-server -m /home/filip/.lmstudio/models/lmstudio-community/Qwen2.5-14B-Instruct-GGUF/Qwen2.5-14B-Instruct-Q8_0.gguf --host 127.0.0.1 --port 8080 -ngl 99 -c 4096 --model-draft /home/filip/.lmstudio/models/lmstudio-community/Qwen2.5-1.5B-Instruct-GGUF/Qwen2.5-1.5B-Instruct-Q8_0.gguf
  [llama.cpp (speculative)] Server pid=56553
  Waiting for server ...
  Server ready. Benchmarking ...
  [14B Q8_0 + 1.5B Q8_0] request 1/3  ERROR: 400 Client Error: Bad Request for url: http://127.0.0.1:8080/v1/chat/completions
  [14B Q8_0 + 1.5B Q8_0] request 2/3  ERROR: 400 Client Error: Bad Request for url: http://127.0.0.1:8080/v1/chat/completions
  [14B Q8_0 + 1.5B Q8_0] request 3/3  ERROR: 400 Client Error: Bad Request for url: http://127.0.0.1:8080/v1/chat/completions
  Server stopped (pid 56553)
  Result: 0 tok/s