Issue with Magentic Marketplace Simulation Stuck in Retry Loop

[liqi8869-maker]

As a student and AI beginner, I find this project truly impressive. Thank you for your outstanding contributions.

However, I've been unable to get it running, which is quite frustrating. After reviewing several existing issues, I noticed that most problems seem related to the adaptation and invocation of large models.

I would be very grateful if a more updated version could be provided. I'm really looking forward to experiencing this amazing project!

#Environment Setup

Platform with Docker restrictions

PostgreSQL managed via Conda

Local LLM served with vllm

Configuration Details

bash

Local model serving

vllm serve /dssg/home/acct-luymin/yelu/.cache/modelscope/hub/models/Qwen/Qwen2.5-7B-Instruct --host 0.0.0.0 --port 8000

Environment variables

LLM_PROVIDER=openai
LLM_MODEL=Qwen2.5-7B-Instruct
LLM_API_BASE=http://localhost:8000/v1
LLM_API_KEY=token-abc123
db_type="postgres"
LLM_MAX_CONCURRENCY=2
LLM_TIMEOUT=60
LLM_TEMPERATURE=0.3

Problem Description
The simulation gets stuck in an infinite retry loop after agent initialization, preventing any meaningful marketplace interactions.

Steps to Reproduce

Start vllm server with Qwen2.5-7B-Instruct model

Set environment variables as shown

Run: python ./experiments/example.py

Actual Behavior
Agents initialize successfully but then enter endless retry cycle. My debug code reveals the system is stuck in retry loops:

text
INFO [customer_0002] Starting autonomous shopping agent
INFO [openai._base_client] Retrying request to /chat/completions in 0.379s
INFO [openai._base_client] Retrying request to /chat/completions in 0.489s

Additional Context

Incomplete data files can be generated

PostgreSQL managed via Conda due to Docker restrictions

vllm server confirmed running at localhost:8000

The retry loop behavior was identified through custom debug instrumentation

[zachary62]

Hi @liqi8869-maker
For the terminal where you "Start vllm server with Qwen2.5-7B-Instruct model", could you share its output?

[Wenyueh]

Can you start using the below command?
vllm serve 'Qwen/Qwen2.5-7B-Instruct' --guided-decoding-backend outlines --tensor-parallel-size $number_of_GPU --port 8001

"--guided-decoding-backend outlines" is required for constraint generation, which is throughout the simulation

wait until it shows application is ready, and then run the simulation

[liqi8869-maker]

你能开始使用下面的命令吗？ vllm serve 'Qwen/Qwen2.5-7B-Instruct' --guided-decoding-backend outlines --tensor-parallel-size $number_of_GPU --port 8001

“--guided-decoding-backend outlines”是约束生成所必需的，这在整个模拟过程中

等到它显示应用程序已准备就绪，然后运行模拟

Title: Seeking Documentation for Deploying Open-Source Models in Rigid LLM Provider Framework

Description:

I am encountering significant challenges when attempting to deploy open-source models within the current system's constrained LLM provider framework. Despite extensive testing, all deployment approaches have failed due to the system's strict validation requirements.

Testing Summary:

1. The experiment using local deployment (Ollama with models qwen3-vl:2b and gemini3:1b) failed.
2. The experiment using server deployment (vLLM with models gpt-oss:20b and qwen3-vl-14b) also failed.
3. While the experiment using standard API calls (e.g., to Gemini 2.5 Flash) could be completed, Each failed attempt initiates duplicate API requests, leading to abnormal token consumption due to numerous invalid calls and process crashes. As a student, I simply cannot bear the substantial API costs resulting from this. This issue makes it exceptionally difficult to proceed with the experiment sustainably.

Core Issue:
The system enforces strict validation, limiting usage to only three LLM providers (openai, gemini, or anthropic) with mandatory official API key authentication. The environment variables appear to be hardcoded:

• GEMINI_API_KEY="todo"

• LLM_PROVIDER="openai"

Any non-standard modifications result in validation errors.

Error Examples:

1. Provider Validation Error:

text

Error: Validation error for BaseLLMConfig
provider: Input should be 'openai', 'gemini' or 'anthropic'
[type=literal_error, input_value='ollama', input_type=str]

2. API Base Redirection Failure:

• Attempted: OPENAI_API_BASE="http://localhost:11434/v1"

• Attempted: OPENAI_API_BASE="http://localhost:11434"

• Result: No communication traffic detected on ports, leading to "LLM decision failed" errors

Additional Testing:

• Attempted to implement project wrapping with proxy server redirection, but was unsuccessful.

• Verified model and port functionality through independent code testing

• Confirmed models and communication ports are operational outside the system

Request:
Could you please provide documentation or reference configurations for deploying open-source models like the mentioned gpt-oss:20b? Specifically, I'm seeking:

1. Working examples of open-source model integration

2. Configuration guidelines for local/server deployments

3. Any known workarounds for the provider validation constraints

I would sincerely appreciate a clear, functional example. It would be crucial for continuing my research in a cost-effective manner.

[Wenyueh]

To support vLLM, we use openai-compatible APIs for it.

After running the vllm serve ... command,

You should set the environment variables as below:

export LLM_MAX_TOKENS=8192
export OPENAI_API_KEY=dummy # no api key needed
export LLM_PROVIDER="openai" # openai-compatible API for vllm support
export LLM_MODEL='openai/gpt-oss-20b'
export OPENAI_BASE_URL="http://localhost:8001/v1"
export LLM_MAX_CONCURRENCY=256 # change to any number that your GPU can support