Skip to content

Commit 145cb04

Browse files
alchemystackalchemystack
committed
open-webui (zenflow cdd61e63)
I really like this https://github.com/open-webui/open-webui I want you to think about the best way to integrate it into users experience with the qr-sampler testing and running the vllm. Think thoroughly how this could be part of the setup and default interaction with the running vllm docker (without being the ultimate goal – it's expected that people would also run the qr-sampler in other ways). Then give me the best options for me to select from and refine with my input.
1 parent 22be263 commit 145cb04

17 files changed

Lines changed: 920 additions & 0 deletions

File tree

‎.zenflow/tasks/open-webui-cdd6/plan.md‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
# Spec and build
2+
3+
## Configuration
4+
- **Artifacts Path**: {@artifacts_path} → `.zenflow/tasks/{task_id}`
5+
6+
---
7+
8+
## Agent Instructions
9+
10+
Ask the user questions when anything is unclear or needs their input. This includes:
11+
- Ambiguous or incomplete requirements
12+
- Technical decisions that affect architecture or user experience
13+
- Trade-offs that require business context
14+
15+
Do not make assumptions on important decisions — get clarification first.
16+
17+
If you are blocked and need user clarification, mark the current step with `[!]` in plan.md before stopping.
18+
19+
---
20+
21+
## Workflow Steps
22+
23+
### [x] Step: Technical Specification
24+
<!-- chat-id: f88e43b4-91a7-4a02-aef2-b0adb1e42274 -->
25+
26+
Assess the task's difficulty, as underestimating it leads to poor outcomes.
27+
- easy: Straightforward implementation, trivial bug fix or feature
28+
- medium: Moderate complexity, some edge cases or caveats to consider
29+
- hard: Complex logic, many caveats, architectural considerations, or high-risk changes
30+
31+
Create a technical specification for the task that is appropriate for the complexity level:
32+
- Review the existing codebase architecture and identify reusable components.
33+
- Define the implementation approach based on established patterns in the project.
34+
- Identify all source code files that will be created or modified.
35+
- Define any necessary data model, API, or interface changes.
36+
- Describe verification steps using the project's test and lint commands.
37+
38+
Save the output to `{@artifacts_path}/spec.md` with:
39+
- Technical context (language, dependencies)
40+
- Implementation approach
41+
- Source code structure changes
42+
- Data model / API / interface changes
43+
- Verification approach
44+
45+
If the task is complex enough, create a detailed implementation plan based on `{@artifacts_path}/spec.md`:
46+
- Break down the work into concrete tasks (incrementable, testable milestones)
47+
- Each task should reference relevant contracts and include verification steps
48+
- Replace the Implementation step below with the planned tasks
49+
50+
Rule of thumb for step size: each step should represent a coherent unit of work (e.g., implement a component, add an API endpoint, write tests for a module). Avoid steps that are too granular (single function).
51+
52+
Important: unit tests must be part of each implementation task, not separate tasks. Each task should implement the code and its tests together, if relevant.
53+
54+
Save to `{@artifacts_path}/plan.md`. If the feature is trivial and doesn't warrant this breakdown, keep the Implementation step below as is.
55+
56+
---
57+
58+
### [x] Step: Add Open WebUI to deployment profiles
59+
<!-- chat-id: aa795d94-e91c-4472-9b84-8f541230597b -->
60+
61+
Add the `open-webui` service with `profiles: ["ui"]` to all three deployment profiles, plus env vars and volume.
62+
63+
- Add `open-webui` service to `deployments/urandom/docker-compose.yml`
64+
- Add `open-webui-data` named volume to `deployments/urandom/docker-compose.yml`
65+
- Add `OPEN_WEBUI_PORT` and `OPEN_WEBUI_AUTH` to `deployments/urandom/.env.example`
66+
- Repeat for `deployments/firefly-1/`
67+
- Repeat for `deployments/_template/`
68+
- Verify: `docker compose --profile ui config` passes in each profile directory (syntax check)
69+
- Verify: `docker compose config` (no profile) does NOT include `open-webui` service
70+
71+
---
72+
73+
### [x] Step: Create qr-sampler Filter Function for Open WebUI
74+
<!-- chat-id: 164157f9-f9d8-4275-82b1-12a9869c3d72 -->
75+
76+
Write the Open WebUI Filter Function that injects `qr_*` per-request parameters into vLLM requests via Valves.
77+
78+
- Create `examples/open-webui/qr_sampler_filter.py` with:
79+
- `Pipeline` class with `type = "filter"`
80+
- `Valves` inner class exposing per-request fields from `_PER_REQUEST_FIELDS` in `config.py`
81+
- `inlet()` method that injects `qr_*` keys as top-level body fields
82+
- `outlet()` passthrough
83+
- Proper docstrings and metadata header (title, author, version, description)
84+
- Create `examples/open-webui/qr_sampler_filter.json` — Open WebUI importable JSON containing the filter source
85+
- Verify: JSON is valid and contains the complete Python source
86+
- Verify: Valve field names match the `qr_*` keys that `resolve_config()` accepts (cross-reference `_PER_REQUEST_FIELDS`)
87+
88+
---
89+
90+
### [x] Step: Update documentation
91+
<!-- chat-id: 741bf761-a443-4e71-97ed-4725a466fc90 -->
92+
93+
Update all READMEs to document the Open WebUI integration and filter function.
94+
95+
- Add "Web UI (optional)" section to `deployments/urandom/README.md`
96+
- Add "Web UI (optional)" section to `deployments/firefly-1/README.md`
97+
- Add "Web UI (optional)" section to `deployments/_template/README.md`
98+
- Update `deployments/README.md` quick start to mention `--profile ui`
99+
- Create `examples/open-webui/README.md` with filter function docs (import steps, Valves config, architecture)
100+
- Add prominent "Web UI" section to main `README.md` recommending Open WebUI
101+
- Write report to `{@artifacts_path}/report.md`
102+

‎.zenflow/tasks/open-webui-cdd6/report.md‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Report: Update Documentation for Open WebUI Integration
2+
3+
## Summary
4+
5+
Added comprehensive documentation for the Open WebUI integration across all READMEs in the project. The documentation follows a layered approach: high-level mention in the main README, profile-specific instructions in each deployment README, and a detailed guide in the `examples/open-webui/` directory.
6+
7+
## Changes Made
8+
9+
### Modified Files
10+
11+
1. **`README.md`** (root)
12+
- Added a "Web UI" section between Quick Start and Configuration Reference
13+
- Documents `--profile ui` usage, filter function import, and links to detailed docs
14+
- Includes a note that Open WebUI is entirely optional
15+
- Updated the project structure tree to include `examples/open-webui/`
16+
17+
2. **`deployments/README.md`**
18+
- Added `--profile ui` instructions to the Quick Start section
19+
- Links to the filter function docs in `examples/open-webui/`
20+
21+
3. **`deployments/urandom/README.md`**
22+
- Added "Web UI (optional)" section with full setup instructions
23+
- Includes filter function import steps, Valves configuration, and port/auth customization
24+
- Links to the detailed guide in `examples/open-webui/README.md`
25+
26+
4. **`deployments/firefly-1/README.md`**
27+
- Added "Web UI (optional)" section (same structure as urandom)
28+
- Placed after the "Testing the connection" section
29+
30+
5. **`deployments/_template/README.md`**
31+
- Added "Web UI (optional)" section (shorter, since this is a template)
32+
- Links to `examples/open-webui/` for filter import instructions
33+
34+
### Created Files
35+
36+
6. **`examples/open-webui/README.md`**
37+
- Comprehensive guide for the Open WebUI integration
38+
- Covers: starting Open WebUI, importing the filter function (JSON import and manual paste), all Valve parameters organized by category, how the request flow works, what is NOT controlled by the filter, disabling the filter, port customization, and authentication
39+
40+
## Documentation Architecture
41+
42+
```
43+
README.md High-level "Web UI" section
44+
| (2 paragraphs + import steps + link)
45+
|
46+
deployments/README.md Quick start mentions --profile ui
47+
|
48+
+-- urandom/README.md Full "Web UI (optional)" section
49+
+-- firefly-1/README.md Full "Web UI (optional)" section
50+
+-- _template/README.md Brief "Web UI (optional)" section
51+
|
52+
examples/open-webui/README.md Detailed filter function guide
53+
(all Valves, request flow, troubleshooting)
54+
```
55+
56+
Each level links down to the next for users who want more detail, avoiding duplication while ensuring discoverability at every entry point.
57+
58+
## Key Design Decisions
59+
60+
- **"(optional)" in section titles**: Reinforces that Open WebUI is not required
61+
- **Filter import JSON method first**: Easiest path for non-technical users, with paste-source as an alternative
62+
- **Valve tables organized by category**: Matches the grouping in the filter source code (filter control, token selection, temperature, signal amplification, logging)
63+
- **Infrastructure exclusion note**: Explicitly documents that gRPC/fallback settings are NOT available as Valves, preventing user confusion
64+
- **Consistent port/auth table**: Same format across all profile READMEs for quick reference

‎.zenflow/tasks/open-webui-cdd6/spec.md‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
# Technical Specification: Open WebUI Integration with qr-sampler
2+
3+
## Difficulty: Medium
4+
5+
The core integration is straightforward (adding an Open WebUI service to Docker Compose), but doing it well requires careful thought about parameter passthrough, user experience, and maintaining qr-sampler's deployment flexibility.
6+
7+
## Technical Context
8+
9+
- **Language**: Python 3.10+, YAML (Docker Compose), Markdown
10+
- **Existing infrastructure**: Deployment profiles in `deployments/` with Docker Compose + `.env.example` + README pattern
11+
- **Key dependency**: Open WebUI (`ghcr.io/open-webui/open-webui:main`) — a self-hosted ChatGPT-style web UI
12+
- **Connection method**: Open WebUI connects to vLLM via its OpenAI-compatible API (`/v1` endpoint)
13+
- **Parameter flow**: Open WebUI request → Filter Function `inlet()` injects `qr_*` keys → vLLM `/v1/chat/completions` → `SamplingParams.extra_args` → qr-sampler `resolve_config()`
14+
15+
## Decisions (User-Confirmed)
16+
17+
1. **Option A selected**: Add Open WebUI to every deployment profile using Docker Compose `profiles: ["ui"]`
18+
2. **Filter Function included**: Ship a pre-built Open WebUI Filter Function for qr-sampler parameter control via admin Valves UI
19+
3. **README prominence**: Add a recommended "Try the Web UI" section to the main README
20+
21+
## Architecture Overview
22+
23+
```
24+
┌─────────────────────┐
25+
│ Open WebUI │ ← Users chat here (port 3000)
26+
│ (profiles: ["ui"]) │
27+
└──────────┬──────────┘
28+
│ HTTP (OpenAI-compatible)
29+
│
30+
│ Filter Function injects qr_* keys
31+
│ into request body before forwarding
32+
│
33+
┌──────────▼──────────┐ gRPC ┌──────────────────┐
34+
│ vLLM │ ◄────────────► │ Entropy Server │
35+
│ + qr-sampler │ │ (optional) │
36+
│ (port 8000) │ │ (port 50051) │
37+
└─────────────────────┘ └──────────────────┘
38+
```
39+
40+
## Implementation Approach
41+
42+
### Part 1: Docker Compose profiles (all deployment profiles)
43+
44+
Add an `open-webui` service with `profiles: ["ui"]` to each `docker-compose.yml`. This ensures:
45+
- `docker compose up` — unchanged behavior, Open WebUI does NOT start
46+
- `docker compose --profile ui up` — starts Open WebUI alongside everything else
47+
48+
Service definition (identical across profiles):
49+
50+
```yaml
51+
open-webui:
52+
image: ghcr.io/open-webui/open-webui:main
53+
profiles: ["ui"]
54+
ports:
55+
- "${OPEN_WEBUI_PORT:-3000}:8080"
56+
environment:
57+
OPENAI_API_BASE_URL: "http://vllm:8000/v1"
58+
OPENAI_API_KEY: "unused"
59+
WEBUI_AUTH: "${OPEN_WEBUI_AUTH:-false}"
60+
volumes:
61+
- open-webui-data:/app/backend/data
62+
depends_on:
63+
- vllm
64+
restart: unless-stopped
65+
```
66+
67+
Plus `open-webui-data:` in the `volumes:` section.
68+
69+
### Part 2: Open WebUI Filter Function for qr-sampler
70+
71+
Open WebUI stores functions in its SQLite database. They cannot be auto-loaded from `.py` files. The approach:
72+
73+
1. **Ship the filter as two files**:
74+
- `examples/open-webui/qr_sampler_filter.py` — the source code (readable, editable)
75+
- `examples/open-webui/qr_sampler_filter.json` — Open WebUI import-ready JSON format
76+
77+
2. **Import workflow** (documented in README):
78+
- Open http://localhost:3000 → Admin Panel → Functions
79+
- Click Import → select `qr_sampler_filter.json`
80+
- Toggle "Global" to apply to all models
81+
- Configure parameters via the Valves gear icon
82+
83+
3. **Filter function design**:
84+
- Type: `filter` with `inlet()` method
85+
- Valves expose all per-request-overridable qr-sampler fields from `_PER_REQUEST_FIELDS`
86+
- `inlet()` injects `qr_*` keys as top-level fields in the request body
87+
- vLLM maps unknown top-level keys to `SamplingParams.extra_args`
88+
- qr-sampler's `resolve_config()` picks them up transparently
89+
90+
**Valve fields** (matching `_PER_REQUEST_FIELDS` in `config.py`):
91+
92+
| Valve | Type | Default | Maps to |
93+
|-------|------|---------|---------|
94+
| `enable_qr_sampling` | `bool` | `True` | (controls whether filter injects anything) |
95+
| `temperature_strategy` | `Literal["fixed", "edt"]` | `"fixed"` | `qr_temperature_strategy` |
96+
| `fixed_temperature` | `float` | `0.7` | `qr_fixed_temperature` |
97+
| `top_k` | `int` | `50` | `qr_top_k` |
98+
| `top_p` | `float` | `0.9` | `qr_top_p` |
99+
| `sample_count` | `int` | `20480` | `qr_sample_count` |
100+
| `log_level` | `Literal["none", "summary", "full"]` | `"summary"` | `qr_log_level` |
101+
| `diagnostic_mode` | `bool` | `False` | `qr_diagnostic_mode` |
102+
103+
Infrastructure fields (`entropy_source_type`, `grpc_*`, `fallback_mode`) are deliberately excluded — they cannot change per-request and are controlled by environment variables.
104+
105+
### Part 3: Documentation
106+
107+
- **`deployments/README.md`**: Add `--profile ui` to the quick start section
108+
- **Each profile's README**: Add "Web UI (optional)" section with usage + filter import instructions
109+
- **`README.md`**: Add a prominent "Web UI" section recommending Open WebUI, linking to the filter function and deployment docs
110+
- **`examples/open-webui/README.md`**: Detailed guide for the filter function (what it does, how to import, how to configure Valves)
111+
112+
## Files to Create
113+
114+
| File | Purpose |
115+
|------|---------|
116+
| `examples/open-webui/qr_sampler_filter.py` | Human-readable filter source code |
117+
| `examples/open-webui/qr_sampler_filter.json` | Open WebUI importable JSON |
118+
| `examples/open-webui/README.md` | Filter function documentation |
119+
120+
## Files to Modify
121+
122+
| File | Change |
123+
|------|--------|
124+
| `deployments/urandom/docker-compose.yml` | Add `open-webui` service + volume |
125+
| `deployments/urandom/.env.example` | Add `OPEN_WEBUI_PORT`, `OPEN_WEBUI_AUTH` |
126+
| `deployments/urandom/README.md` | Add "Web UI (optional)" section |
127+
| `deployments/firefly-1/docker-compose.yml` | Add `open-webui` service + volume |
128+
| `deployments/firefly-1/.env.example` | Add `OPEN_WEBUI_PORT`, `OPEN_WEBUI_AUTH` |
129+
| `deployments/firefly-1/README.md` | Add "Web UI (optional)" section |
130+
| `deployments/_template/docker-compose.yml` | Add `open-webui` service + volume |
131+
| `deployments/_template/.env.example` | Add `OPEN_WEBUI_PORT`, `OPEN_WEBUI_AUTH` |
132+
| `deployments/_template/README.md` | Mention UI option |
133+
| `deployments/README.md` | Add `--profile ui` to quick start |
134+
| `README.md` | Add prominent "Web UI" section |
135+
136+
## Verification
137+
138+
1. **Compose syntax**: `docker compose --profile ui config` in each profile directory — validates YAML
139+
2. **Default behavior**: `docker compose config` (no profile) — confirm Open WebUI is NOT listed in resolved services
140+
3. **Filter function**: Verify `qr_sampler_filter.json` is valid JSON and contains the full source code
141+
4. **Filter Valves**: Verify all Valve field names match the `qr_*` keys that `resolve_config()` accepts
142+
5. **Manual test**: If Docker + GPU available — `docker compose --profile ui up`, open `http://localhost:3000`, import filter, chat, check vLLM logs for qr-sampler activity
143+
144+
## Data Model / API / Interface Changes
145+
146+
None. This is a deployment infrastructure + documentation addition. No Python source code in `src/qr_sampler/` is modified. No tests are affected. The filter function is an Open WebUI plugin, not part of the qr-sampler package.

‎README.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -150,6 +150,44 @@ curl http://localhost:8000/v1/completions \
150150

151151
---
152152

153+
## Web UI
154+
155+
qr-sampler works with [Open WebUI](https://github.com/open-webui/open-webui), a
156+
self-hosted ChatGPT-style interface that connects to vLLM's OpenAI-compatible
157+
API. Every deployment profile includes it as an optional service — add
158+
`--profile ui` to start it alongside vLLM:
159+
160+
```bash
161+
cd deployments/urandom
162+
docker compose --profile ui up --build
163+
```
164+
165+
Then open http://localhost:3000 to start chatting. Without `--profile ui`, Open
166+
WebUI does not start and nothing changes.
167+
168+
### Controlling qr-sampler from the UI
169+
170+
A pre-built [filter function](examples/open-webui/) injects qr-sampler
171+
per-request parameters into every chat message via the Open WebUI Valves system.
172+
This lets you adjust temperature, top-k, top-p, sample count, and other sampling
173+
parameters from the admin panel without editing environment variables or writing
174+
API calls.
175+
176+
To set it up:
177+
178+
1. Go to **Admin Panel > Functions** in Open WebUI.
179+
2. Click **Import** and select [`examples/open-webui/qr_sampler_filter.json`](examples/open-webui/qr_sampler_filter.json).
180+
3. Toggle the function to **Global**.
181+
4. Click the **gear icon** to adjust parameters.
182+
183+
See [`examples/open-webui/README.md`](examples/open-webui/README.md) for the
184+
full guide, including all available Valve parameters and how the filter works.
185+
186+
> Open WebUI is entirely optional. qr-sampler works the same way with direct API
187+
> calls, `curl`, Python clients, or any OpenAI-compatible tool.
188+
189+
---
190+
153191
## Configuration reference
154192

155193
All configuration is done via environment variables with the `QR_` prefix. Per-request overrides use the `qr_` prefix in `extra_args`.
@@ -645,6 +683,10 @@ examples/
645683
│ ├── simple_urandom_server.py # Minimal reference server (~50 lines)
646684
│ ├── timing_noise_server.py # CPU timing entropy server
647685
│ └── qrng_template_server.py # Annotated template for custom QRNGs
686+
├── open-webui/
687+
│ ├── qr_sampler_filter.py # Open WebUI filter function (source)
688+
│ ├── qr_sampler_filter.json # Open WebUI importable JSON
689+
│ └── README.md # Filter function docs
648690
├── docker/
649691
│ ├── Dockerfile.vllm # vLLM + qr-sampler image (build-time install)
650692
│ └── Dockerfile.entropy-server # Docker image for entropy servers

‎deployments/README.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,17 @@ cp .env.example .env
3535
docker compose up --build
3636
```
3737

38+
To also start [Open WebUI](https://github.com/open-webui/open-webui) (a
39+
ChatGPT-style web interface), add `--profile ui`:
40+
41+
```bash
42+
docker compose --profile ui up --build
43+
```
44+
45+
Then open http://localhost:3000. See each profile's README for UI setup details
46+
and the optional [qr-sampler filter function](../examples/open-webui/) for
47+
controlling sampling parameters from the UI.
48+
3849
## Creating your own profile
3950

4051
1. Copy the template:

‎deployments/_template/.env.example‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,3 +43,8 @@ QR_LOG_LEVEL=summary
4343
# --- Ports (host-side) ---
4444
VLLM_PORT=8000
4545
# ENTROPY_SERVER_PORT=50051 # Uncomment if using a co-located server
46+
47+
# --- Open WebUI (optional, --profile ui) ---
48+
OPEN_WEBUI_PORT=3000
49+
# Set to true to require login (recommended for shared/public servers).
50+
OPEN_WEBUI_AUTH=false

0 commit comments

Comments
 (0)