Conversation
- Makefile test-pg: use --wait instead of sleep 5 - Makefile docker: use --wait instead of sleep 10 - QA_RUNBOOK.md: update both Docker steps to use --wait The --wait flag waits for service health checks to pass, making tests deterministic on slower machines and CI.
- test-pg: wrap in bash with trap to run cleanup on EXIT - docker: wrap in bash with trap to run cleanup on EXIT - Add docker-clean target for manual cleanup of leftover containers The trap pattern ensures docker compose down runs regardless of whether pytest/smoke tests pass or fail.
- Add StubBackend class that returns canned responses for testing - Add STUB enum value to ModelBackend in config.py - Wire stub backend into LLMService._build_backend() - Update docker-compose.test.yml to use MODEL_BACKEND=stub - Update smoke_test.sh to require 200 from chat (not 503) This makes chat smoke tests deterministic without requiring a real LLM.
The runtime resolves model_backend as:
sys_settings.get("model_backend") or settings.model_backend
This meant DB defaults (which seed "openai") would override the
MODEL_BACKEND env var, causing the stub backend to not be used.
Fix: bootstrap_admin.py now syncs MODEL_BACKEND env to system_settings
if set, ensuring the test stack actually uses the stub backend.
The admin settings validator rejected 'stub' as a model_backend value, which would prevent setting it via the API. Added to allowed values with a comment noting it's for testing only.
The Makefile uses `docker compose --wait` which requires Compose v2.1+. Added version requirements and a note about checking/upgrading.
Tests ensure StubBackend doesn't regress: - generate() returns canned response with usage stats - generate_stream() yields tokens then message_done event
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Adds a testing-only
stubmodel backend and stabilizes QA and Docker-based workflows.StubBackendwith canned responses and streaming; integrated viaLLMService._build_backendand allowed in/admin/settings(model_backendnow includesstub).docker-compose.test.ymlsetsMODEL_BACKEND=stub;bootstrap_admin.pysyncsMODEL_BACKENDintosystem_settingsfor runtime.scripts/smoke_test.shnow requires 200 and checks for the stub response; new unit tests intests/test_stub_backend.py.test-pganddockertargets usetrap-based cleanup and--wait; addsdocker-cleantarget.DATABASE_URLscheme topostgresql://; QA runbook updated for Compose--wait; deployment and OpenBSD env examples adjusted accordingly.Written by Cursor Bugbot for commit 6d4ebe2. This will update automatically on new commits. Configure here.