Cache Matrix0 parameter count for health endpoint#104
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR improves the performance of the /health endpoint by caching the Matrix0 model parameter count instead of recreating the model on each request.
- Introduces caching mechanism for Matrix0 model parameter count
- Refactors
/healthendpoint to use cached values instead of instantiating PolicyValueNet repeatedly - Adds comprehensive test coverage to verify the caching behavior works correctly
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| webui/server.py | Adds parameter count caching logic and refactors health endpoint to use cached values |
| tests/test_webui_server.py | Adds test to verify health endpoint caches model parameter count and doesn't rebuild model unnecessarily |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| def _get_matrix0_param_count() -> Optional[int]: | ||
| """Return the cached parameter count, computing it lazily if needed.""" | ||
|
|
||
| global _matrix0_model_params, _cfg |
There was a problem hiding this comment.
Missing _matrix0_model in the global declaration. The function accesses _matrix0_model on line 785 but doesn't declare it as global, which could lead to UnboundLocalError if the global variable doesn't exist.
| except Exception as exc: # pragma: no cover - defensive fallback | ||
| logger.debug("Failed to count parameters on loaded model: %s", exc) |
There was a problem hiding this comment.
Using broad Exception catching is discouraged. Consider catching more specific exceptions that count_parameters() might raise, or document why broad exception handling is necessary here.
| finally: | ||
| del model | ||
| return _matrix0_model_params | ||
| except Exception as exc: # pragma: no cover - defensive fallback |
There was a problem hiding this comment.
Using broad Exception catching is discouraged. Consider catching more specific exceptions related to model configuration loading and parameter counting, or document why broad exception handling is necessary here.
| except Exception as exc: # pragma: no cover - defensive fallback | |
| except (FileNotFoundError, OSError, ValueError, AttributeError) as exc: # pragma: no cover - defensive fallback |
Summary
Testing
https://chatgpt.com/codex/tasks/task_e_68e6f992a86083238c7118dc33ca7287