feat(python-sidecar): secure foundation and deterministic packaged runtime

.gitignore

@@ -29,6 +29,10 @@ node_modules
 /.sass-cache
 /connect.lock
 /coverage
+.coverage
+.pytest_cache/
+__pycache__/
+*.pyc
 /libpeerconnection.log
 npm-debug.log
 yarn-error.log
@@ -61,3 +65,8 @@ playwright-report/
 test-results/
 
 CONTEXT.md
+
+# local bundled python runtime payload (large binaries, machine-local)
+build/python-runtime/*
+!build/python-runtime/README.md
+!build/python-runtime/.gitkeep

CURRENT-SPRINT.md

@@ -23,6 +23,10 @@ Advance post-refactor hardening by improving auth lifecycle completeness, IPC in
 - Production hardening: exclude lab routes/navigation from production bundle surface.
 - Update model proof: deterministic bundled-file demo patch cycle (`v1` to `v2`) with integrity check and UI diagnostics.
 
+## Highest Priority Follow-Up
+
+- `BL-028` Enforce robust file signature validation for privileged file ingress (extension + header/magic validation with fail-closed behavior before parser execution).
+
 ## Out Of Scope (This Sprint)
 
 - `BL-019`, `BL-022`, `BL-024`.

README.md

@@ -55,6 +55,7 @@ Desktop API surface includes:
 - `desktop.api.invoke()`
 - `desktop.updates.check()`
 - `desktop.telemetry.track()`
+- `desktop.python.*` (local Python sidecar probe/inspect/stop for privileged helper workflows)
 
 ## Expected Behaviors
 
@@ -68,6 +69,7 @@ Desktop API surface includes:
 
 - Node.js `^24.13.0`
 - pnpm `^10.14.0`
+- Python `3.11+` (required for Python sidecar lab/tests)
 
 ## Setup
 
@@ -101,6 +103,7 @@ pnpm renderer:serve
 ```bash
 pnpm lint
 pnpm unit-test
+pnpm nx run desktop-main:test-python
 pnpm integration-test
 pnpm e2e-smoke
 pnpm a11y-e2e
@@ -122,6 +125,7 @@ Flavor behavior:
 - `forge:make:staging`
   - sets `APP_ENV=staging`
   - enables packaged DevTools (`DESKTOP_ENABLE_DEVTOOLS=1`)
+  - builds renderer in `staging` mode so Labs routes remain available for verification
 - `forge:make:production`
   - sets `APP_ENV=production`
   - disables packaged DevTools (`DESKTOP_ENABLE_DEVTOOLS=0`)
@@ -174,6 +178,66 @@ Examples:
 
 If not configured, calling `call.secure-endpoint` returns a typed `API/OPERATION_NOT_CONFIGURED` failure.
 
+## Python Sidecar Backend Pattern
+
+This workspace supports a local Python helper backend model for privileged desktop capabilities (for example, file parsing libraries such as PyMuPDF).
+
+Current status:
+
+- Implemented as a lab capability (`Python Sidecar Lab` route).
+- Runs a local sidecar HTTP service bound to loopback (`127.0.0.1`).
+- Main process remains the security policy enforcement point.
+
+Pattern:
+
+1. Renderer selects a file via typed desktop dialog API.
+2. Renderer receives a short-lived file token, not a raw path.
+3. Preload/main validates contract envelopes.
+4. Main resolves token (window-scoped + expiring), validates file extension and magic header, then calls Python sidecar.
+5. Main returns safe diagnostics/results to renderer.
+
+Security properties:
+
+- Renderer cannot pass arbitrary filesystem paths for privileged parsing.
+- File ingress is fail-closed on extension/signature mismatch.
+- Helper runtime is local-only and not a renderer-controlled authority.
+
+How to run/verify:
+
+- Open `Python Sidecar Lab`.
+- `Probe Sidecar` to start/diagnose local runtime.
+- `Select PDF` then `Inspect Selected PDF` to verify end-to-end file handoff.
+- Run test gate:
+  - `pnpm nx run desktop-main:test-python`
+
+Deterministic packaged runtime (staging/production):
+
+- Provide a local bundled runtime payload at:
+  - `build/python-runtime/<platform>-<arch>/`
+  - example: `build/python-runtime/win32-x64/`
+- Pin sidecar dependencies in:
+  - `apps/desktop-main/python-sidecar/requirements-runtime.txt`
+- Fast local bootstrap from your current Python install:
+  - `pnpm run python-runtime:prepare-local`
+- Add `manifest.json` with `executableRelativePath` (see `build/python-runtime/README.md`).
+- Run validation:
+  - `pnpm run python-runtime:assert`
+  - assertion verifies interpreter exists and imports `fitz` when PyMuPDF is declared in manifest
+- Runtime payload is copied into desktop build artifacts by:
+  - `pnpm run build-desktop-main`
+  - `pnpm run forge:make:staging`
+  - `pnpm run forge:make:production`
+- Staging/production package commands fail fast if runtime bundle is missing or invalid.
+- Runtime diagnostics include `pythonExecutable` so packaged builds can prove the sidecar path at runtime.
+
+How to extend for new Python-backed operations:
+
+- Add a new typed channel/contract in `libs/shared/contracts`.
+- Add preload API binding in `apps/desktop-preload`.
+- Add main handler validation and token/scope checks in `apps/desktop-main`.
+- Add sidecar endpoint behavior in `apps/desktop-main/src/assets/python_sidecar/service.py`.
+- Add/extend Python tests under `apps/desktop-main/python-sidecar/tests`.
+
 ## Repository Layout
 
 - `apps/` runnable applications

apps/desktop-main/project.json

@@ -17,7 +17,13 @@
         "bundle": false,
         "main": "apps/desktop-main/src/main.ts",
         "tsConfig": "apps/desktop-main/tsconfig.build.json",
-        "assets": ["apps/desktop-main/src/assets"],
+        "assets": [
+          {
+            "input": "apps/desktop-main/src/assets",
+            "glob": "**/*",
+            "output": "apps/desktop-main/src/assets"
+          }
+        ],
         "generatePackageJson": true,
         "esbuildOptions": {
           "sourcemap": true,
@@ -28,6 +34,14 @@
       },
       "configurations": {
         "development": {},
+        "staging": {
+          "esbuildOptions": {
+            "sourcemap": false,
+            "outExtension": {
+              ".js": ".js"
+            }
+          }
+        },
         "production": {
           "esbuildOptions": {
             "sourcemap": false,
@@ -76,6 +90,9 @@
         "development": {
           "buildTarget": "desktop-main:build:development"
         },
+        "staging": {
+          "buildTarget": "desktop-main:build:staging"
+        },
         "production": {
           "buildTarget": "desktop-main:build:production"
         }
@@ -97,11 +114,23 @@
         "command": "pnpm nx run-many -t build --projects=renderer,desktop-preload,desktop-main && node tools/scripts/smoke-desktop-artifacts.mjs"
       }
     },
-    "test": {
+    "test-ts": {
       "executor": "nx:run-commands",
       "options": {
         "command": "vitest run --config apps/desktop-main/vitest.config.mts"
       }
+    },
+    "test-python": {
+      "executor": "nx:run-commands",
+      "options": {
+        "command": "node tools/scripts/run-python-sidecar-tests.mjs"
+      }
+    },
+    "test": {
+      "executor": "nx:run-commands",
+      "options": {
+        "command": "pnpm nx run desktop-main:test-ts && pnpm nx run desktop-main:test-python"
+      }
     }
   }
 }

apps/desktop-main/python-sidecar/requirements-runtime.txt

@@ -0,0 +1 @@
+PyMuPDF==1.26.7

apps/desktop-main/python-sidecar/requirements-test.txt

@@ -0,0 +1,2 @@
+pytest==8.4.2
+pytest-cov==7.0.0

apps/desktop-main/python-sidecar/tests/test_service.py

@@ -0,0 +1,185 @@
+from __future__ import annotations
+
+import importlib.util
+import json
+import tempfile
+import threading
+from http.client import HTTPConnection
+from pathlib import Path
+from socketserver import TCPServer
+from typing import Any
+
+
+def _load_service_module():
+    service_path = (
+        Path(__file__).resolve().parents[2]
+        / "src"
+        / "assets"
+        / "python_sidecar"
+        / "service.py"
+    )
+    spec = importlib.util.spec_from_file_location("python_sidecar_service", service_path)
+    assert spec is not None
+    assert spec.loader is not None
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module
+
+
+def _request(
+    port: int,
+    method: str,
+    path: str,
+    payload: dict[str, Any] | None = None,
+) -> tuple[int, dict[str, Any] | str]:
+    conn = HTTPConnection("127.0.0.1", port, timeout=3)
+    headers = {}
+    body = None
+    if payload is not None:
+        body = json.dumps(payload).encode("utf-8")
+        headers["Content-Type"] = "application/json"
+    conn.request(method, path, body=body, headers=headers)
+    response = conn.getresponse()
+    raw = response.read().decode("utf-8")
+    conn.close()
+    try:
+        return response.status, json.loads(raw)
+    except json.JSONDecodeError:
+        return response.status, raw
+
+
+def test_build_health_payload_contains_core_diagnostics():
+    service = _load_service_module()
+
+    payload = service._build_health_payload()
+
+    assert payload["status"] == "ok"
+    assert payload["service"] == "python-sidecar"
+    assert "pythonVersion" in payload
+    assert "pythonExecutable" in payload
+    assert isinstance(payload["pythonExecutable"], str)
+    assert isinstance(payload["pymupdfAvailable"], bool)
+
+
+def test_health_endpoint_returns_ok_payload():
+    service = _load_service_module()
+    server = TCPServer(("127.0.0.1", 0), service._Handler)
+    thread = threading.Thread(target=server.serve_forever, daemon=True)
+    thread.start()
+    try:
+        status, payload = _request(server.server_address[1], "GET", "/health")
+        assert status == 200
+        assert isinstance(payload, dict)
+        assert payload["status"] == "ok"
+    finally:
+        server.shutdown()
+        server.server_close()
+        thread.join(timeout=1)
+
+
+def test_inspect_pdf_endpoint_accepts_safe_pdf_file():
+    service = _load_service_module()
+    server = TCPServer(("127.0.0.1", 0), service._Handler)
+    thread = threading.Thread(target=server.serve_forever, daemon=True)
+    thread.start()
+
+    try:
+        with tempfile.TemporaryDirectory() as temp_dir:
+            pdf_path = Path(temp_dir) / "sample.pdf"
+            pdf_path.write_bytes(b"%PDF-1.7\n% safe\n")
+
+            status, payload = _request(
+                server.server_address[1],
+                "POST",
+                "/inspect-pdf",
+                {"filePath": str(pdf_path)},
+            )
+
+        assert status == 200
+        assert isinstance(payload, dict)
+        assert payload["accepted"] is True
+        assert payload["fileName"] == "sample.pdf"
+        assert payload["headerHex"] == "255044462d"
+    finally:
+        server.shutdown()
+        server.server_close()
+        thread.join(timeout=1)
+
+
+def test_inspect_pdf_endpoint_rejects_missing_filepath():
+    service = _load_service_module()
+    server = TCPServer(("127.0.0.1", 0), service._Handler)
+    thread = threading.Thread(target=server.serve_forever, daemon=True)
+    thread.start()
+    try:
+        status, payload = _request(server.server_address[1], "POST", "/inspect-pdf", {})
+        assert status == 400
+        assert isinstance(payload, dict)
+        assert "filePath is required" in payload["message"]
+    finally:
+        server.shutdown()
+        server.server_close()
+        thread.join(timeout=1)
+
+
+def test_unknown_paths_return_404():
+    service = _load_service_module()
+    server = TCPServer(("127.0.0.1", 0), service._Handler)
+    thread = threading.Thread(target=server.serve_forever, daemon=True)
+    thread.start()
+    try:
+        get_status, _ = _request(server.server_address[1], "GET", "/unknown")
+        post_status, _ = _request(server.server_address[1], "POST", "/unknown", {})
+        assert get_status == 404
+        assert post_status == 404
+    finally:
+        server.shutdown()
+        server.server_close()
+        thread.join(timeout=1)
+
+
+def test_main_initializes_server_and_closes_on_shutdown(monkeypatch):
+    service = _load_service_module()
+
+    class _FakeServer:
+        def __init__(self, address, handler):
+            self.address = address
+            self.handler = handler
+            self.closed = False
+
+        def serve_forever(self):
+            raise KeyboardInterrupt("stop server")
+
+        def server_close(self):
+            self.closed = True
+
+    class _Args:
+        host = "127.0.0.1"
+        port = 43124
+
+    fake_server: _FakeServer | None = None
+
+    def _fake_http_server(address, handler):
+        nonlocal fake_server
+        fake_server = _FakeServer(address, handler)
+        return fake_server
+
+    class _FakeParser:
+        def add_argument(self, *_args, **_kwargs):
+            return None
+
+        def parse_args(self):
+            return _Args()
+
+    monkeypatch.setattr(service.argparse, "ArgumentParser", lambda: _FakeParser())
+    monkeypatch.setattr(service, "HTTPServer", _fake_http_server)
+
+    try:
+        service.main()
+    except KeyboardInterrupt:
+        pass
+
+    assert fake_server is not None
+    assert fake_server.address == ("127.0.0.1", 43124)
+    assert fake_server.handler is service._Handler
+    assert fake_server.closed is True