fix: SBC installation and system setup improvements #8
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Remove PulseAudio from service lists in both Docker and SBC deployments: - Not installed on Raspberry Pi OS Lite - Not needed - we use sounddevice → PortAudio → ALSA directly - Removes from services page and logs page UI Changes: - Remove pulseaudio from SERVICES_CONFIG in system_control.py - Remove [program:pulseaudio] section from supervisord.conf - Update test to remove pulseaudio priority assertion
…e contract The update daemon was writing status to Redis with field 'update_available', but the API endpoint expected 'available', causing Pydantic validation errors. Changes: - UpdateManager.check_for_updates() now returns fields matching UpdateStatusResponse: - 'update_available' → 'available' - Added 'deployment_type' (from SystemUtils.get_deployment_environment) - Added 'can_auto_update' (true for release versions on SBC) - Removed 'version_type' and 'checked_at' (not in contract) - Added contract test to validate UpdateManager output matches API schema - Updated all tests to use new field names (106 tests passing) This fixes /api/update/status endpoint returning validation errors instead of update status, which may have prevented the web UI from properly detecting daemon status. Closes: Update daemon control issues on SBC
Two related fixes for git-based update functionality: 1. Update API gracefully handles non-git deployments - Catches subprocess exit code 128 (not a git repository) - Returns empty lists for remotes/branches instead of HTTP 500 - Provides helpful error message when adding remotes to non-git install - Prevents "failed-to-load-remotes" error in UI 2. Installer now creates proper git repository - Clone directly to /opt/birdnetpi instead of temp directory - Preserves .git directory for git-based updates - Eliminates unnecessary file copying (src/, config_templates/) - Simplifies installer by using git clone as source of truth These changes enable git-based updates on SBC deployments and provide graceful degradation for installations without git. Modified: - src/birdnetpi/web/routers/update_api_routes.py - install/install.sh - install/setup_app.py
- Default branch to 'main' for production use - Prevent cleanup.sh from deleting birdnetpi user when run by that user For testing, override with: BIRDNET_BRANCH=feature/sbc-bugfixes
- Show 'SKIPPED' status when user removal is skipped - Include preservation notice in final summary - Makes it clear to users why their account wasn't deleted
Makes it clearer that install.sh is the bootstrap step before setup_app.py runs the actual installation
Use ANSI escape codes to clear line and ensure proper positioning: - \r moves cursor to column 0 - \033[K clears from cursor to end of line - \n advances to next line This prevents parallel task logs from appearing on the same line
- Remove /dev/shm/birdnet-installer cleanup (no longer created) - Clarify that birdnetpi user removal is conditional - Update warning message to match actual behavior
Switch from sys.stdout.write() to print() which handles platform-specific line endings correctly (LF vs CRLF). This ensures cursor returns to column 0 after each line.
Add clear progress indicators for parallel job groups: - 'Starting:' message before parallel tasks begin - 'Completed:' message after parallel tasks finish - Blank lines between sections for better readability - Time estimate for asset download (1-10 minutes) This helps users track progress through smaller, logical chunks rather than watching individual tasks complete.
ANSI escape codes (\r\033[K) were interacting badly with subprocess output, causing terminal state corruption that required 'reset' to fix. Simplified to use plain print() which is more robust, even if output alignment isn't perfect. Terminal reliability > perfect formatting.
Explicitly flush both stdout and stderr before each log message to ensure subprocess output doesn't interfere with cursor positioning. This prevents log messages from appearing mid-line or having incorrect cursor positions.
Based on research, subprocess calls can interfere with parent terminal state when they try to access stdin. Adding stdin=subprocess.DEVNULL to all subprocess.run() calls prevents this issue. This fixes the terminal corruption that was requiring 'reset' to fix, where cursor positioning would become corrupted and text wouldn't display on new lines. Simplified log() function back to simple print() since the root cause is now fixed. References: - stackoverflow.com/questions/6488275 - stackoverflow.com/questions/38244830
- Remove manual venv creation (uv creates it automatically) - Install uv to system Python in wave 2 alongside system packages - Reorder wave 4 to run Caddy config, systemd install, and asset download in parallel - Split systemd setup into install and start phases - Add health check as final step to verify all services are running - Put longer-running tasks (asset download) at bottom for better progress visibility
- Remove apt_update() since install.sh already does apt-get update - Combine directory creation, system packages, and uv install in Wave 1 - Update wave numbering (now 5 waves instead of 6)
Show actual pip3 error instead of suppressing output to help diagnose installation failures
- Download and run standalone uv installer from astral.sh - Avoids PEP 668 externally-managed-environment error - Install as birdnetpi user to ~/.cargo/bin/uv - Update install_python_dependencies to use full uv path
Show actual error from uv sync instead of suppressing output to help diagnose dependency installation failures
Run 'curl ... | sh' as a single shell command instead of capturing output and passing to sh. This matches the official installation method.
Use 'sudo -u birdnetpi -i' to start a login shell that sources the shell profile where uv installer added ~/.cargo/bin to PATH
- Create /opt/uv directory owned by birdnetpi - Use UV_INSTALL_DIR environment variable to specify installation location - Update install_python_dependencies to use /opt/uv/bin/uv - Update cleanup.sh to remove /opt/uv directory
- Create user with -d /opt/birdnetpi (no -m since dir already exists) - For existing users, use usermod -d to change home directory - Eliminates /home/birdnetpi directory entirely - Update cleanup.sh to use userdel without -r flag (home dir already removed)
- Remove custom UV_INSTALL_DIR since home is now /opt/birdnetpi - uv installs to default ~/.local/bin (/opt/birdnetpi/.local/bin) - Remove /opt/uv directory creation and cleanup - Update install_python_dependencies to use /opt/birdnetpi/.local/bin/uv
Detect if current user is birdnetpi and skip usermod -d command since you cannot modify a user's home directory while that user is logged in
Move uv installation out of parallel wave to avoid DNS resolution race condition with apt-get operations. System package installation can temporarily affect network/DNS, causing github.com resolution to fail. Installation waves now: - Wave 1: data directories + system packages (parallel) - Wave 2: uv installation (sequential) - Wave 3: Python dependencies (sequential) - Wave 4: Caddy + systemd + assets (parallel) - Wave 5: Start services (sequential) - Wave 6: Health check (sequential)
Revert to using /opt/uv instead of relying on user home directory location. This ensures consistent installation path regardless of whether home directory is /home/birdnetpi or /opt/birdnetpi. - Create /opt/uv and set ownership to birdnetpi - Use UV_INSTALL_DIR=/opt/uv environment variable - Update install_python_dependencies to use /opt/uv/bin/uv - Add /opt/uv back to cleanup.sh
Add 2-second sleep after apt-get operations to allow DNS resolution to stabilize before git clone. APT operations can temporarily affect network/DNS configuration.
Add setup-system CLI tool that initializes critical configuration before systemd services start during installation: - Initialize config from template (birdnetpi.yaml) - Auto-detect audio devices (prefer USB, select by sample rate) - Auto-detect GPS for location - Interactive prompts for attended installs (device name, location, language) - Support boot volume pre-configuration from flasher - Integrate into installer as Wave 4.5 (after assets, before services) The system setup tool ensures the device is properly configured at the end of installation, resolving the issue where the system claimed to be ready but lacked critical settings.
Implement all TODOs in setup_system.py: - GPS detection using gpsdclient with automatic timezone detection - Interactive timezone selection with validation - Dynamic language support from IOC database with fallback Add comprehensive test coverage: - 22 tests for setup system functionality - Test audio device detection, GPS, boot config, and configuration - Mock GPS client with proper spec - BDD-style test docstrings The language selection now queries the IOC database to show actual translation coverage for each language, sorted by species count. Falls back to hardcoded list if database unavailable.
Add comprehensive tests for boot config integration: - Device name configuration from boot config - Location configuration from boot config with GPS override handling - Language configuration from boot config - Fallback to prompts when boot config values not present The boot config integration allows pre-configuration via /boot/firmware/birdnetpi_config.txt for headless installations. Coverage increased from 52% to 69% on setup_system.py.
Document the boot volume pre-configuration feature: - How to create birdnetpi_config.txt - Supported configuration options - Priority order for configuration sources - Examples for headless installation - Validation requirements - Finding configuration values This enables users to pre-configure BirdNET-Pi installations via SD card flasher tools for completely headless setup.
Add optional BirdNET-Pi configuration questions to SD card flasher: - Device name - Latitude/Longitude - Timezone (with common examples) - Language code (with common options) When provided, creates birdnetpi_config.txt on boot partition. Values are saved with other flasher settings for reuse. All fields are optional - skip by pressing Enter. If latitude is provided, longitude and timezone are offered. Updated documentation to recommend using flasher for easiest setup.
Refactor BirdNET-Pi configuration prompts to use data-driven approach: - Define prompts in a dict with metadata (prompt text, help, conditions) - Use sentinel value (object()) to distinguish unset from empty/None - Single loop handles all prompts with conditional display - Properly handles saved config: only uses non-empty saved values Benefits: - DRY: Add new config fields by adding dict entries - Clear: Prompt logic separated from data - Correct: Empty strings won't be reused from saved config
The UV_INSTALL_DIR environment variable doesn't work correctly with the uv installer. Instead, install to default location ($HOME/.cargo/bin) then move the binary to /opt/uv/bin/uv for consistency. Fixes: sudo: /opt/uv/bin/uv: command not found
Run mv command as birdnetpi user so $HOME expands correctly to the user's home directory, regardless of the actual path.
Use ~birdnetpi to expand to the birdnetpi user's home directory, and run with sudo to have permission to write to /opt/uv/bin/.
Use Python's pwd module to reliably get the birdnetpi user's home directory, avoiding shell expansion issues with tilde and $HOME.
The uv installer actually installs to ~/.local/bin/uv, not ~/.cargo/bin/uv. Update the source path for the move command.
English (scientific names) is not in the IOC languages table since it's the base nomenclature. Always include it in the language list so users can select it as their preferred language.
- Use git ls-remote instead of fetch for listing branches/tags (much faster) - Hide Apply/Test buttons when manual update required (can_auto_update=false) - Hide update banner when on development version (dev banner takes priority) - Include version_type in update status response for banner logic Resolves: - Branches/tags not loading (git fetch timing out) - Disabled buttons still visible when manual update required - Update banner showing alongside dev warning banner
The reboot modal was not providing adequate feedback to users during the reboot process. The modal would remain visible with buttons still available instead of showing a clear loading state. Changes: - Replace jQuery modal.hide() with closeAllModals() for consistency - Display full-screen loading overlay immediately after reboot initiation - Add 5-minute timeout with helpful error message if system doesn't recover - Poll system availability every 5 seconds during reboot - Show troubleshooting steps on timeout (SSH, logs, LED indicators) - Add proper i18n support for all new user-facing strings - Include full accessibility attributes (role, aria-label, aria-live) The loading overlay now clearly indicates the system is rebooting with a spinner and estimated time. If the system fails to respond within 5 minutes, users receive actionable troubleshooting guidance.
Asset tags (prefixed with "assets-") are not actual code branches and should not appear in the tag list on the update/branch selection page. These tags are used for model downloads and other release assets, not software versions. Changes: - Filter out tags starting with "assets-" in GitOperationsService.list_tags() - Apply filtering for both local and remote tag queries - Update existing tests to use correct ls-remote output format - Add comprehensive tests for asset tag filtering - Increase test coverage of git_operations.py to 94% The filtering happens at the service layer so all callers automatically benefit from the exclusion of non-code tags.
The deployment type badge appeared orphaned at the top of the page without context. The Current Status and other panels had cramped typography with insufficient spacing that looked unprofessional. Changes: - Hide deployment banner entirely (unnecessary visual element) - Increase panel padding from 1.5rem to 2rem throughout - Add consistent h2 styling (1.5rem, bold, with bottom border) - Increase status row padding from 0.5rem to 1rem for better breathing room - Add proper styling for Update Actions, Docker Config, and Help panels - Improve instructions and code block styling with better contrast - Add hover effects to clickable summary elements - Standardize all panel spacing and typography for consistency The update page now has a clean, professional appearance with proper visual hierarchy and comfortable spacing throughout.
The template was intentionally changed in c19ada2 to hide the update banner when on a development version (dev banner takes priority). This test was expecting both banners to show simultaneously, which contradicted the new behavior. Updated test: - Renamed test_both_banners_can_show → test_dev_banner_takes_priority_over_update - Verify only dev banner shows when on dev version with update available - Add docstring explaining the priority behavior
- Add PYLEAK_BLOCKING_THRESHOLD env var (default: 0.2s for local) - Set to 0.5s in CI to accommodate slower I/O - Prevents false positives from file I/O operations in CI - Keeps fast feedback for local development Fixes test_get_database_stats pyleak failure in CI (0.286s > 0.200s)
The pytest_configure approach didn't work because PyLeak reads the threshold before pytest_configure runs. Using --blocking-threshold=0.5 as a command-line argument ensures PyLeak picks up the correct value.
mverteuil
force-pushed
the
feature/sbc-bugfixes
branch
from
b1ce582 to
daa731a
Compare
mverteuil
added a commit
that referenced
this pull request
Oct 24, 2025
- Improved installer reliability and user experience for SBC deployments - Added system setup CLI for pre-service configuration - Fixed update page UI and functionality issues - Enhanced boot config pre-configuration for SD card flashing - **Fix uv installation**: Use official installer instead of pip, with proper PATH handling - **Parallel execution**: Restructure installer for optimized parallel package installation - **Terminal output**: Fix mangled output, ANSI escape codes, and stdout/stderr flushing - **User management**: Prevent birdnetpi user deletion, set proper home directory - **Error handling**: Capture and display installation errors properly - **Pre-service configuration**: New CLI tool to configure settings before services start - **GPS support**: Configure GPS settings during setup - **Timezone configuration**: Set timezone before services launch - **Language selection**: Configure interface language at setup time - **Boot config**: Integration tests for configuration workflow - **Git branch loading**: Fix branch detection and display - **Banner priority**: Correct warning banner hierarchy - **Tag filtering**: Remove assets- prefixed tags from update page - **Typography**: Improve spacing and layout consistency - **Release notes**: Hide empty release notes section - **Status rows**: Ensure consistent heights with min-height - **Reboot feedback**: Add loading overlay and timeout handling - **Badge cleanup**: Remove orphaned badges - **Pre-configuration**: Add BirdNET-Pi setup prompts to SD card flasher - **Configuration prompts**: Dict-driven loop for cleaner prompt handling - **Documentation**: Add boot config pre-configuration guide - **DNS stability**: Add delay after apt operations for DNS to settle - **Non-git deployments**: Handle installations without git repository - **PulseAudio**: Remove from service monitoring (not used) - **UpdateManager contract**: Align check_for_updates() with API response schema - Boot config integration tests - Installation error handling tests - Update page functionality tests
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Installer Improvements
System Setup CLI
Update Page Fixes
UI Improvements
Flasher Enhancements
Other Fixes
Test Coverage