feat: SPI auto-enablement + SD card flasher TUI with profile management #16
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add enable_spi_interface() to check and enable SPI in boot config - Prompt user to reboot when SPI is enabled - Run SPI check early in installer before hardware detection - After reboot, SPI devices will be available for HAT detection This ensures the Waveshare e-paper HAT is properly detected even on fresh installations where SPI is disabled by default.
…tion - Retry up to 3 times with 5-second delay between attempts - Specifically handles GitHub network errors when installing waveshare-epd - Improves reliability when installing epaper extras on unstable networks Fixes intermittent 'Could not resolve host: github.com' errors during waveshare-epd installation from Git repository.
- Replace Path.exists() with sudo test -f to check files - Avoids Permission denied errors on /etc/redis/redis.conf - Use cp -n flag to avoid overwriting existing backup Fixes: [Errno 13] Permission denied: '/etc/redis/redis.conf'
The previous logic incorrectly checked if the entire file content started with '#dtparam=spi=on', which would never be true. Changed to properly check if any line in the file equals 'dtparam=spi=on' (uncommented).
- Add detect_device_specs() to detect RAM and device type - Create redis.conf.j2 Jinja2 template for dynamic configuration - Update configure_redis() to render template based on device: - 512MB (Pi Zero 2W): 32MB Redis limit - 1GB (Pi 3B): 64MB Redis limit - 2GB (Pi 4B 2GB): 128MB Redis limit - 4GB+ (Pi 4B/5): 256MB Redis limit - Display detected device info during installation
- Add DeviceSpecs TypedDict for type-safe device detection - Add disable_unnecessary_services() to free ~12MB RAM on 512MB devices - Disable ModemManager, Bluetooth, triggerhappy, avahi-daemon on Pi Zero 2W - Integrate into Wave 4.5 of installation process - Services only disabled if device has ≤512MB RAM
- Add swap disabling to disable_unnecessary_services() - Run dphys-swapfile swapoff, uninstall, and disable service - Prevents SD card wear from swap thrashing - Only applies to devices with ≤512MB RAM (Pi Zero 2W)
- Install uv in install.sh before running setup_app.py - Run uv sync in install.sh (makes Jinja2 available for Redis config) - Detect epaper HAT in install.sh and conditionally install extras - Remove install_uv() and install_python_dependencies() from setup_app.py - Conditionally install epaper display service based on hardware detection - Execute setup_app.py via 'uv run' so Jinja2 is available This allows setup_app.py to use Jinja2 for rendering the Redis config template with device-specific memory limits.
- Check and enable SPI interface at start of install.sh - Reboot immediately after enabling SPI (before uv installation) - Detect epaper HAT after reboot when SPI devices exist - Fix uv paths: /opt/uv/uv not /opt/uv/bin/uv - Remove enable_spi_interface() from setup_app.py (now in install.sh) - Prompt user to re-run installer after SPI reboot Flow: 1. install.sh checks SPI -> enables if needed -> reboots 2. User re-runs install.sh 3. SPI now enabled -> /dev/spidev* exists 4. epaper HAT detected -> extras installed 5. Continue with normal installation
… retry - Add profile management to flasher (list, select, edit, duplicate) - Fix bug where edited profiles always saved as 'default' - Add LibreComputer Le Potato (AML-S905X-CC) device support - Clone portability script to boot partition - Create helper script (lepotato_setup.sh) with correct model number - Add two-step boot instructions in README - Add 'Enable SPI (for ePaper HAT)?' option to flasher - Uncomments dtparam=spi=on in config.txt - No reboot needed on first boot - Add retry mechanism with exponential backoff to install.sh - Handles transient network/DNS failures - 3 retries with 5s/10s/20s delays
- Run setup_app.py with .venv/bin/python instead of uv run - Avoids permission errors when venv is owned by birdnetpi user - setup_app.py needs sudo for system operations, can't run as birdnetpi
- Wait up to 60 seconds for network/DNS to be ready - Ping github.com to verify DNS resolution works - Prevents 'Could not resolve host' errors during early boot - Continues after timeout with warning if network still not ready
- Ping alone isn't enough - git uses different DNS mechanism - Test actual git ls-remote to waveshare repo before uv sync - Add 2 second stabilization delay after DNS ready - Should prevent git DNS failures that ping doesn't catch
- Show 'Retrying dependency installation...' after sleep - Prevents appearance of stalled installation - User sees clear feedback that retry is happening
Major enhancements for SD card flashing and first-boot installation: ## ePaper HAT Support - Add standalone test script (install/test_epaper.py) for hardware verification - Auto-detect ePaper display models (2.13", 2.9", 2.7", 4.2", 7.5") - Add SPI enablement option in flasher (uncommets dtparam=spi=on) - Download Waveshare library to boot partition during flashing - Patch pyproject.toml at boot to use local library (no network needed) ## LibreComputer Le Potato Support - Add Le Potato (AML-S905X-CC) to device selection - Clone portability scripts to boot partition - Patch oneshot.sh to support Raspbian 12 (Bookworm) - Create helper script for easy setup ## Installation Reliability - Add retry mechanism with exponential backoff for dependency installation - Implement offline Waveshare installation using boot partition cache - Use uv.sources pattern for dependency source management ## Cleanup - Remove unused install/ui_whiptail.py from old installer design Changes enable: - Offline ePaper HAT installation after SD flashing - Hardware testing without full installation (--test-epaper flag) - Support for non-Raspberry Pi ARM SBCs - More reliable first-boot installation with retry logic
- Add warning if version check pattern not found - Split multiline replace into separate steps for reliability - Add diagnostic output to debug patching issues
The waveshare_epd library requires gpiozero for GPIO operations, and Pillow is needed for image rendering in the test script. Without these, display modules fail with ModuleNotFoundError.
Show spinner with estimated time (1-2 minutes) while cloning the e-Paper repository to prevent appearing frozen. Uses rich console.status() for visual feedback.
Only download the Python subdirectory (~45MB) instead of the full repo to fit on the boot partition. Uses git sparse-checkout to efficiently fetch only RaspberryPi_JetsonNano/python directory. Update install.sh path to match the new directory structure.
The LibreComputer repository has an expired GPG key which causes apt-get update to fail. Patch oneshot.sh to make apt operations non-fatal with || echo, allowing bootloader conversion to proceed. This is a workaround for upstream issue - the main bootloader conversion functionality doesn't require their package repository.
The boot partition (FAT32) is read-only for non-root users, causing uv build to fail with 'Permission denied'. Copy the library from /boot/firmware/waveshare-epd to /opt/birdnetpi/waveshare-epd with proper ownership before installing. This allows uv to write build artifacts during package installation.
Use regex to append '|| true' to all apt-get commands in oneshot.sh. This ensures the bootloader conversion continues even when apt operations fail due to the expired LibreComputer repository GPG key. The bootloader conversion is the critical functionality - apt operations are secondary and can safely fail.
When we patch pyproject.toml to use local path instead of git URL, the lockfile becomes outdated. Run 'uv lock' after patching to regenerate the lockfile, allowing 'uv sync --locked' to succeed. This fixes the dependency installation failure when using the pre-downloaded Waveshare library from boot partition.
The LibreComputer oneshot.sh script uses 'apt' commands (not 'apt-get'), and also uses apt-mark and apt-key. Updated regex to match: - apt, apt-get, apt-mark, apt-key (using apt[-\w]*) - Piped apt commands like: wget ... | sudo apt-key add - This ensures all apt operations are non-fatal when the LibreComputer repository has an expired GPG key.
The RPi.GPIO 0.7.1 package redirects to Jetson.GPIO which doesn't work on Raspberry Pi. Add lgpio (the modern GPIO library for Bookworm) so gpiozero can use proper GPIO access. This fixes the 'Could not determine Jetson model' error when initializing the ePaper display.
Add B (three-color) versions of display models to the auto-detection list: - epd2in13b_V4 (2.13" V4 with red) - epd2in13b_V3 (2.13" V3 with red) Try B versions first as they're common and can fall back to two-color drivers if needed.
Three-color (B/W/Red) ePaper HAT models require two image buffers: - Black/white image buffer - Red image buffer (can be blank if not using red) Detects B versions by checking for 'b' in model name and provides both buffers to display() function. Two-color displays continue to use single buffer. Fixes TypeError: EPD.display() missing 1 required positional argument: 'imagered' Also adds proper type annotations with type: ignore comments for waveshare_epd dynamic attributes. Allows ANN401 (Any usage) for install/* files since they deal with dynamically imported modules.
The birdnetpi user needs to be in the spi and gpio groups to access /dev/spidev* and /dev/gpiochip* devices without sudo. This allows the epaper-display-daemon to run as the birdnetpi user instead of requiring root permissions.
Official solution from LibreComputer support for expired GPG key: https://hub.libre.computer/t/signatures-were-invalid-expkeysig-2e5fb7fc58c58ffb/4166 The oneshot.sh script now installs libretech-keyring_2024.05.19 at the beginning, which contains updated certificates and allows the LibreComputer repository to work properly. This fixes: - E: The repository 'https://deb.libre.computer/repo linux InRelease' is not signed - W: GPG error: EXPKEYSIG 2E5FB7FC58C58FFB - E: Unable to locate package linux-image-lc-lts-arm64 - E: Unable to locate package linux-headers-lc-lts-arm64
During first boot, network/DNS may not be ready when oneshot.sh runs, causing wget to download incomplete/corrupted files. Added: - 30 second network wait with ping check - File validation to ensure .deb package is valid before installing - Proper error handling that warns but continues on failure This should fix the 'unexpected end of file in archive magic version number' error when dpkg tries to install a corrupted download.
The sparse-checkout transfers ~6MB over the network (compressed git objects), not 45MB. The 45MB is the final on-disk size after checkout. Updated message to show actual transfer size to avoid confusion.
The libretech-keyring_2024.05.19 package installs updated GPG keys, but then oneshot.sh downloads the old expired key from libre-computer-deb.gpg and overwrites them. Now comment out the wget command that downloads the old key, so the updated keys from the keyring package are preserved. This should fix: - W: GPG error: EXPKEYSIG 2E5FB7FC58C58FFB - E: The repository 'https://deb.libre.computer/repo linux InRelease' is not signed
The regex was adding the comment to the end of the line instead of commenting out the beginning. The wget command was still running. Changed from: wget ... 'libre-computer-deb.gpg' # Commented: using updated keyring To: # wget ... 'libre-computer-deb.gpg' # Commented: using updated keyring Removed \s* from regex start so it properly matches lines beginning with 'wget' and adds '#' at the beginning.
On fresh Raspbian images, the system clock is set to the image creation date (2024-07-04), causing wget to fail with SSL certificate errors: ERROR: The certificate of 'deb.libre.computer' is not trusted. ERROR: The certificate of 'deb.libre.computer' is not yet activated. Added --no-check-certificate to wget since: 1. We're downloading from the official LibreComputer repository 2. We validate the downloaded file is a valid .deb package before installing 3. The clock will be corrected by NTP after boot completes 4. This is a trusted first-party package needed to fix the GPG key issue This is safe because we verify the file type before installing.
Le Potato uses u-boot bootloader, not grub. The oneshot.sh script tries to run grub-install for x86 boards, which fails on ARM: grub-install: error: /boot doesn't look like an EFI partition. Since the script uses 'set -e', this failure causes it to exit before completing the u-boot installation. Added || true to grub_install_cmd so the script continues past the error and completes the conversion process.
The ePaper display daemon was hardcoded to check health on port 8000, but FastAPI runs on configurable port 8888. This caused health checks to fail and the display to show nothing. Now parses the API base URL from detections_endpoint config instead of hardcoding localhost:8000. For example, if detections_endpoint is 'http://127.0.0.1:8888/api/detections/', extracts 'http://127.0.0.1:8888' as the API base URL. Tested on Pi 4B - health checks now succeed and display shows status.
Implements partial refresh mode to eliminate flicker on 2-color displays. Falls back gracefully to full refresh for 3-color displays that don't support partial refresh (like 2in13b_V4). - Add partial refresh counter and interval tracking - Init display in partial refresh mode after initial clear - Use displayPartial() for updates when supported - Full refresh every 20 partial updates to prevent ghosting - Graceful fallback if partial refresh not available 3-color displays will continue using full refresh due to hardware limitations with the red pigment layer.
Split the complex _update_display() method into focused helper methods: - _update_display_partial(): Attempt partial refresh with method name variants - _save_simulation_images(): Handle simulation mode file writes - _update_3color_display(): Handle 3-color displays (always full refresh) - _update_2color_display(): Handle 2-color displays (partial refresh capable) Also optimized red layer creation to only occur during animations, creating an empty red buffer otherwise to maintain 3-color display compatibility. Fixes Ruff C901 complexity warning.
- Display local time based on configured timezone (not UTC) - Show update interval and next update time on display - Swap health status and system stats rows for better layout - Convert all timestamps (current time, detections) to local timezone - Add timezone helper methods to reduce complexity - Track update timing for display on screen This ensures users see their local time on the display instead of UTC, and provides visibility into when the next refresh will occur.
- Add set_system_timezone() function to configure system time - Uses timedatectl to set timezone based on config.timezone - Validates timezone against pytz before applying - Called automatically during initial system setup - Ensures system logs and timestamps match user's timezone This fixes the issue where the system shows UTC even when the user configured a different timezone like America/Toronto.
- Add sudo-to-su conversion in install.sh for root execution - Add subprocess wrapper in setup_app.py to strip sudo when root - Add build-essential and python3.11-dev for compiling C extensions - Add piwheels.org as extra index for pre-built ARM wheels - Increase UV_HTTP_TIMEOUT to 300s to prevent timeout on slow connections - Handle environment variables in sudo wrapper function This allows the installer to work on systems where only root access is available (e.g., DietPi fresh installs) while maintaining compatibility with non-root users who have sudo privileges.
The previous implementation only stripped 'sudo' but left flags like '-u' which caused '[Errno 2] No such file or directory: -u' errors. Now properly strips: - sudo command itself - User/group flags (-u, --user, -g, --group) and their arguments - Other sudo flags Also moved wrapper class outside main() to reduce complexity.
The install_assets() function uses 'sudo -u birdnetpi' to ensure files are created with correct ownership. When setup_app.py runs as root, we need to convert these commands to 'su - birdnetpi -c "..."' to maintain proper file ownership. Changes: - Add subprocess wrapper that converts sudo -u commands to su - Properly parse sudo flags (-u, --user, -g, --group) - Use shlex.quote to safely construct su command strings - Require root privileges (needed for systemctl, apt-get, etc.)
DietPi doesn't have DBus available during installation, so timedatectl
fails with 'Failed to connect to bus'. Add a fallback that directly
manipulates /etc/timezone and /etc/localtime when DBus is unavailable.
Fallback method:
- Write timezone to /etc/timezone
- Symlink /etc/localtime to /usr/share/zoneinfo/{timezone}
This ensures timezone configuration works on minimal systems like DietPi
while still preferring timedatectl on full systems.
Problem: Service lists were hardcoded in 3 different places, causing the epaper display service to be installed but not shown in status output. Solution: Create ServiceRegistry class with: - SYSTEM_SERVICES: Redis, Caddy (not managed by BirdNET) - CORE_SERVICES: Always-installed BirdNET services - _optional_services: Runtime-detected services (e.g., epaper) Benefits: - Single source of truth for all services - Epaper service now shows in final status output - Easy to add new optional services in the future - No more duplicate service lists to maintain Changes: - Add ServiceRegistry.add_optional_service() for runtime registration - Register epaper service when hardware is detected - Replace hardcoded lists in start_services(), check_services_health(), and show_final_summary() with ServiceRegistry methods
Implement comprehensive TUI using Textual framework for SD card flashing:
Profile Management:
- Load existing profiles with pre-filled values when editing
- Save new profiles with custom names
- Track profile state (loaded vs new) throughout wizard flow
- Pre-populate profile name when editing existing profile
Wizard Flow Improvements:
- Fixed edit workflow to restart from OS selection (full config edit)
- Only show ProfileSaveScreen for new configurations, not loaded profiles
- Added scrolling support for content that exceeds terminal height
- Pre-select all inputs (OS, device, network, etc.) when editing
Select Widget Integration:
- Fixed Textual Select behavior (uses display text, not keys)
- Implemented reverse lookup pattern for all Select widgets
- Show friendly names ("Raspberry Pi OS") in dropdowns, not keys ("raspbian")
- Consistent display name handling across all screens
Device Selection Integration:
- Replaced Rich prompts with Textual TUI screens
- DeviceSelectionForFlashScreen for SD card selection
- ConfirmFlashScreen for flash confirmation
- Standalone DeviceSelectionApp for post-configuration device selection
DietPi Branch Override Support:
- Added repository URL and branch fields to BirdNETConfigScreen
- Modified preserve_installer.sh to source environment variables
- Support BIRDNETPI_REPO_URL and BIRDNETPI_BRANCH for testing
- Automatic DietPi install.sh preservation regardless of checkbox
Bug Fixes:
- Fixed KeyError when displaying final summary (lookup from OS_IMAGES)
- Fixed unused variable warnings (renamed to _key, _idx)
- Fixed profile name not pre-populating when editing
- Fixed SPI option not showing for Pi 4 + Raspberry Pi OS
- Removed unnecessary original_name variable causing save workflow bug
Code Quality:
- Added noqa comments for unavoidable complexity in UI composition
- Fixed line length issues with URL/SHA256 formatting
- Added missing docstring parameters
- Improved comment clarity and formatting
The textual library is an external dependency for the SD card flasher tool and not part of the main codebase dependencies. Add type ignore comments to suppress pyright import errors in CI where textual isn't installed.
mverteuil
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR includes two major improvements to the BirdNET-Pi installation and setup experience:
Part 1: SPI Auto-Enablement
Problem
The e-paper HAT detection was failing because SPI is disabled by default in Raspberry Pi OS:
Without SPI enabled, the
/dev/spidev*devices don't exist, so the HAT detection fails even when the hardware is physically connected.Solution
Early SPI check in
install.sh(lines 88-118):/boot/firmware/config.txtfordtparam=spi=onUser-friendly reboot handling:
Flow
Part 2: SD Card Flasher TUI
Overview
Complete rewrite of the SD card configuration interface using Textual framework, replacing command-line prompts with a guided wizard TUI.
Features Implemented
1. Profile Management
2. Complete Wizard Flow
All configuration steps in a guided TUI:
3. Textual Select Widget Integration
4. Device Selection Integration
5. DietPi Branch Override Support
preserve_installer.shreads frombirdnetpi_config.txtBIRDNETPI_REPO_URLandBIRDNETPI_BRANCHenvironment variablesFiles Changed
New Files:
install/flasher_tui.py(1400+ lines) - Complete Textual TUI implementationinstall/flasher.tcss- Textual CSS stylesheetModified Files:
install/flash_sdcard.py:Bug Fixes
_key,_idx)original_namevariable causing save workflow bugCode Quality
# noqa: C901for unavoidable UI composition complexityUI/UX Improvements
Testing
SPI Auto-Enablement
On a fresh Raspberry Pi with SPI disabled:
install.shSD Card Flasher TUI
CI Status
✅ All checks passing (lint, format, tests, expensive tests)
Benefits