Skip to content

Added clean up language files tool#10

Merged
k9ert merged 30 commits intok9ert:mainfrom
maggo83:Create-CleanUpLanguageFiles-Tool
Feb 20, 2026
Merged

Added clean up language files tool#10
k9ert merged 30 commits intok9ert:mainfrom
maggo83:Create-CleanUpLanguageFiles-Tool

Conversation

@maggo83
Copy link
Copy Markdown
Collaborator

@maggo83 maggo83 commented Jan 15, 2026

Added a developer tool to synch i18n language files with gui and each other

maggo83 and others added 30 commits January 15, 2026 17:04
…trings and each other

-Find and remove strings not used in GUI anymore
-Find and add strings used in GUI but not in files (add <FILL> as placeholder text)
-Update english reference texts in translation files when needed
Add flashable MockUI firmware with build target, manifest, and boot/main
entry points. Fix two issues blocking firmware operation:

1. Icon MemoryError: Switch icon format from ARGB8888 to A8 (alpha-only)
   in icon.py. The pattern bytes from btc_icons.py are used directly as
   A8 image data (zero heap allocation, references frozen bytecode in
   flash). Color is applied via LVGL image_recolor style. This eliminates
   the 3120-byte contiguous allocation that failed on the fragmented
   245KB MicroPython heap.

2. Missing i18n translations: Create translations_embedded.py with
   English translations as a Python dict. Modify i18n_manager.py to fall
   back to embedded translations when JSON files aren't available (frozen
   firmware can't include JSON). Simulator still uses JSON-first loading.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ap; added size sanity check

-After refactoring to A8 instead of ARGB format that parameter is not needed anymore

-W/o size check unintended memory access could be possible
- Replace JSON-based i18n with efficient binary format for embedded systems
- Add lang_compiler.py for JSON→binary conversion with validation
- Implement flash filesystem storage (/flash/i18n/) for runtime language loading
- Update i18n_manager.py to use binary files with zero RAM string storage
- Add build system integration with automatic i18n compilation
- Support both embedded (data/lang/) and user-added (flash) language files
- Add comprehensive translation key management and validation
- Update documentation with binary format specifications

Technical changes:
- Binary format: 12-byte header + offset index + null-terminated strings
- Search paths: data/lang (embedded) → /flash/i18n (user languages)
- Lazy loading: strings read directly from flash on demand
- Fallback system: missing translations use default language
- Build integration: make mockui now includes build-i18n step

Known issue: Firmware hangs during boot, possibly due to early flash access
get_language() can return None, which needs proper handling in gui
… changes in i18n_manager.py

-removed unnecessary folders/files from build process: only i18n folder or build folder are used for files created during build
-language files are stored during build in FAT filesystem, not littleFS (as this matches STM32 Micropython default)
-made i18n_manager handle the fact that FAT uses upper case filenames correctly
-re-added get_language_name (should not have been removed -> not used by i18n itself, but by language_menu)
… from file

-needed to add language name in binary file (read during conversion from JSON)
this is a general tool for debugging, not specific to MockUI hence, moved in the general /tool folder
to more clearly separate infrastructure/code from data files
To match current design, workflow and implementation status
E.g. for debugging
Add flashable MockUI firmware with build target, manifest, and boot/main
entry points for the STM32F469 Discovery board.

Key changes:
- Add mockui_fw/ scenario with boot.py and main.py entry points
- Add manifest and Makefile build target for 'make mockui'
- Fix icon MemoryError: switch from ARGB8888 to A8 alpha-only format
  (eliminates 3120-byte allocation that failed on fragmented heap)
- Add size sanity check to icon bitmap creation
- Fix status bar crash when no language is available
- Enable USB at startup for debugging
Replace JSON-based i18n with an efficient binary format optimized for
MicroPython on resource-constrained embedded systems.

- Add lang_compiler.py for JSON-to-binary conversion with validation
- Binary format: 12-byte header + offset index + null-terminated strings
- Implement flash filesystem storage (/flash/i18n/) for runtime loading
- Lazy loading: strings read directly from flash on demand (zero RAM)
- Fallback: missing translations use default language
- MicroPython-compatible: use os module instead of platform/pathlib
- Remove translations_embedded.py in favor of binary files
Add build system integration for compiling and deploying i18n binary
files to the STM32F469 Discovery board's flash filesystem.

- Add make_fat_image.py for creating FAT filesystem images
- Add merge_firmware_flash.py for combining firmware with flash data
- Integrate i18n compilation into 'make mockui' build pipeline
- Change precompiled filesystem from LittleFS to FAT (matches STM32
  MicroPython default)
- Handle FAT uppercase filename convention in i18n_manager.py
- Add language name lookup from binary file metadata
- Organize JSON source files into dedicated languages/ folder
Rewrite i18n README to match current binary format design, covering
architecture, build workflow, file format specification, and usage
instructions for adding new languages.
- Unit tests for lang_compiler.py (binary compilation, validation)
- Unit tests for i18n_manager.py (loading, fallback, filesystem)
- Integration tests for end-to-end i18n workflow
- Hardware-in-the-loop tests for STM32F469 device validation
- Shared test fixtures via conftest.py
Move sync_i18n.py from i18n/helpers/ to tools/ alongside the other
build-time utilities, and update it substantially:

Key detection:
- Add recognition of all i18n access patterns beyond t("KEY"):
  i18n["KEY"], i18n("KEY"), i18n_manager["KEY"], i18n_manager("KEY"),
  single- and double-quote variants of all of the above

Language file discovery:
- Use extract_language_code_from_filename() from lang_compiler.py to
  enforce naming conventions (specter_ui_XX.json); non-conforming files
  are skipped with a warning instead of silently included

CLI interface:
- Rename --i18n-dir to --languages-dir (points directly at languages/)
- Default log output goes to build/ instead of inside the source tree

Logging:
- Rewrite to write logs incrementally (open/write/close per entry),
  so partial logs survive crashes
- Log files are initialised with a header before any work starts;
  if --dry-run, "*** DRY RUN — no changes will be made ***" appears
  as the second line, not buried at the end
- Always write logs, including during dry-run

FILL_PLACEHOLDER:
- Define FILL_PLACEHOLDER = "<FILL>" in lang_compiler.py as the single
  source of truth; imported by sync_i18n.py to keep them in sync
- Add runtime check in i18n_manager.t(): if the resolved translation
  equals FILL_PLACEHOLDER, fall back to English; if even English has
  it, return STR_MISSING — the placeholder never reaches the UI

Build integration:
- Add sync-i18n Makefile target (runs --dry-run to detect drift without
  modifying files); make it the first dependency of build-i18n so every
  build logs the current sync state
- Update README.md: file structure, per-component docs, build targets,
  and a dedicated sync tool section with CLI examples
@maggo83 maggo83 force-pushed the Create-CleanUpLanguageFiles-Tool branch from ec0f068 to a890142 Compare February 20, 2026 18:23
@k9ert k9ert merged commit a6027fb into k9ert:main Feb 20, 2026
1 check passed
k9ert added a commit that referenced this pull request Feb 20, 2026
@maggo83 maggo83 deleted the Create-CleanUpLanguageFiles-Tool branch February 21, 2026 07:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants