Ownfoil develop / refactor testing

[a1ex4]

Over the past few months, I’ve been working—when time allows—on the biggest refactor since v2. This overhaul fundamentally changes how libraries, files, and identified apps are tracked. It’s a crucial first step toward enabling the upcoming features of Ownfoil.

⚠️ Testing Note

This is a major change. Updating to the latest develop branch will reset your existing library to ensure a clean start and proper migration support.

List of main changes

- Migrated from single table to normalized database schema with Libraries, Files, Apps, and Titles tables
- Database migration system with Alembic integration and automatic backups
- Enhanced file identification system supporting both CNMT metadata and filename parsing
- Better separation of file management and title identification
- File identification status, errors, number of attempts and last attempt date now tracked
- Multi-content file support
- Improved ownership tracking with separate Apps table for missing/owned status
- Refactored app initialization with proper Flask app factory pattern
- Enhanced library scanning with better error handling and progress tracking
- Enhanced UI filtering with ownership status (Owned/Missing) filters
- Show missing BASE / DLC in the UI with a yellow badge

🚫 Before

Previously, everything lived in a single Files table, which stored file paths along with title_id, app_id, version, and the identification method. It was extremely limited:

• Multi-content files couldn’t be tracked.
• Identification errors and attempts weren’t stored.
• All logic for identifying files and linking them to apps, title IDs, and DLCs existed only in the UI—nothing persisted in the backend.

This made backend logic, inspection, or automation impossible.

✅ Now

With this refactor, Ownfoil now supports database migrations—unlocking the ability to evolve the data model without disrupting users. Libraries can now scale, and powerful new features can be added with ease.

Key improvements:

• File identification is now separated from app ownership.

  • The Files table now tracks individual files with rich metadata: identification results, errors, compression type, download counts, multi-content status (fixes split nsp files #19 and Support multi content XCI file #121), etc.
  • This lays the groundwork for managing files directly from the UI: compressing, organizing, deleting, and more.

• A new Apps table tracks app ownership per Title ID.

  • Ownfoil now knows what you have and what you’re missing, across base games, updates, and DLCs.
  • This enables smarter UI features and prepares for automatic indexing and search.
  • Files and apps are fully linked, allowing better duplicate detection and visualization.

---

Feedback is appreciated, I'm testing with a small, personal library, and I'm sure your experience will differ and we will need further improvements.

[grantbi]

Getting this error. I'm using 2.1.1

Exception in thread Thread-9:

ownfoil  |   File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
ownfoil  |     self.run()
ownfoil  |   File "/usr/local/lib/python3.11/threading.py", line 1401, in run
ownfoil  |     self.function(*self.args, **self.kwargs)
ownfoil  |   File "/app/utils.py", line 43, in call_it
ownfoil  |     fn(*args, **kwargs)
ownfoil  |   File "/app/app.py", line 475, in post_library_change
ownfoil  |     titles_library = generate_library()
ownfoil  |                      ^^^^^^^^^^^^^^^^^^
ownfoil  |   File "/app/library.py", line 553, in generate_library
ownfoil  |     'hash': compute_apps_hash(),
ownfoil  |             ^^^^^^^^^^^^^^^^^^^
ownfoil  |   File "/app/library.py", line 420, in compute_apps_hash
ownfoil  |     for app in sorted(apps, key=lambda x: (x['app_id'], x['app_version'])):
ownfoil  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ownfoil  | TypeError: '<' not supported between instances of 'NoneType' and 'str'

I'm guessing it's a filename? Not sure. It just errors and then the library no longer shows up.