Automatic fix and detection of partial installations, option to bundle extra wheels for simpler offline and monorepo builds

[Tremeschin]

Edit (14/04/2025): Apologies for the rambled write-up. In hindsight, not really something that fits the direction of the project. I've since moved on from PyApp, feel free to delete this discussion. Appreciate your time and work regardless!

Original comment (click to expand)

Hi, I've got a fork with two interesting features that adds extra safety and flexibility for monorepos and offline installations.

Pardon the length, I like to add reasoning and context!

Partial installation detection

Problem: Whenever pyapp is closed mid-way an installation, the heuristic in ensure_ready() of checking for the existence of !app::install_dir().is_dir() fails to notice this partial installation, and subsequent runs gives module not found (as it's not.. installed). The only way out is doing a self restore or manually deleting the installation path.

While this is trivial for us programmers, giving such instructions to (Windows) users are annoying and error-prone

Solution: A ready.flag file is created in the root of the installation after install_project(), and ensure_ready() logic now checks for its existence before materialize() and creates it after so. The file works like a checkpoint ensuring previous runs reached the end of the installation correctly! :)

^{I'm not sure if this is the best place to put it, maybe calling self restore at the beginning of run_project() which also works, but could break isolation compatibility}

In the same idea, we can save the current binary hash to a file and call self restore if a same-version but different contents release was run (mostly in dev mode), I also have coded it, but within Python this time, and it works perfectly.

Bundling extra wheels

Problem: In a monorepo scenario like mine, I'm having to bundle all sub-projects packages into a monolithic wheel in PYAPP_PROJECT_PATH with [tool.hatch.build.targets.wheel], and defining main-lib[all] dependency to be installed only for executable releases (dynamically editing pyproject.toml pre-build), this is clunky to say the least.

Solution: While the previous works, having an option to bundle multiple local wheels like in PYAPP_PROJECT_PATH="/A.whl;/B.whl;C.whl.." would solve this issue, just list all wheels out of uv build --wheel --all and do a single pip install *.whl in pyapp when bootstrapping the pyapp project.

The implementation I made is slightly different, I added a PYAPP_EXTRA_WHEELS=a;b;c env that gets installed after PYAPP_PROJECT_PATH, mostly due the already implement complexity and assumptions of the later. But I think the current logic could grab the first file of this new proposed semicolon separated PYAPP_PROJECT_PATH as the main file?

The effects and possibilities of this are:

• If we bundle all packages of uv tree, we pretty much created an easier method for making offline distributions
• Simplifies the build process for monorepos substantially, which uv upstreams and promotes it a lot:
  • It now finds all workspace members than listing manually, allowing for private projects as well
  • No need to list main-lib[all], as the subprojects defines optional deps of the bundled main-lib

The important thing is to pip install all bundled wheels in a single go, as individual wheels would look for pypi for deps not already installed! So joining EXTRA_WHEELS and PROJECT_PATH could do it, we don't need to modify PROJECT_PATH, keeping current compatibility of how it works in essence.

---

Thanks a lot for the work and attention, I hope you consider adding these mechanisms as they'd help me a lot ✨

You can base-off the work in my proof of concept no problem if you do it!