[Proposal] use rspack, pnpm to improve assets build time

[utnapischtim]

Motivation

Improve the time a invenio-cli assets build needs to build the assets.

Summary

• create a create_cli app as small as possible to load as fast as possible
• move from webpack to rspack to improve assets build time from around >50sec to <10sec
• move from npm to pnpm to improve installation time from >60sec to <11sec
• combined improvement from around 3 minutes to 22seconds

PR's

• WIP build: migrate to Rspack invenio-assets#172
• WIP: make invenio-cli assets build faster invenio-cli#377
• Upgrade PDFJS invenio-previewer#203
• WIP: add a create_cli app creator invenio-app#95
• migrate setup and black flask-webpackext#24 (updates WebpackBundle to be independend of current_app and adds PNPMPackage to use pnpm instead of npm)

open problems

pdfjs

the invenio-previewer PR is not only necessary to fix bugs in the pdf viewer but also necessary because the current used version of pdfjs couldn't be build with rspack. Further upgrading to the newest version fixes the build error but doesn't produce a asset which works in the browser. (without testing i hope that this PR fixes both problems)

not yet a PR because it needs some discussion npm -> pnpm, webpack -> rspack

how could we move from npm to pnpm? https://github.com/inveniosoftware/pynpm/blob/1d16a60df39ca5ba23feeca9e3142be35cf69c92/pynpm/package.py#L31 chaning the default value for npm_bin seems the easiest solution but could irritate users of that package. the only other option is to give it to the construction: https://github.com/inveniosoftware/pywebpack/blob/fcb464a46bb88cbb7731689e9869354536fe4df8/pywebpack/project.py#L48 which is also not that easy, because we are in the webpack configuration, but we are moving from webpack to rspack. so should we only change from webpack to rspack and keep the whole pywebpack package, or should we implement that completely new to have the wording correct?

NOTE: pnpm install needs the --shamefully-hoist parameter otherwise the peer dependencies are not found by rspack

test out the configuration for invenio-cli assets watch

not yet tested

with_appcontext

most of the code copied over to the invenio-cli PR has been copied because of the with_appcontext problem. the code in the various packages are based on current_app which needs the app context. this has to be solved somehow to make the rewrite in invenio-cli possible

possible solutions (needs more thinking)

rename Webpack* classes to Asset* classes
move pynpm and pywebpack functionality to invenio-assets and rename the class to the problem they solve without the tool in its name. reason to move: the functionality is used exclusively there (to be proved)
add with_appcontext decorator to the invenio-cli assets build cli command to not have to refactor a lot of code

NOTE: replacing npm with pnpm is still a little bit difficult if pynpm should not be changed? the problem is the parameter --shamefully-hoist which is necessary that pnpm install works

big problem in my eyes at the moment

flask has to be installed by invenio-cli which is not the case at the moment!

[max-moser]

For the PDF.js upgrade in Invenio-Previewer, there's a handful of pull requests that implement @ntarocco 's feedback on the original PDF.js upgrade PR:

• Allow setting of copy instructions per bundle pywebpack#45 – introduce the new copy to/from instruction for the build config
• Read CopyWebpackPlugin patterns from config.json invenio-assets#173 – use the copy instruction in the webpack build
• project: provide defaults for the allowed_copy_paths flask-webpackext#25 – add the instruction to the wrapper, and provide flask-based defaults for the sanity checks
• Upgrade to PDF.js v4 invenio-previewer#211 – the core PR, utilizing the above PRs to copy pre-built pdfjs-dist assets to the static directory, avoiding the requirement to build from source as well as to vendor the assets
• Webpack: Configure copy patterns for TinyMCE invenio-app-rdm#2859 – use the new mechanism for TinyMCE as well

[utnapischtim]

has been implemented in invenio-cli and can be used. not yet the recommended way, but after we start with v14 it could be considered to be the recommended way of developing invenio