Skip to content

Conversation

@utnapischtim
Copy link
Contributor

@utnapischtim utnapischtim commented Nov 18, 2024

this PR is a combination of

this PR should not be merged as it is. it is a starting point to discuss improvements!

discussion points

  • some arguable (provoking) performance improvements e.g. reverted. setting it over the invenio call is necessary
  • should we go in and change without backwards compatibility to uv + rspack + pnpm? no, it will be configurable which tools will be used by the developer
  • should the lock file approach be applied completely? b6bbbe2
  • making python packages manager configurable would keep invenio-cli backwards compatible commit yes
  • making javascript packages manager configurable would keep invenio-cli backwards compatible commit needs commit yes
  • making assets builder configurable would keep invenio-cli backwards compatible commit needs commit invenio-assets commit has to be finished if we use the approach (webpack.config.js has to be reverted back to real webpack configuration) yes
  • how make it possible to use invenio-cli installed globally without doing sys.path.insert like

steps to how to use it

1.) check that uv and pnpm are installed
2.) create a new my-site directory with the regular content as usual OR go into your current development my-site directory
2.a) deactivate the active environment
3.) create with uv a virtual environment with uv venv --prompt uv-env && source .venv/bin/activate
4.) install invenio-cli with uv pip install "git+https://github.com/utnapischtim/invenio-cli@WIP-merged-up-uv-ports-branches"
5.) copy following text into the .invenio file into the cli section

python_packages_manager = uv
javascript_packages_manager = pnpm
assets_builder = rspack

5.a) please copy following to invenio.cfg

JAVASCRIPT_PACKAGES_MANAGER = "pnpm"
ASSETS_BUILDER = "rspack"

6.) copy following text into the pyproject.toml file. the file should be created beside the invenio.cfg file

[project]
name = "InvenioRDM"
requires-python = ">= 3.12"
dynamic = ["version"]

dependencies = [
  "invenio-app-rdm[opensearch2]~=13.0.0b2.dev2",
  "uwsgi>=2.0",
  "uwsgitop>=0.11",
  "uwsgi-tools>=1.1.1",

  # keep the custom invenio-cli installed in the virtual environment
  # could possible create problems with global installed invenio-cli
  # because the path to the virtual environmental invenio-cli will not
  # be updated
  "invenio-cli",

  # rspack
  "flask-webpackext",
  "invenio-assets",
  "pywebpack",
]

[tool.setuptools]
py-modules = [] # necessary to make the packages with setup.py usable with uv

# overrides packages from "dependencies"
[tool.uv.sources]
invenio-cli = { git = "https://github.com/utnapischtim/invenio-cli", branch = "WIP-merged-up-uv-ports-branches" }
flask-webpackext = { git = "https://github.com/utnapischtim/flask-webpackext", branch = "make-ready-for-rspack" }
invenio-assets = { git = "https://github.com/slint/invenio-assets", branch = "rspack" }
pywebpack = { git = "https://github.com/utnapischtim/pywebpack", branch = "add-scripts" }
# invenio-cli = { path = "path/to/invenio-cli", editable = true } # would be local example

7.) run time invenio-cli install to install and to see how long it takes
8.) run time invenio-cli install again to see the behavior with hot cache

utnapischtim and others added 13 commits November 5, 2024 15:20
* this gives a huge performance improvement on installing and setting up
  an environment.
* Explicitly makes the Celery worker listen to the "default" and "low"
  queues.
* the problem was that the packages which are installed editable and
  will be used on the assets build step arent't found if the assets
  build method is called during the invenio-cli install command.
* this commit assumes that the instance path isn't customizable and
  therefore removes the necessity of starting up the app to get the
  instance path.

* the case where the instance path is overridable should be handled in
  another commit.
@utnapischtim utnapischtim force-pushed the WIP-merged-up-uv-ports-branches branch from d9fc688 to 128053b Compare November 18, 2024 14:25
@utnapischtim utnapischtim force-pushed the WIP-merged-up-uv-ports-branches branch 4 times, most recently from 94f5361 to 79603c3 Compare January 18, 2025 19:54
@utnapischtim utnapischtim force-pushed the WIP-merged-up-uv-ports-branches branch from 79603c3 to bc977c8 Compare January 18, 2025 20:12
* to create the assets create_ui is enough. the assets only need the
  blueprints and they exists in create_ui. creat_ui only creates one
  app and not two as create_app does
* since rspack configuration is not more in webpack.config.js this is
  necessary
@utnapischtim utnapischtim force-pushed the WIP-merged-up-uv-ports-branches branch from f48e721 to 5ddbc3f Compare January 21, 2025 13:12
@max-moser
Copy link
Contributor

max-moser commented Feb 18, 2025

for the record, the configuration needs to be added into the [cli] section at .invenio:

[cli]
flavour = RDM
logfile = /logs/invenio-cli.log
javascript_packages_manager = pnpm
python_packages_manager = uv

⚠️ note that there are no quotation marks around the values (i.e. uv, not "uv").

* the command is for both assets builder the same, because the
  configuration has been moved to another place
@utnapischtim utnapischtim force-pushed the WIP-merged-up-uv-ports-branches branch from 466efff to f5a6949 Compare February 18, 2025 19:58
@utnapischtim utnapischtim force-pushed the WIP-merged-up-uv-ports-branches branch from 1ef694e to 3c96ce3 Compare February 19, 2025 20:59
* change in flask
* this enables again the option to have invenio-cli installed globally
* there are use cases where the instance_path needs to be updated, it is
  safer to run that every time
@utnapischtim utnapischtim force-pushed the WIP-merged-up-uv-ports-branches branch from 3c96ce3 to f483fc1 Compare February 19, 2025 21:12
@utnapischtim
Copy link
Contributor Author

superseded by #383

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants