Installer plugin demo

[soapy1]

This demo demonstrates using conda installer plugin hooks to make conda install/create/update commands support installing pip packages.

This is achieved by:

• introducing a new install plugin hook
• the installer plugin hooks are only responsible for installing non-conda packages. Conda packages will continue to be handled by core conda.
• 2 sample plugin implementations that install packages form pypi.
  • The pip plugin, shells out to pip to install packages
  • The uv plugin, does not actually do anything - it just outputs to the terminal
• Plugins can specify which type of package they can install - these have a relationship with the Environemnt.external_packages field, that is populated by environment spec plugins.
• Any plugin can install many types of packages. Plugins can overlap in the packages that they can install.
• Users may select their preferred installer with the config var <installer_type>_preferred_installer

Note, this is just a demo! Lots of the code is ugly and many things are not quite right 💔. Despite that, here are some helpful command to demonstrate this approach:

Consider a conda installation with the following .condarc

channels:
 - conda-forge
plugins:
 pip_preferred_installer: uv

Let’s also consider the following tests/env/support/with-pip.yml

name: pip
dependencies:
 - python > 3.11
 - numpy
 - pip
 - pip:
   - scipy

1. Running a create command without the provided condarc will result in an error. Since both the pip plugin and uv plugin are configured to be viable backends for installing pip packages

$ conda env create --file tests/env/support/with-pip.yml
Channels:
 - defaults
 - conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

PluginError: Too many installers registered for 'pip': 
  - pypi
  - uv

Set the default installer for package type `pip` by configuring the setting `pip_preferred_installer`.
For example:
  `export CONDA_PLUGINS_PIP_PREFERRED_INSTALLER=uv`.

2. Running the create command with the condarc from above applied results in conda correctly selecting UV as the backend for installing pip packages.

$ conda env create --file tests/env/support/with-pip.yml
Channels:
 - defaults
 - conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done
pretending to install stuff with uv
  - scipy

3. Users may also set the config from the command line.

$ CONDA_PLUGINS_PIP_PREFERRED_INSTALLER=pypi conda env create --file tests/env/support/with-pip.yml
Channels:
 - defaults
 - conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done
Installing pip dependencies: | Ran pip subprocess with arguments:
['/home/sophia/projects/conda/devenv/Linux/x86_64/envs/devenv-3.10-c/envs/pip/bin/python', '-m', 'pip', 'install', '-U', '-r', '/tmp/condaenv.r3hisznb.requirements.txt', '--exists-action=b']
Pip subprocess output:
Requirement already satisfied: scipy in ./lib/python3.13/site-packages (from -r /tmp/condaenv.r3hisznb.requirements.txt (line 1)) (1.16.1)
Requirement already satisfied: numpy<2.6,>=1.25.2 in ./lib/python3.13/site-packages (from scipy->-r /tmp/condaenv.r3hisznb.requirements.txt (line 1)) (2.3.1)

done

4. Conda will produce an error if the user sets an unavailable preferred installer

$ CONDA_PLUGINS_PIP_PREFERRED_INSTALLER=idontexist conda env create --file tests/env/support/with-pip.yml
Channels:
 - defaults
 - conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

CondaValueError: Could not find preferred installer plugin 'idontexist' for installing package from 'pip'.

5. conda env create and conda create commands can be use (almost) interchangeably:

$ CONDA_PLUGINS_PIP_PREFERRED_INSTALLER=pypi_uv conda create --file tests/env/support/with-pip.yml -n testpip
Channels:
 - defaults
 - conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/sophia/projects/conda/devenv/Linux/x86_64/envs/devenv-3.10-c/envs/testpip

  added / updated specs:
    - numpy
    - pip
    - python[version='>3.11']

. . .

Proceed ([y]/n)? y


Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
pretending to install stuff with uv
  - scipy
 . . .

Notes: https://docs.google.com/document/d/1ca1jvxV1a82wS-veU3WYwoP9fk6I7DljzjUSB2Y8yZ4/edit?tab=t.0

[soapy1]

Installing conda packages does not touch the plugin system.

[soapy1]

Just for the purposes of demoing conda env create and conda create being used (almost) interchangeably.

[soapy1]

clearly not actually implementing an install with uv. It's helpful to have this output for visual confirmation that the right plugins are getting invoked.

[soapy1]

These types are mapping to the Environment.external_packages dict. The environment.yaml environment spec plugin will put all pip depenencies in the pip key of that dict. This relationship is not well definied. It would be better if it could be more strict.

[soapy1]

I'm not super confident about how well this works for overlapping plugin types. Needs more testing.