Use C++ package managers instead of build scripts

[Risto97]

Currently, we have custom CMake functions for building libraries and tools: https://github.com/HEP-SoC/SoCMake/tree/master/cmake/build_scripts

This can work quite well, but it's inefficient for us to support this.
Since there are already projects that deal with this issue, we should consider just using those.

Here is a list of C++ package managers that I tried:

1. Hunter

• Implemented on CMake
• Integrates into CMake project
• Looks similar to current build_scripts

To add package, simply call in the CMake project

hunter_add_package(ZLIB)

More examples here https://github.com/cpp-pm/gate

2. vcpkg

• Implemented in CMake
• Developed by Microsoft
• Can be used in CLI mode or in manifest mode, where it requires separate manifest json file
• Only manifest mode can do versioning of packages
• The CMake build scripts are unaware of vcpkg, need to pass -DCMAKE_TOOLCHAIN_FILE when configuring

3. conan

• Implemented in Python, which makes it a bit tricky to use
• Has a conan center repository of packages https://conan.io/center
• In theory already has Verilator and SystemC packages, but they seem outdated.
• Should have good CMake integration

#139
@benoitdenkinger

[benoitdenkinger]

@adrianf0 is currently investigating the use of Nix to better manage system configuration and package dependencies. It seems to be a very powerful and helpful tool. The integration with cmake should also come for free as Nix is playing with env variable to setup your system level configuration.

@Risto97 Do you have experience with Nix? @adrianf0 is testing it on a new internal project but we think this could be a good solution for our tools deployment in general. We are also discussing/negotiating the installation of it in our private cluster.

[adrianf0]

Hi @Risto97 @benoitdenkinger, just for clarity, the idea is to use nix-shell, which provides declarative shell environments and can install specified packages from the Nixpkgs collection, e.g., Verilator, Verible, etc. It also supports defining custom packages. This would satisfy the software dependencies through nix-shell and let SoCmake focus on the actual project's source.

[Risto97]

Hello @adrianf0, Welcome to the project.

@benoitdenkinger @adrianf0
I have never used Nix, but I am aware of it.
I think its a great way of getting the dependencies, and with what I listed in the first comment is one of the possibilities for the users to get the dependencies.

The initial build scripts came because we did not have possibilities to install any packages without root permissions, so the only option was to build it from source.
Nix was probably option also at that time as it could been run as user without root permissions, but I didn't try it.

With that said, it is probably a good idea to leave this to the users to decide on what they would use.
What we could do is maybe provide a page in the documentation or a blog post on how to use these package managers with SoCMake.
That would mean we would delete the current build scripts to reduce the maintenance effort, which I am in favor of.

And as Adrian said focus on managing project sources, EDA tools and HW-SW co-simulation.

[Risto97]

BTW.

If we want to have a more tighter integration with Nix or any other package manager, we can consider something like https://github.com/Risto97/uvtarget does for Python dependencies, where Python packages are installed at configure time and virtual environment is created.

So maybe at configure time Nix environment can be created and packages installed.

---

Another option is to be package manager agnostic, and use the official CMake Dependency Provider API https://cmake.org/cmake/help/latest/command/cmake_language.html#dependency-providers.

The purpose of this is that you can intercept the find_package() calls and route them to the package manager you chose.
The build scripts would then be agnostic of the fact that you are using Conan, VCPKG or Nix.

You would have something like this:

find_package(Verilator CONFIG)
find_package(Verible CONFIG)

And this calls would be routed to the dependency provider to fetch it.

[adrianf0]

That would mean we would delete the current build scripts to reduce the maintenance effort, which I am in favor of.

I also agree that SoCMake should not take care of installing any software dependencies (including Python tools), and this part should be removed from the project.

I quickly reviewed the three possibilities listed in the first comment. What I noticed is that Hunter seems to have a quite limited set of packages. vcpkg seems to focus on C++, and I haven't noticed Python packages provided by it. For Conan, I have a similar impression to @Risto97 (Python-based, outdated packages).

In the workflow I am trying, nix is responsible for managing all dependencies. This means it also sets up SoCMake itself, CMake, and all Python dependencies (including those not related to SoCMake, like pre-commit). This way, the dependencies of different projects are isolated on my host machine.

I went quickly through uvtarget, and in the nix workflow, it may not even be necessary. nix-shell can prepare the Python environment with all required packages. It can also populate a project-specific Python virtual environment if venv and pip (over direct nix package installation) are preferred or required (e.g., if a Python package is missing from the nix package repository).

At the top level of a project, all dependencies should be defined in the nix-shell configuration file, and the nix tool will handle the rest. The top-level configuration could potentially also reference nix-shell configuration files from subprojects (e.g., submodules) — to be verified.

If a user chooses not to use nix-shell, they can still find all project dependencies listed in the nix configuration file. Since package names are often portable, this list could be used with a local distribution's package manager, though dependency versions will not be guaranteed.

Another option is to be package manager agnostic and use the official CMake Dependency Provider API: https://cmake.org/cmake/help/latest/command/cmake_language.html#dependency-providers

I believe this is intended for tools like Conan and vcpkg, where, as I understand, packages can be added dynamically. nix-shell is declarative; dependencies are specified in its configuration file. It is possible to spawn a nested nix-shell session from within the current one using the -p (or --packages) flag. Packages added this way are available only temporarily in the nested shell, but this is rather an unintended approach.

However, nix-shell already does what dependency providers aim to do:

• Automatically provides all dependencies via environment variables
• Sets CMAKE_PREFIX_PATH automatically
• Ensures reproducibility across machines
• Requires no additional configuration

Dependency providers require:

• Extra CMake setup (e.g., via CMAKE_PROJECT_TOP_LEVEL_INCLUDES)
• A custom .cmake provider script
• Only one provider per CMake run