Best practices for integrating Dockerized tools (Verilator) with FuseSoC?

[eminakgun]

Description

We are developing a SystemVerilog library and aiming for a reproducible build and test environment. To achieve this, we want to use a specific version of Verilator running in a Docker container, rather than relying on the system-installed version.

We are currently struggling to find a "native" or recommended way to configure FuseSoC to use containerized tools.

Current Approach (Workaround)

Our current solution involves "tricking" FuseSoC by manipulating the PATH to point to a wrapper script that invokes Docker.

1. Core File (project.core):
   We define our targets using the standard verilator tool definition:

   targets:
     lint:
       default_tool: verilator
       filesets: [rtl]
       tools:
         verilator:
           mode: lint-only

2. Wrapper Script (.fusesoc/verilator):
   We created a script named verilator that acts as a shim:

   #!/bin/bash
   sudo docker run --rm \
     -v $(pwd):/work \
     -w /work \
     verilator/verilator:latest \
     "$@"

3. Invocation Script (fusesoc-docker.sh):
   We run FuseSoC via a script that prepends our wrapper directory to the PATH:

   export PATH="$(pwd)/.fusesoc:$PATH"
   fusesoc "$@"

The Problem

While this "works" for simple cases, it feels fragile and hacky:

• Path Issues: It relies on volume mounting $(pwd) to /work. If FuseSoC passes absolute paths that are outside of $(pwd) (e.g., from a cache directory or a dependency in another location), the container won't see them.
• Permissions: It requires sudo (or a configured docker group), which complicates CI/CD environments.
• Tool Awareness: FuseSoC doesn't know it's running a container, so it can't intelligently handle things like environment variables or file paths that need translation.

The SVUnit Complication

The SVUnit Complication

The situation gets even more complex when integrating SVUnit. SVUnit is a framework that generates a test runner and then invokes the simulator. We run this via a hook_script in our core file.

1. Core File Configuration (project.core):
   We define a script that runs the tests and attach it to the post_run hook of our test target.

   scripts:
     run_tests:
       filesets:
         - hook_scripts
       cmd:
         - sh
         - -c
         - "$FILES_ROOT/test/run_svunit.sh"
   
   targets:
     test:
       default_tool: verilator
       filesets:
         - rtl
         - hook_scripts
       tools:
         verilator:
           mode: lint-only
       hooks:
         post_run:
           - run_tests
       toplevel: TopLevel_pkg

2. The Script (test/run_svunit.sh):
   This script handles the complex Docker invocation required to run SVUnit inside the container.

   #!/bin/bash
   
   # ... checks for SVUNIT_INSTALL_PATH ...
   
   sudo docker run --entrypoint bash \
     -e SVUNIT_INSTALL=/eda/svunit \
     -v ${SVUNIT_INSTALL_PATH}:/eda/svunit \
     -v ./${FILES_ROOT}:/work \
     -w /work/test \
     verilator/verilator:latest \
     -c "cd \$SVUNIT_INSTALL && source Setup.bsh && cd /work/test && \
           runSVUnit -s verilator --c_arg '-I../src'"

This setup forces us to:

• Mount multiple volumes: The project code and the SVUnit installation (which lives on the host) must be mounted into the container.
• Handle Environment Variables: We need to pass SVUNIT_INSTALL_PATH from the host to the container.
• Bypass FuseSoC: The command running inside the container is a complex chain that FuseSoC is unaware of. We are essentially just using FuseSoC as a glorified make target that calls a shell script.

Questions

1. Is there a native way to specify a container image for a tool?
   Ideally, we'd like something like:

   tools:
     verilator:
       container: verilator/verilator:latest

   Or a global configuration to map tools to containers.

2. How should we handle file paths?
   If we stick to the wrapper approach, is there a standard way to ensure all necessary files (including dependencies managed by FuseSoC) are available to the container?

3. Are there examples of "Flow API" usage for this?
   We looked into the Flow API but weren't sure if creating a custom flow just to wrap Docker is the right path, or if there's a simpler configuration we missed.

We would appreciate any guidance on the "best practice" way to achieve reproducible, containerized builds with FuseSoC.

[tymonx]

I would consider two things:

• A) Running FuseSoC inside container
• B) FuseSoC is running containers as part of flow

Option A) is the most straightforward Way-of-Working with CI and locally. It doesn't require any support from FuseSoC. Let's consider GitLab CI as CI environment (GitHub Actions or Jenkins will be similar). .gitlab-ci.yml file as example:

tests:
    image: registry.gitlab.com/<group>/verilator:<version>
    script:
        - fusesuc run <name>

Working locally:

podman run -it --rm --userns keep-id --workdir "$(pwd)" --volume "$(pwd):$(pwd):rw,z" registry.gitlab.com/<group>/verilator:<version> fusesoc run <name>

In this scenario I would recommend to create own container images and install at least Python + pip/uv + specific tool + other required dependencies.

You could consider using container devs, toolbx or distrobox to create long-lived containers. Then entering container environment would be just distrobox enter <tool-name>-<version>. Or do it manually with docker|podman create + start + exec commands. And normally run fusesoc commands inside container.

Option B)

• https://blog.award-winning.me/2022/10/from-simulation-to-soc-with-fusesoc-and.html

There is el_docker script as part of the Edalize project. Usage:

EDALIZE_LAUNCHER=el_docker fusesoc run <name>

But running this in CI requires Docker-in-Docker or Podman-in-Podman and it will pull container images each time when CI job will run. There are some tricks to avoid that by sharing docker/podman socket with host or other long-lived service, sharing storage for container images or using local container registry.

You could mix two approaches 🤔 Use approach A) in CI and el_docker for local development. I personally prefer option A). We have similar command line interface to toolbx or distrobox but focused in security hardening because these utilities are relaxing containers too much like mounting home directory, using net/host, creating containers with root user etc.

[eminakgun]

Thanks for the detailed breakdown.

To give some context on why I initially attempted the wrapper script approach: My system package manager provided a very outdated version of Verilator. I wanted to use the latest version without the overhead of building from source or polluting my local environment. My goal was to simply consume the verilator/verilator:latest image directly, avoiding the need to write or maintain a custom Dockerfile.

I still don't want to prepare Dockerfiles but I'll need to get a similar github file for the CI and have a better way of development locally. Personally I don't want to induce too much of tooling since this is a something that I play with in my free time, I want to focus more on the project itself. container devs looks like a good option, I'll evaluate others as well. On the other hand I'm still open to suggestions.

[olofk]

I'm running containerized EDA tools through FuseSoC/Edalize all the time. Try running EDALIZE_LAUNCHER=el_docker fusesoc run ... and see if that works for you.

The magic is that when you set the EDALIZE_LAUNCHER environment variable, the command defined in that variable will be used instead of calling the EDA tool directly. So when Edalize would normally call verilator -f some_file, it will instead run el_docker verilator -f some_file. The el_docker script then downloads and runs a container with the requested EDA tool. el_docker comes installed with Edalize and you can find the script here if you want to see for yourself or make your own version

[olofk]

Ok, my reading comprehension apparently isn't very good. Completely missed that you had already discovered el_docker. Nevermind then

[olofk]

Ok, one more thing related to the Flow API question. Edalize will by default create a Makefile that executes the tool graph, but since quite recently it is possible to override this with any custom code using a build runner (See https://edalize.readthedocs.io/en/latest/user/index.html#build-runners). That could potentially be useful here too.