IP block installation

[Risto97]

Experimental support for installing IP blocks.

The idea is to install an IP block that does not have any targets, instead, all the generated files are statically included in the package.
Additionally, it should be possible to install the elaborated simulation databases for different simulators, so when the importing projects link to the installed simulation database, it should only add it to the search path, and not recompile the source files.

A new function is provided ip_install() that installs the IP block to a new location.

Compared to the installation of C/C++ libraries, we have a bit of a different problem.

In C++ installation, typically only the static, shared libraries or executables are installed as binary files.
In addition to that, header files can also be installed, typically in the case of header files a whole include directory could be copied to the new location.
In C++ source .cc files are not installed.

This would be equivalent in HW flow where only the simulation database is installed but the HDL source files are not.
Although this is a valid approach, in HW flows it is often needed to deliver also the generated HDL files without the need to run the generators on the IP consumer user side.
The consumer can then use these generated HDL files to run simulation, synthesis, implementation etc...
Often there might also be a requirement to export the SoCMake's CMake recipe into something more build system agnostic, like IPXact, json, or a simple txt file with the list of files.

In case of installing source files, a problem is deciding on where should the new source files be placed?

• Should they all be dumped into a single directory?
• What happens when there are 2 files in the IP with the same file name, but are in different directories?
• Does the filesystem hierarchy of the files and their distance from each other need to be the same?
• What to do when a source is given to the original IP as an absolute path, somewhere outside the original IP source directory?

Similar issues exist also for the include directories, where the include directories of the IP, need to keep the header files in the same directory tree relative to include directories in the destination directory.
In case of include "inc_dir/some_dir/header.svh" no directory levels can be lost to not break the functionality.

[benoitdenkinger]

Regarding the various open questions:

For source files:

• For the installation/export of hardware designs, I think it would be nice to translate the VLNV into a folder hierarchy. This would give a very clear export.
• Ideally, each file should have the name of the module it contains, which should be unique. With the hierarchy proposed above, two files with the same name shouldn't be a problem, as long as they have modules with different names.
• The original file hierarchy can be ignore and this shouldn't be a problem if the IPs are properly defined, this also remove the problem of absolute path outside the original IP source directory.

For header files:

For include directories, the situation is different and the hierarchy must be preserved if include path sare used. The same problem arises for software libraries and the target_sources command with FILE_SETS and BASE_DIR arguments is there for that.

The use of FILE_SETS is also mentioned in #123 to distinguish various configuration (e.g., simulation, ASIC, FPGA), so it could be one more reason to investigate this solution.

[benoitdenkinger]

Is the example supposed to work? For me it does not. I get this error:

-- Install configuration: ""
CMake Error at cmake_install.cmake:41 (file):
  file cannot create directory: /usr/local/VERILOG.  Maybe need
  administrative privileges.

It looks like it tries to install the files under /usr/local/VERILOG

[Risto97]

Hello @benoitdenkinger ,

Yes, that works correctly.
It is consistent with where CMake installs by default, by default it will try to install the package system-wide, in /usr/local.
I do agree that that doesn't make too much sense for hardware IPs, because why would you want IP installed on the system.

To change the install directory, you need to pass -DCMAKE_INSTALL_PREFIX=<dir> to where you want to install it instead.
Similar to autotools --prefix argument

[Risto97]

For the installation/export of hardware designs, I think it would be nice to translate the VLNV into a folder hierarchy. This would give a very clear export.

I am not sure I fully understand, are you saying to have something like this:
If VLNV is vendor::lib::ip1::0.0.1 the destination directory structure would be:
vendor/lib/ip1/ip1.sv

Ideally, each file should have the name of the module it contains, which should be unique. With the hierarchy proposed above, two files with the same name shouldn't be a problem, as long as they have modules with different names.

This would be ideal, but I am not sure this can be enforced easily

The original file hierarchy can be ignore and this shouldn't be a problem if the IPs are properly defined, this also remove the problem of absolute path outside the original IP source directory.

I agree, this is probably the way to go

For header files:

For include directories, the situation is different and the hierarchy must be preserved if include path sare used. The same problem arises for software libraries and the target_sources command with FILE_SETS and BASE_DIR arguments is there for that.

The use of FILE_SETS is also mentioned in #123 to distinguish various configuration (e.g., simulation, ASIC, FPGA), so it could be one more reason to investigate this solution.

Yes there are some good ideas, I am currently working with IPXact, and I am looking into how FileSets are used there.
There is a draft pull request to use IPXact files directly for file lists, maybe that can drive the implementation of FileSets.
#138

[benoitdenkinger]

I am not sure I fully understand, are you saying to have something like this:
If VLNV is vendor::lib::ip1::0.0.1 the destination directory structure would be:
vendor/lib/ip1/ip1.sv

Yes, even:
vendor/lib/ip1/version/ip1.sv

This open the possibility to have multiple versions of the same IP

There is a draft pull request to use IPXact files directly for file lists, maybe that can drive the implementation of FileSets.

Do you mean that in IPXact you can define any fileset? In that case, again, the question is how to support these filesets on SoCMake (see #123). Or did you mean something else?

[Risto97]

Yes, even:
vendor/lib/ip1/version/ip1.sv

I think this is a great idea.
This is how Leon2 IPXact examples from Accellera are organized, take a look here:
https://www.accellera.org/images/activities/committees/ip-xact/Leon2_1685-2022.zip

We can even think of organizing the default binary directories for targets like that.
Now its something like:
${VENDOR}__${LIBRARY}_${NAME}__${VERSION}__${FUNCTION}
We can instead have:
${VENDOR}/${LIBRARY}/${NAME}/${VERSION}/${FUNCTION}
Gets rid of the nasty underscores.

[Risto97]

Do you mean that in IPXact you can define any fileset? In that case, again, the question is how to support these filesets on SoCMake (see #123). Or did you mean something else?

So IPXact can have arbitrary filesets, the fileSet element looks something like this:

   <ipxact:fileSets>
      <ipxact:fileSet>
         <ipxact:name>fs-vhdlSource</ipxact:name>
         <ipxact:file>
            <ipxact:name>hdlsrc/gpio.vhd</ipxact:name>
            <ipxact:fileType>vhdlSource</ipxact:fileType>
            <ipxact:logicalName>gpio_lib</ipxact:logicalName>
         </ipxact:file>
      </ipxact:fileSet>
   </ipxact:fileSets>

You can have arbitrary number of ipxact:fileSet elements in ipxact:fileSets element.
You can also have arbitrary number of ipxact:file elements in ipxact::fileSet element.
ipxact:fileType is LANGUAGE in SoCMake.

On #138 , it is already implemented to extract the fileType and file list from the ipxact:fileSet.
We could also extract the name of the fileSet.

The proposal from #123 would still work, and would look like this:

ip_sources(spiritconsortium.org::Leon2RTL::gpio::1.2 VHDL
   FILE_SET fs-vhdlSource
        hdlsrc/gpio.vhd
)

xcelium(spiritconsortium.org::Leon2RTL::gpio::1.2
    FILE_SETS fs-vhdlSource
)

P.S. one additional issue to consider is that IPXact can define a VHDL library per file, while in SoCMake a VHDL library is unique per IP block.
So potentially that would be a limitation, unless we find a way around it

[benoitdenkinger]

So IPXact can have arbitrary filesets

Ok, nice. To me this is an additional strong point in favor of moving toward fileset support in SoCMake. If we move forward with the latest proposal in #123, I think we have an elegant and flexible solution as in your example.

one additional issue to consider is that IPXact can define a VHDL library per file

This shouldn't be difficult to support, no? We can create different IPs for each vhdl library.

This is how Leon2 IPXact examples from Accellera are organized

Nice match indeed. What is the FUNCTION field, I don't see it in the IPXact files?

[Risto97]

Ok, nice. To me this is an additional strong point in favor of moving toward fileset support in SoCMake. If we move forward with the latest proposal in #123, I think we have an elegant and flexible solution as in your example.

I think we have enough information to start implementing this, I will open a pull request in the next days.

This shouldn't be difficult to support, no? We can create different IPs for each vhdl library.

I think its not, so trivial.
Its more that IPXact library, can contain multiple VHDL libraries, while in SoCMake an IP library can contain only 1 VHDL library.
But for now its fine to just say that this is a limitation, and we can think later

Nice match indeed. What is the FUNCTION field, I don't see it in the IPXact files?

The FUNCTION is in SoCMake the target that generates the output directory, for example for Xcelium:
vendor__library__name__0.0.1__xcelium

Well something like that doesnt really exist in IPXact.

[benoitdenkinger]

The FUNCTION is in SoCMake the target that generates the output directory, for example for Xcelium:
vendor__library__name__0.0.1__xcelium

Ok, I see.

So what would/could be the hierarchy exactly? Would it also include the language/fileset? Any chance to strip the FUNCTION name to have something cleaner (e.g., reglbock instead of vendor__library__name__0.0.1__peakrdl_regblock)?

./<install-dir>
├── vendor1
│   ├── lib1
│   │   ├── ip1
│   │   │   ├── 1.2
│   │   │   │   ├── reglbock/socgen/tmrg? (can we let a target define its installation folder?)
│   │   │   │   ├── vhdl/systemverilog/verilog? (maybe for files that are not generated?)
│   │   │   │   └── something else?
│   │   │   └── 1.3
│   │   │       └── ...
│   │   └── ip2
│   │       └── ...
│   ├── lib2
│   │   └── ...
│   └── ...
├── vendor2
│   └── ...
└── ...

Currently, the implementation is exporting the files in a way it can be imported by cmake/socmake. Do you think we could/should export a list of the files (and other important parameters, e.g., include directories) so it can also be imported without SoCMake? Or should we have dedicated scripts to generate such files to import the design in other tools?

[Risto97]

So what would/could be the hierarchy exactly? Would it also include the language/fileset? Any chance to strip the FUNCTION name to have something cleaner (e.g., reglbock instead of vendor__library__name__0.0.1__peakrdl_regblock)?

Ah, sorry, I jumped to another topic there.
I am talking about the BINARY_DIR, not the INSTALL_DIR.
Basically I am thinking of changing the current BINARY_DIR structure to use directories instead of underscores.
I am not sure if we gain anything there though.

---

Regarding the INSTALL_DIR structure.
I am not sure if we should distinguish between generated files and static files.
It wouldn't be easy to figure out what function generates which file from the installation function.
Unless we do as you are suggesting that the generating function defines also the installation directory.

But do we really need this distinction? If not, it becomes simpler.
So the following would be enough

./<install-dir>
├── vendor1
│   ├── lib1
│   │   ├── ip1
│   │   │   ├── 1.2
│   │   │   │   ├── vhdl/systemverilog/verilog? (generated and static files go here)
│   │   │   └── 1.3
│   │   │       └── ...
│   │   └── ip2
│   │       └── ...
│   ├── lib2
│   │   └── ...
│   └── ...
├── vendor2
│   └── ...
└── ...

Currently, the implementation is exporting the files in a way it can be imported by cmake/socmake. Do you think we could/should export a list of the files (and other important parameters, e.g., include directories) so it can also be imported without SoCMake? Or should we have dedicated scripts to generate such files to import the design in other tools?

Well, sure, it can be done, but the question is what would that format be?
There is no universal standard for this kind of "manifest" files, and every build system uses its own homebrew standard.
The closest to a standard would be IPXact, but still most of the build systems would not be able to accept IPXact as input.

We can play around with CMake script mode, to provide these "build system manifest conversion" functions.
This would have the advantage that you could convert any CMake IP block into the target build system manifest without having to configure the project.

So you would run something like this:

socmake_manifest_gen gpio/CMakeLists.txt json > gpio.json
socmake_manifest_gen gpio/CMakeLists.txt ipxact > gpio.xml
...

---

Few things to keep in mind while installing the IP blocks

1. Installed IP blocks should be possible to retrieve with find_package() call in the parent project.

find_package(gpio CONFIG REQUIRED)

add_ip(soc)
ip_link(soc gpio)

I implemented the basic version of it, but maybe we can use something more sophisticated like https://github.com/TheLartians/PackageProject.cmake

2. Should make install also install any of the linked IP blocks?

Usually, the generated Config.cmake would contain the find_dependency() call to make sure the dependency of the installed package is also available.
But should this dependency be automatically installed if the parent IP is installed?

3. Time of building and installing dependency.

a) Dependency can be built and installed during the parent project configure time, that way the package would be available as installed package to the parent project. No targets need to be executed
b) Depencency can be built and installed during the parent project build time. This way the dependency is integrated into the build graph of the main project, and the targets of the dependency would have to be run together with main project.

[benoitdenkinger]

From your last comment and from the offline discussion:

Should make install also install any of the linked IP blocks?

Yes, the installation will install everything. By default, each IP will be installed independently and can be retrieved also independently. As discussed, we could add a FLATTEN option to install (install(${IP} FLATTEN)) to install all the IPs in a single IP.

What could be interesting is to also have a FILE_SETS option. By default, all the FILE_SETS are installed, but we could also specify the ones to install. Or an option to exclude a certain IP in the hierarchy? Maybe you want to export your block, but not a certain sub-module. You can also control this when you import again the files, but maybe you don't even want to expose these files to the user importing them. What do you thing @Risto97?