NTT vectorization for RISC-V Vector (RVV) - CHES2025
Two methods are made available for running the tests: through a Docker container, and natively in Linux. The first option is easier, while the second allows for more customization of the test's source code.
A docker image (Dockerfile) is made available in this repository. If you wish to run the tests through the container, it is only necessary to download the Dockerfile itself, instead of the whole repository. To build the container and run it:
# in the directory you have the Dockerfile
# create the docker container
docker build -t ntt-rvv-ches2025:latest .
# runs the container, which by default will run the two tests
docker run ntt-rvv-ches2025:latestThis will clone the repository inside the container, compile everything, and run the two tests with the second command. The second test is expected to take multiple hours.
It is recommended that the project is cloned with the --recurse-submodules option, to fetch OpenFHE as well. Otherwise, it will be necessary to init and update the submodules.
After the OpenFHE submodule has been initialized, it must be patched to include the required changes. To do so, simply run the patch-openfhe.sh script from this repository's root directory.
The provided code makes use of the EPI Vector Intrinsics, and therefore requires an RVV 1.0 EPI LLVM compiler toolchain. The easiest way to obtain this is to run the download-toolchain.sh script from the repository's root, which will download a version of the EPI cross-compilation toolchain that is compatible with gem5.
To set the environment variables CC/CXX with the appropriate cross-compilers, simply run source set-compilers.sh.
Other compiler toolchains can be obtained at BSC's FTP server. For compiling natively (on a RISC-V machine), look for a llvm-EPI-*-toolchain-native toolchain.
To compile the vectorized NTT/INTT implementations, cd into ntt/, and run make. This will create ntt/bin/ntt-rvv.o, which will need to be linked with any binaries that make use of OpenFHE. It will also statically compile a simple speedup test binary, ntt/bin/test.
OpenFHE is compiled normally, after patching it and sourcing the set-compilers.sh script. You may want to specify an alternative install location, which does not require sudo. Building static libraries will also facilitate compiling binaries for gem5 simulation. This is done by configuring with cmake -DCMAKE_INSTALL_PREFIX=/your/path -DBUILD_STATIC=ON ... More information about OpenFHE installation can be found in the respective documentation.
example-ofhe-app contains an application compiled with OpenFHE and the custom RVV code, and shows how to link the ntt-rvv.o object. A static build is also recommended for gem5 compatibility.
You may use gem5 (version = 24.1.0.2) to simulate the basic test ntt/bin/test or the neural-net app generated by example-ofhe-app. As outlined in the paper, gem5 previously had an issue with false dependencies on dummy registers, a fix for which is made available in this fork of gem5, the vlstride-microop branch in particular. This fork is included in this repository as a submodule. This version should provide the same results as the paper, and is stable. Note that the neural-net was designed for the FPGA platform, and thus will take multiple hours to run in gem5.
After you've downloaded and built gem5, you can use the provided gem5 model: gem5.opt gem5-model/main.py <static_executable>.
# Clone repo and cd into it
git clone --recurse-submodules https://github.com/hpc-ulisboa/NTT-RVV-CHES2025
cd NTT-RVV-CHES2025
# Patch the OpenFHE submodule with the modifications
./patch-openfhe.sh
# Download RISC-V cross-compiler toolchain
./download-toolchain.sh
# Compile gem5
cd gem5
scons build/RISCV/gem5.opt -j {threads}
cd ..
# Set the environment variables for RISC-V cross-compilation
source set-compilers.sh
# Make a build directory in OpenFHE
cd openfhe-development/
mkdir build
cd build
# Configure cmake, set the install directory to <project_root>/install/, build static libraries
cmake -DCMAKE_INSTALL_PREFIX=../../install -DBUILD_STATIC=ON ..
# Compile with 8 cores, install
make -j 8
make install
# Build the RVV NTT code
cd ../ntt
make
# Build the example OpenFHE app
cd ../example-ofhe-app
mkdir build
cd build
# Built statically for gem5 simulation
cmake -DBUILD_STATIC=ON ..
make
# Simulate with gem5
cd ../../
gem5/build/RISCV/gem5.opt gem5-model/main.py ntt/bin/test
# This test will take a long time
gem5/build/RISCV/gem5.opt gem5-model/main.py example-ofhe-app/build/neural-netntt/bin/test
p: 1024
scalar: 78542, vector: 7268 cycles
scalar: 77616, vector: 6477 cycles
example-ofhe-app/build/neural-net
CCKKS scheme is using ring dimension 1024
bootstrap evaluated
keys generated
input generated
input encrypted
matrices generated
weights encrypted, entering measured code
Cycles: 7916310291