From c01bba226474545ced8c6c503e750c1ee023cfa9 Mon Sep 17 00:00:00 2001 From: JernKunpittaya <61564542+JernKunpittaya@users.noreply.github.com> Date: Thu, 30 Jan 2025 13:00:18 +0700 Subject: [PATCH 1/4] clean docs --- DevConDemo/README.md | 4 + ExternalDemo/README.md | 72 +-------- ExternalStats/README.md | 84 ++++++---- README.md | 331 ++++++++++++++++++++++------------------ 4 files changed, 239 insertions(+), 252 deletions(-) diff --git a/DevConDemo/README.md b/DevConDemo/README.md index aea054ed3..b9950e600 100644 --- a/DevConDemo/README.md +++ b/DevConDemo/README.md @@ -1,3 +1,7 @@ +## Comprehensive skeleton for DevCon Demo + +This Folder is used for final draft of DevCon demo flow before migrating to the actual DevCon demo [repo](https://github.com/ZKStats/mpc-demo-infra) + Note that we're working with 256-bit integer, resulting in 320-bit prime number, which is greater than 256 bit. Hence need to write MOD = -DGFP_MOD_SZ=5 into CONFIG.mine **Test Flow** diff --git a/ExternalDemo/README.md b/ExternalDemo/README.md index c9cadfe79..00b3a62b2 100644 --- a/ExternalDemo/README.md +++ b/ExternalDemo/README.md @@ -1,4 +1,6 @@ -This Folder is Used for validating ideas/necessary properties for our application for DevCon. +## First draft of DevCon Demo + +This Folder is used for validating ideas/necessary properties for our application for DevCon, mainly on save & load private shares. The computation function in this demo is the same as bankers_bonus aka determine the highest bonus value @@ -83,71 +85,3 @@ Array, Matrix, multiArray, sfix, sint, Compiler.ml.FixAveragePool2d, Compiler.ml ``` 3.2 Now the clients can interact as 2.1 and 2.2 (Yet no way to save and crash this running, due to the reason mentioned above in Limitation/constraints section) - -**Legacy from Originial ReadMe** - -The ExternalIO directory contains an example of managing I/O between -external client processes and parties running MP-SPDZ engines. These -instructions assume that MP-SPDZ has been built as per the [project -readme](../README.md). - -## Working Examples - -[bankers-bonus-client.cpp](../ExternalIO/bankers-bonus-client.cpp) and -[bankers-bonus-client.py](../ExternalIO/bankers-bonus-client.py) act as a -client to [bankers_bonus.mpc](../Programs/Source/bankers_bonus.mpc) -and demonstrates sending input and receiving output as described by -[Damgård et al.](https://eprint.iacr.org/2015/1006) The computation -allows up to eight clients to input a number and computes the client -with the largest input. You can run the C++ code as follows from the main -directory: - -``` -make bankers-bonus-client.x -./compile.py bankers_bonus 1 -Scripts/setup-ssl.sh -Scripts/setup-clients.sh 3 -PLAYERS= Scripts/.sh bankers_bonus-1 & -./bankers-bonus-client.x 0 100 0 & -./bankers-bonus-client.x 1 200 0 & -./bankers-bonus-client.x 2 50 1 -``` - -`` can be any arithmetic protocol (e.g., `mascot`) but not a -binary protocol (e.g., `yao`). -This should output that the winning id is 1. Note that the ids have to -be incremental, and the client with the highest id has to input 1 as -the last argument while the others have to input 0 there. Furthermore, -`` refers to the number of parties running the computation -not the number of clients, and `` can be the name of -protocol script. The setup scripts generate the necessary SSL -certificates and keys. Therefore, if you run the computation on -different hosts, you will have to distribute the `*.pem` files. - -For the Python client, make sure to install -[gmpy2](https://pypi.org/project/gmpy2), and run -`ExternalIO/bankers-bonus-client.py` instead of -`bankers-bonus-client.x`. - -## I/O MPC Instructions - -### Connection Setup - -1. [Listen for clients](https://mp-spdz.readthedocs.io/en/latest/Compiler.html#Compiler.library.listen_for_clients) -2. [Accept client connections](https://mp-spdz.readthedocs.io/en/latest/Compiler.html#Compiler.library.accept_client_connection) -3. [Close client connections](https://mp-spdz.readthedocs.io/en/latest/instructions.html#Compiler.instructions.closeclientconnection) - -### Data Exchange - -Only the `sint` methods used in the example are documented here, equivalent methods are available for other data types. See [the reference](https://mp-spdz.readthedocs.io/en/latest/Compiler.html#module-Compiler.types). - -1. [Public value from client](https://mp-spdz.readthedocs.io/en/latest/Compiler.html#Compiler.types.regint.read_from_socket) -2. [Secret value from client](https://mp-spdz.readthedocs.io/en/latest/Compiler.html#Compiler.types.sint.receive_from_client) -3. [Reveal secret value to clients](https://mp-spdz.readthedocs.io/en/latest/Compiler.html#Compiler.types.sint.reveal_to_clients) - -## Client-Side Interface - -The example uses the `Client` class implemented in -`ExternalIO/Client.hpp` to handle the communication, see -[this reference](https://mp-spdz.readthedocs.io/en/latest/io.html#reference) for -documentation. diff --git a/ExternalStats/README.md b/ExternalStats/README.md index a8901475d..6dc766b41 100644 --- a/ExternalStats/README.md +++ b/ExternalStats/README.md @@ -1,10 +1,41 @@ -**Statistics operations with client Example** -This folder is for doing client interface with statistics function where client does nothing but getting the output. +## Benchmarking stats operations with client interface -Here, we consider 2 servers with private data doing MPC together, only to output to the client. -We input private data of these 2 servers in Player-Data/Input-P0-0, and Player-Data/Input-P1-0. For example, you can just put 0 1 2 3 170 160 152 180 in Player-Data/Input-P0-0, and 3 0 4 5 50 60 70 100 in Player-Data/Input-P1-0 +This folder is for comparing & contrast the calculation of statistics function with and without client interface where client does nothing but getting the output. See more about client-interface[here](https://mp-spdz.readthedocs.io/en/latest/client-interface.html#client-interface) -Run the following commands for client version, here for example of 2 computing servers and one client +### How to run + +**Setup** + +We Input private data to computing servers (server parties that perform MPC calculation) in Player-Data. For example, if we consider using 2 servers, we can put 0 1 2 3 170 160 152 180 in Player-Data/Input-P0-0, and 3 0 4 5 50 60 70 100 in Player-Data/Input-P1-0 + +**Case 1 : 2 Computing Servers (without any client)** + +Run the following commands (as shown in main README of the repo ) + +``` +./compile.py stats-noclient +Scripts/../semi-party.x 0 stats-noclient -pn 17090 -h localhost -N 2 -OF . +Scripts/../semi-party.x 1 stats-noclient -pn 17090 -h localhost -N 2 -OF . +``` + +Note that here, we only delivers output to the 2nd client by modifying stats-noclient.mpc a bit as seen in comment, to make its behavior close to client interface + +**Case 2 : 3 Computing Servers (without any client)** + +Run the following commands (as shown in main README of the repo ) + +``` +./compile.py stats-noclient +Scripts/../semi-party.x 0 stats-noclient -pn 17090 -h localhost -N 3 -OF . +Scripts/../semi-party.x 1 stats-noclient -pn 17090 -h localhost -N 3 -OF . +Scripts/../semi-party.x 2 stats-noclient -pn 17090 -h localhost -N 3 -OF . +``` + +Note that here, we only delivers output to the 2nd client by modifying stats-noclient.mpc a bit as seen in comment, to make its behavior close to client interface + +**Case 3: 2 Computing Servers + 1 Client** + +Run the following commands ``` make stats-client.x @@ -19,44 +50,35 @@ Commands above do the following - make stats-client.x : Run Makefile to handle stats-client.cpp in folder ExternalStats - ./compile.py stats-client: Compile stats-client.mpc file in Programs/Source -- Scripts/setup-ssl.sh 2: Create triple shares for each party (spdz engine). 2 means 2 computing servers -- Scripts/setup-clients.sh 1: Create SSL keys and certificates for clients. 1 means 1 client +- Scripts/setup-ssl.sh 2: Create triple shares for each party (spdz engine). 2 indicates 2 computing servers +- Scripts/setup-clients.sh 1: Create SSL keys and certificates for clients. 1 indicates 1 client - PLAYERS=2 Scripts/semi.sh stats-client: Run server engines -- ./stats-client.x 0 2 1: Run client. 0 means client index 0, 2 means number of party, 1 means finish (dont need to wait for more clients) - -Run the following commands for non-client version (as shown in main README of the repo) Here in case of 2 computating servers - -``` -./compile.py stats-noclient -Scripts/../semi-party.x 0 stats-noclient -pn 17090 -h localhost -N 2 -OF . -Scripts/../semi-party.x 1 stats-noclient -pn 17090 -h localhost -N 2 -OF . -``` - -**BenchMark** -We compare this 2 servers + 1 client implementation with naive 2 servers in stats-noclient.mpc in Programs/Source and get the following results +- ./stats-client.x 0 2 1: Run client. 0 indicates client index 0, 2 indicates the number of party, 1 indicates finish (dont need to wait for more clients) -With Client +### BenchMark -- Data sent = 3.28978 MB in ~158 rounds (party 0 only; use '-v' for more details) - Global data sent = 6.59185 MB (all parties) - -Without Client. +**Case 1 : 2 Computing Servers (without any client)** - Data sent = 3.28973 MB in ~154 rounds (party 0 only; use '-v' for more details) Global data sent = 6.59176 MB (all parties) -Hence, the overhead is very small. Again the trust assumption of client-interface only depends on the computing servers - -Now we compare this 2 servers + 1 client implementation with naive 3 servers in stats-noclient.mpc (that only delivers output to the 2nd client, we just modify stats-noclient.mpc a bit as seen in comment) in Programs/Source and get the following results +**Case 2 : 3 Computing Servers (without any client)** - Data sent = 25.6026 MB in ~305 rounds (party 2 only; use '-v' for more details) Global data sent = 76.9298 MB (all parties) -Hence, by introducing another computating server for the sake of just getting output costs so so much, but we get more secure trusted assumption, since we can always trust ourselves. +**Case 3 : 2 Computing Servers + 1 Client** + +- Data sent = 3.28978 MB in ~158 rounds (party 0 only; use '-v' for more details) + Global data sent = 6.59185 MB (all parties) + +Comparing case 1 with case 3, we can see that the overhead is very small, which makes sense since the trust assumption of client-interface only depends on the computing servers + +Now considering case 2, we can see that introducing another computating server for the sake of just getting output costs so so much. This trade-off comes from the fact that we get more secure trusted assumption, since we can always trust ourselves. -**Issues** +### Some weird behavior -- However, when we run without client version with main.py, the amount of data skyrocket to - Data sent = 4.77346 MB in ~214 rounds (party 0 only; use '-v' for more details) +- When we run without client version with main.py, the amount of data skyrocket to - Data sent = 4.77346 MB in ~214 rounds (party 0 only; use '-v' for more details) Global data sent = 9.55921 MB (all parties) This is likely due to @@ -68,7 +90,7 @@ Hence, by introducing another computating server for the sake of just getting ou inside main.py -**Observation** +## Notes - We use sfix instead of sint in our program because stats operation likely results in floating number, hence the result the client receives is already multiplied by the scale. For example, if the result is 55., the client will receive 55\*2^16 ~ 3604480 - Anyway, by default, sfix is also faster than sint: https://github.com/data61/MP-SPDZ/issues/1400#issuecomment-2107550939 diff --git a/README.md b/README.md index 4aba415a5..19b61ea1f 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,28 @@ +# MPCStats (Powered by Multi-Protocol SPDZ) + +This repo is our fork of MP-SPDZ main repo (original README is below) to build MPCStats library as a framework for people to calculate statistics function with ease. + +## MPCStats lib + +All contents are in [mpcstat Folder](mpcstats/) + +## Miscellaneous + +This section consists of folder experimenting with client interface and some early flow for DevCon demo that some might prove useful. (ranked by latest to earliest) + +- [DevConDemo Folder](DevConDemo/): Comprehensive skeleton for DevCon Demo. This Folder is used for final draft of DevCon demo flow before migrating to the actual DevCon demo [repo](https://github.com/ZKStats/mpc-demo-infra) + +- [ExternalDemo Folder](ExternalDemo/): First draft of DevCon Demo. This Folder is used for validating ideas/necessary properties for our application for DevCon, mainly on save & load private shares. + +- [ExternalStats](ExternalStats/): Benchmarking stats operations with client interface. + This folder is for comparing & contrast the calculation of statistics function with and without client interface where client does nothing but getting the output. See more about client-interface [here](https://mp-spdz.readthedocs.io/en/latest/client-interface.html#client-interface) + +===================================================== + +#### After this line is the original README from MP-SPDZ master branch + +===================================================== + # Multi-Protocol SPDZ [![Documentation Status](https://readthedocs.org/projects/mp-spdz/badge/?version=latest)](https://mp-spdz.readthedocs.io/en/latest/?badge=latest) [![Build Status](https://dev.azure.com/data61/MP-SPDZ/_apis/build/status/data61.MP-SPDZ?branchName=master)](https://dev.azure.com/data61/MP-SPDZ/_build/latest?definitionId=7&branchName=master) [![Gitter](https://badges.gitter.im/MP-SPDZ/community.svg)](https://gitter.im/MP-SPDZ/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge) This is a software to benchmark various secure multi-party computation @@ -55,9 +80,11 @@ parties and malicious security. On Linux, this requires a working toolchain and [all requirements](#requirements). On Ubuntu, the following might suffice: + ``` sudo apt-get install automake build-essential clang cmake git libboost-dev libboost-iostreams-dev libboost-thread-dev libgmp-dev libntl-dev libsodium-dev libssl-dev libtool python3 ``` + On MacOS, this requires [brew](https://brew.sh) to be installed, which will be used for all dependencies. It will execute [the @@ -75,6 +102,7 @@ On strong enough hardware setups (several cores and GB of RAM), you can speed up the last step by running `make -j8 mascot-party.x` beforehand. #### TL;DR (Docker) + Build a docker image for `mascot-party.x`: ``` @@ -101,19 +129,19 @@ with critical production code. The following table lists all protocols that are fully supported. -| Security model | Mod prime / GF(2^n) | Mod 2^k | Bin. SS | Garbling | -| --- | --- | --- | --- | --- | -| Malicious, dishonest majority | [MASCOT / LowGear / HighGear](#secret-sharing) | [SPDZ2k](#secret-sharing) | [Tiny / Tinier](#secret-sharing) | [BMR](#bmr) | -| Covert, dishonest majority | [CowGear / ChaiGear](#secret-sharing) | N/A | N/A | N/A | -| Semi-honest, dishonest majority | [Semi / Hemi / Temi / Soho](#secret-sharing) | [Semi2k](#secret-sharing) | [SemiBin](#secret-sharing) | [Yao's GC](#yaos-garbled-circuits) / [BMR](#bmr) | -| Malicious, honest majority | [Shamir / Rep3 / PS / SY](#honest-majority) | [Brain / Rep3 / PS / SY](#honest-majority) | [Rep3 / CCD / PS](#honest-majority) | [BMR](#bmr) | -| Semi-honest, honest majority | [Shamir / ATLAS / Rep3](#honest-majority) | [Rep3](#honest-majority) | [Rep3 / CCD](#honest-majority) | [BMR](#bmr) | -| Malicious, honest supermajority | [Rep4](#honest-majority) | [Rep4](#honest-majority) | [Rep4](#honest-majority) | N/A | -| Semi-honest, dealer | [Dealer](#dealer-model) | [Dealer](#dealer-model) | [Dealer](#dealer-model) | N/A | +| Security model | Mod prime / GF(2^n) | Mod 2^k | Bin. SS | Garbling | +| ------------------------------- | ---------------------------------------------- | ------------------------------------------ | ----------------------------------- | ------------------------------------------------ | +| Malicious, dishonest majority | [MASCOT / LowGear / HighGear](#secret-sharing) | [SPDZ2k](#secret-sharing) | [Tiny / Tinier](#secret-sharing) | [BMR](#bmr) | +| Covert, dishonest majority | [CowGear / ChaiGear](#secret-sharing) | N/A | N/A | N/A | +| Semi-honest, dishonest majority | [Semi / Hemi / Temi / Soho](#secret-sharing) | [Semi2k](#secret-sharing) | [SemiBin](#secret-sharing) | [Yao's GC](#yaos-garbled-circuits) / [BMR](#bmr) | +| Malicious, honest majority | [Shamir / Rep3 / PS / SY](#honest-majority) | [Brain / Rep3 / PS / SY](#honest-majority) | [Rep3 / CCD / PS](#honest-majority) | [BMR](#bmr) | +| Semi-honest, honest majority | [Shamir / ATLAS / Rep3](#honest-majority) | [Rep3](#honest-majority) | [Rep3 / CCD](#honest-majority) | [BMR](#bmr) | +| Malicious, honest supermajority | [Rep4](#honest-majority) | [Rep4](#honest-majority) | [Rep4](#honest-majority) | N/A | +| Semi-honest, dealer | [Dealer](#dealer-model) | [Dealer](#dealer-model) | [Dealer](#dealer-model) | N/A | Modulo prime and modulo 2^k are the two settings that allow integer-like computation. For k = 64, the latter corresponds to the -computation available on the widely used 64-bit processors. GF(2^n) +computation available on the widely used 64-bit processors. GF(2^n) denotes Galois extension fields of order 2^n, which are different to computation modulo 2^n. In particular, every element has an inverse, which is not the case modulo 2^n. See [this @@ -147,7 +175,7 @@ there are a few things to consider: addition and multiplication at low cost. However, binary circuits might be a better option if there is very little integer computation. [See below](#finding-the-most-efficient-variant) to - find the most efficient mixed-circuit variant. Furthermore, local + find the most efficient mixed-circuit variant. Furthermore, local computation modulo a power of two is cheaper, but MP-SPDZ does not offer this domain with homomorphic encryption. @@ -186,19 +214,19 @@ there are a few things to consider: - Minor variants: Some command-line options change aspects of the protocols such as: - - `--bucket-size`: In some malicious binary computation and - malicious edaBit generation, a smaller bucket size allows - preprocessing in smaller batches at a higher asymptotic cost. - - `--batch-size`: Preprocessing in smaller batches avoids generating - too much but larger batches save communication rounds. - - `--direct`: In dishonest-majority protocols, direct communication - instead of star-shaped saves communication rounds at the expense - of a quadratic amount. This might be beneficial with a small - number of parties. - - `--bits-from-squares`: In some protocols computing modulo a prime - (Shamir, Rep3, SPDZ-wise), this switches from generating random - bits via XOR of parties' inputs to generation using the root of a - random square. + - `--bucket-size`: In some malicious binary computation and + malicious edaBit generation, a smaller bucket size allows + preprocessing in smaller batches at a higher asymptotic cost. + - `--batch-size`: Preprocessing in smaller batches avoids generating + too much but larger batches save communication rounds. + - `--direct`: In dishonest-majority protocols, direct communication + instead of star-shaped saves communication rounds at the expense + of a quadratic amount. This might be beneficial with a small + number of parties. + - `--bits-from-squares`: In some protocols computing modulo a prime + (Shamir, Rep3, SPDZ-wise), this switches from generating random + bits via XOR of parties' inputs to generation using the root of a + random square. #### Paper and Citation @@ -227,9 +255,10 @@ protocol](https://eprint.iacr.org/2011/535). This repository combines the functionality previously published in the following repositories: - - https://github.com/bristolcrypto/SPDZ-2 - - https://github.com/mkskeller/SPDZ-BMR-ORAM - - https://github.com/mkskeller/SPDZ-Yao + +- https://github.com/bristolcrypto/SPDZ-2 +- https://github.com/mkskeller/SPDZ-BMR-ORAM +- https://github.com/mkskeller/SPDZ-Yao #### Overview @@ -256,64 +285,64 @@ compute the preprocessing time for a particular computation. #### Requirements - - GCC 5 or later (tested with up to 11) or LLVM/clang 6 or later - (tested with up to 14). The default is to use clang because it performs - better. Note that GCC 5/6 and clang 9 don't support libOTe, so you - need to deactivate its use for these compilers (see the next - section). - - For protocols using oblivious transfer, libOTe with [the necessary - patches](https://github.com/mkskeller/softspoken-implementation) - but without SimplestOT. The easiest way is to run `make libote`, - which will install it as needed in a subdirectory. libOTe requires - CMake of version at least 3.15, which is not available by default - on older systems such as Ubuntu 18.04. You can run `make cmake` to - install it locally. - libOTe also requires boost of version at least 1.75, which is not - available by default on relatively recent systems such as Ubuntu - 22.04. You can install it locally by running `make boost`. - - GMP library, compiled with C++ support (use flag `--enable-cxx` - when running configure). Tested against 6.2.1 as supplied by - Ubuntu. - - libsodium library, tested against 1.0.18 - - OpenSSL, tested against 3.0.2 - - Boost.Asio with SSL support (`libboost-dev` on Ubuntu), tested against 1.81 - - Boost.Thread for BMR (`libboost-thread-dev` on Ubuntu), tested against 1.81 - - x86 or ARM 64-bit CPU (the latter tested with AWS Gravitron and - Apple Silicon) - - Python 3.5 or later - - NTL library for homomorphic encryption (optional; tested with NTL 11.5.1) - - If using macOS, Sierra or later - - Windows/VirtualBox: see [this - issue](https://github.com/data61/MP-SPDZ/issues/557) for a discussion +- GCC 5 or later (tested with up to 11) or LLVM/clang 6 or later + (tested with up to 14). The default is to use clang because it performs + better. Note that GCC 5/6 and clang 9 don't support libOTe, so you + need to deactivate its use for these compilers (see the next + section). +- For protocols using oblivious transfer, libOTe with [the necessary + patches](https://github.com/mkskeller/softspoken-implementation) + but without SimplestOT. The easiest way is to run `make libote`, + which will install it as needed in a subdirectory. libOTe requires + CMake of version at least 3.15, which is not available by default + on older systems such as Ubuntu 18.04. You can run `make cmake` to + install it locally. + libOTe also requires boost of version at least 1.75, which is not + available by default on relatively recent systems such as Ubuntu + 22.04. You can install it locally by running `make boost`. +- GMP library, compiled with C++ support (use flag `--enable-cxx` + when running configure). Tested against 6.2.1 as supplied by + Ubuntu. +- libsodium library, tested against 1.0.18 +- OpenSSL, tested against 3.0.2 +- Boost.Asio with SSL support (`libboost-dev` on Ubuntu), tested against 1.81 +- Boost.Thread for BMR (`libboost-thread-dev` on Ubuntu), tested against 1.81 +- x86 or ARM 64-bit CPU (the latter tested with AWS Gravitron and + Apple Silicon) +- Python 3.5 or later +- NTL library for homomorphic encryption (optional; tested with NTL 11.5.1) +- If using macOS, Sierra or later +- Windows/VirtualBox: see [this + issue](https://github.com/data61/MP-SPDZ/issues/557) for a discussion #### Compilation 1. Edit `CONFIG` or `CONFIG.mine` to your needs: - - On x86, the binaries are optimized for the CPU you are compiling - on. For all optimizations on x86, a CPU supporting AES-NI, - PCLMUL, AVX2, BMI2, ADX is required. This includes mainstream - processors released 2014 or later. If you intend to run on a - different CPU than compiling, you might need to change the `ARCH` - variable in `CONFIG` or `CONFIG.mine` to `-march=`. See the - [GCC - documentation](https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html) - for the possible options. - To run on CPUs without AVX2 (CPUs from before 2014), you should - also add `AVX_OT = 0` to `CONFIG.mine`. - - For optimal results on Linux on ARM, add `ARCH = -march=armv8.2-a+crypto` - to `CONFIG.mine`. This enables the hardware support for AES. See the [GCC - documentation](https://gcc.gnu.org/onlinedocs/gcc/AArch64-Options.html#AArch64-Options) on available options. - - To benchmark online-only protocols or Overdrive offline phases, add the following line at the top: `MY_CFLAGS = -DINSECURE` - - `PREP_DIR` should point to a local, unversioned directory to store preprocessing data (the default is `Player-Data` in the current directory). - - `SSL_DIR` should point to a local, unversioned directory to store ssl keys (the default is `Player-Data` in the current directory). - - For homomorphic encryption with GF(2^40), set `USE_NTL = 1`. - - To use KOS instead of SoftSpokenOT, add `USE_KOS = 1` and - `SECURE = -DINSECURE` to `CONFIG.mine`. This is necessary with - GCC 5 and 6 because these compilers don't support the C++ - standard used by libOTe. - - On macOS, there have been issues with non-system compilers. Add - `CXX = /usr/bin/g++` to fix them. + - On x86, the binaries are optimized for the CPU you are compiling + on. For all optimizations on x86, a CPU supporting AES-NI, + PCLMUL, AVX2, BMI2, ADX is required. This includes mainstream + processors released 2014 or later. If you intend to run on a + different CPU than compiling, you might need to change the `ARCH` + variable in `CONFIG` or `CONFIG.mine` to `-march=`. See the + [GCC + documentation](https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html) + for the possible options. + To run on CPUs without AVX2 (CPUs from before 2014), you should + also add `AVX_OT = 0` to `CONFIG.mine`. + - For optimal results on Linux on ARM, add `ARCH = -march=armv8.2-a+crypto` + to `CONFIG.mine`. This enables the hardware support for AES. See the [GCC + documentation](https://gcc.gnu.org/onlinedocs/gcc/AArch64-Options.html#AArch64-Options) on available options. + - To benchmark online-only protocols or Overdrive offline phases, add the following line at the top: `MY_CFLAGS = -DINSECURE` + - `PREP_DIR` should point to a local, unversioned directory to store preprocessing data (the default is `Player-Data` in the current directory). + - `SSL_DIR` should point to a local, unversioned directory to store ssl keys (the default is `Player-Data` in the current directory). + - For homomorphic encryption with GF(2^40), set `USE_NTL = 1`. + - To use KOS instead of SoftSpokenOT, add `USE_KOS = 1` and + `SECURE = -DINSECURE` to `CONFIG.mine`. This is necessary with + GCC 5 and 6 because these compilers don't support the C++ + standard used by libOTe. + - On macOS, there have been issues with non-system compilers. Add + `CXX = /usr/bin/g++` to fix them. 2. Run `make` to compile all the software (use the flag `-j` for faster compilation using multiple threads). See below on how to compile specific @@ -383,7 +412,6 @@ documentation](https://mp-spdz.readthedocs.io/en/latest/compilation.html#compila for what this means when using Python data structures and Python language features. - ### Compiling high-level programs There are three computation domains, and the high-level programs have @@ -391,7 +419,7 @@ to be compiled accordingly. #### Arithmetic modulo a prime -```./compile.py [-F ] [-P ] ``` +`./compile.py [-F ] [-P ] ` The integer bit length defaults to 64, and the prime defaults to none given. If a prime is given, it has to be at least two bits longer than @@ -422,7 +450,7 @@ directly. For fixed-point computation this is done via #### Arithmetic modulo 2^k -```./compile.py -R ``` +`./compile.py -R ` The length is communicated to the virtual machines and automatically used if supported. By default, they support bit lengths 64, 72, and @@ -431,7 +459,7 @@ used if supported. By default, they support bit lengths 64, 72, and #### Binary circuits -```./compile.py -B ``` +`./compile.py -B ` The integer length can be any number up to a maximum depending on the protocol. All protocols support at least 64-bit integers. @@ -457,9 +485,9 @@ several ways of achieving this as described below. You can activate this by adding `-X` when compiling arithmetic circuits, that is -```./compile.py -X [-F ] ``` +`./compile.py -X [-F ] ` for computation modulo a prime and -```./compile.py -X -R ``` +`./compile.py -X -R ` for computation modulo 2^k. Internally, this uses daBits described by [Rotaru and @@ -619,7 +647,7 @@ contains further documentation on available layers. For arithmetic circuits modulo a power of two and binary circuits, you can emulate the computation as follows: -``` ./emulate.x ``` +`./emulate.x ` This runs the compiled bytecode in cleartext computation. @@ -635,23 +663,23 @@ available. The following table shows all programs for dishonest-majority computation using secret sharing: -| Program | Protocol | Domain | Security | Script | -| --- | --- | --- | --- | --- | -| `mascot-party.x` | [MASCOT](https://eprint.iacr.org/2016/505) | Mod prime | Malicious | `mascot.sh` | -| `mama-party.x` | MASCOT* | Mod prime | Malicious | `mama.sh` | -| `spdz2k-party.x` | [SPDZ2k](https://eprint.iacr.org/2018/482) | Mod 2^k | Malicious | `spdz2k.sh` | -| `semi-party.x` | OT-based | Mod prime | Semi-honest | `semi.sh` | -| `semi2k-party.x` | OT-based | Mod 2^k | Semi-honest | `semi2k.sh` | -| `lowgear-party.x` | [LowGear](https://eprint.iacr.org/2017/1230) | Mod prime | Malicious | `lowgear.sh` | -| `highgear-party.x` | [HighGear](https://eprint.iacr.org/2017/1230) | Mod prime | Malicious | `highgear.sh` | -| `cowgear-party.x` | Adapted [LowGear](https://eprint.iacr.org/2017/1230) | Mod prime | Covert | `cowgear.sh` | -| `chaigear-party.x` | Adapted [HighGear](https://eprint.iacr.org/2017/1230) | Mod prime | Covert | `chaigear.sh` | -| `hemi-party.x` | Semi-homomorphic encryption | Mod prime | Semi-honest | `hemi.sh` | -| `temi-party.x` | Adapted [CDN01](https://eprint.iacr.org/2000/055) | Mod prime | Semi-honest | `temi.sh` | -| `soho-party.x` | Somewhat homomorphic encryption | Mod prime | Semi-honest | `soho.sh` | -| `semi-bin-party.x` | OT-based | Binary | Semi-honest | `semi-bin.sh` | -| `tiny-party.x` | Adapted SPDZ2k | Binary | Malicious | `tiny.sh` | -| `tinier-party.x` | [FKOS15](https://eprint.iacr.org/2015/901) | Binary | Malicious | `tinier.sh` | +| Program | Protocol | Domain | Security | Script | +| ------------------ | ----------------------------------------------------- | --------- | ----------- | ------------- | +| `mascot-party.x` | [MASCOT](https://eprint.iacr.org/2016/505) | Mod prime | Malicious | `mascot.sh` | +| `mama-party.x` | MASCOT\* | Mod prime | Malicious | `mama.sh` | +| `spdz2k-party.x` | [SPDZ2k](https://eprint.iacr.org/2018/482) | Mod 2^k | Malicious | `spdz2k.sh` | +| `semi-party.x` | OT-based | Mod prime | Semi-honest | `semi.sh` | +| `semi2k-party.x` | OT-based | Mod 2^k | Semi-honest | `semi2k.sh` | +| `lowgear-party.x` | [LowGear](https://eprint.iacr.org/2017/1230) | Mod prime | Malicious | `lowgear.sh` | +| `highgear-party.x` | [HighGear](https://eprint.iacr.org/2017/1230) | Mod prime | Malicious | `highgear.sh` | +| `cowgear-party.x` | Adapted [LowGear](https://eprint.iacr.org/2017/1230) | Mod prime | Covert | `cowgear.sh` | +| `chaigear-party.x` | Adapted [HighGear](https://eprint.iacr.org/2017/1230) | Mod prime | Covert | `chaigear.sh` | +| `hemi-party.x` | Semi-homomorphic encryption | Mod prime | Semi-honest | `hemi.sh` | +| `temi-party.x` | Adapted [CDN01](https://eprint.iacr.org/2000/055) | Mod prime | Semi-honest | `temi.sh` | +| `soho-party.x` | Somewhat homomorphic encryption | Mod prime | Semi-honest | `soho.sh` | +| `semi-bin-party.x` | OT-based | Binary | Semi-honest | `semi-bin.sh` | +| `tiny-party.x` | Adapted SPDZ2k | Binary | Malicious | `tiny.sh` | +| `tinier-party.x` | [FKOS15](https://eprint.iacr.org/2015/901) | Binary | Malicious | `tinier.sh` | Mama denotes MASCOT with several MACs to increase the security parameter to a multiple of the prime length. @@ -727,7 +755,6 @@ e.g. if this machine is name `diffie` on the local network: The software uses TCP ports around 5000 by default, use the `-pn` argument to change that. - ### Yao's garbled circuits We use half-gate garbling as described by [Zahur et @@ -747,8 +774,8 @@ and the high-level program: Then run as follows: - - Garbler: ```./yao-party.x [-I] -p 0 ``` - - Evaluator: ```./yao-party.x [-I] -p 1 -h ``` +- Garbler: `./yao-party.x [-I] -p 0 ` +- Evaluator: `./yao-party.x [-I] -p 1 -h ` When running locally, you can omit the host argument. As above, `-I` activates interactive input, otherwise inputs are read from @@ -762,27 +789,27 @@ whenever received.You can activate garbling all at once by adding The following table shows all programs for honest-majority computation: -| Program | Sharing | Domain | Malicious | \# parties | Script | -| --- | --- | --- | --- | --- | --- | -| `replicated-ring-party.x` | Replicated | Mod 2^k | N | 3 | `ring.sh` | -| `brain-party.x` | Replicated | Mod 2^k | Y | 3 | `brain.sh` | -| `ps-rep-ring-party.x` | Replicated | Mod 2^k | Y | 3 | `ps-rep-ring.sh` | -| `malicious-rep-ring-party.x` | Replicated | Mod 2^k | Y | 3 | `mal-rep-ring.sh` | -| `sy-rep-ring-party.x` | SPDZ-wise replicated | Mod 2^k | Y | 3 | `sy-rep-ring.sh` | -| `rep4-ring-party.x` | Replicated | Mod 2^k | Y | 4 | `rep4-ring.sh` | -| `replicated-bin-party.x` | Replicated | Binary | N | 3 | `replicated.sh` | -| `malicious-rep-bin-party.x` | Replicated | Binary | Y | 3 | `mal-rep-bin.sh` | -| `ps-rep-bin-party.x` | Replicated | Binary | Y | 3 | `ps-rep-bin.sh` | -| `replicated-field-party.x` | Replicated | Mod prime | N | 3 | `rep-field.sh` | -| `ps-rep-field-party.x` | Replicated | Mod prime | Y | 3 | `ps-rep-field.sh` | -| `sy-rep-field-party.x` | SPDZ-wise replicated | Mod prime | Y | 3 | `sy-rep-field.sh` | -| `malicious-rep-field-party.x` | Replicated | Mod prime | Y | 3 | `mal-rep-field.sh` | -| `atlas-party.x` | [ATLAS](https://eprint.iacr.org/2021/833) | Mod prime | N | 3 or more | `atlas.sh` | -| `shamir-party.x` | Shamir | Mod prime | N | 3 or more | `shamir.sh` | -| `malicious-shamir-party.x` | Shamir | Mod prime | Y | 3 or more | `mal-shamir.sh` | -| `sy-shamir-party.x` | SPDZ-wise Shamir | Mod prime | Y | 3 or more | `sy-shamir.sh` | -| `ccd-party.x` | CCD/Shamir | Binary | N | 3 or more | `ccd.sh` | -| `malicious-cdd-party.x` | CCD/Shamir | Binary | Y | 3 or more | `mal-ccd.sh` | +| Program | Sharing | Domain | Malicious | \# parties | Script | +| ----------------------------- | ----------------------------------------- | --------- | --------- | ---------- | ------------------ | +| `replicated-ring-party.x` | Replicated | Mod 2^k | N | 3 | `ring.sh` | +| `brain-party.x` | Replicated | Mod 2^k | Y | 3 | `brain.sh` | +| `ps-rep-ring-party.x` | Replicated | Mod 2^k | Y | 3 | `ps-rep-ring.sh` | +| `malicious-rep-ring-party.x` | Replicated | Mod 2^k | Y | 3 | `mal-rep-ring.sh` | +| `sy-rep-ring-party.x` | SPDZ-wise replicated | Mod 2^k | Y | 3 | `sy-rep-ring.sh` | +| `rep4-ring-party.x` | Replicated | Mod 2^k | Y | 4 | `rep4-ring.sh` | +| `replicated-bin-party.x` | Replicated | Binary | N | 3 | `replicated.sh` | +| `malicious-rep-bin-party.x` | Replicated | Binary | Y | 3 | `mal-rep-bin.sh` | +| `ps-rep-bin-party.x` | Replicated | Binary | Y | 3 | `ps-rep-bin.sh` | +| `replicated-field-party.x` | Replicated | Mod prime | N | 3 | `rep-field.sh` | +| `ps-rep-field-party.x` | Replicated | Mod prime | Y | 3 | `ps-rep-field.sh` | +| `sy-rep-field-party.x` | SPDZ-wise replicated | Mod prime | Y | 3 | `sy-rep-field.sh` | +| `malicious-rep-field-party.x` | Replicated | Mod prime | Y | 3 | `mal-rep-field.sh` | +| `atlas-party.x` | [ATLAS](https://eprint.iacr.org/2021/833) | Mod prime | N | 3 or more | `atlas.sh` | +| `shamir-party.x` | Shamir | Mod prime | N | 3 or more | `shamir.sh` | +| `malicious-shamir-party.x` | Shamir | Mod prime | Y | 3 or more | `mal-shamir.sh` | +| `sy-shamir-party.x` | SPDZ-wise Shamir | Mod prime | Y | 3 or more | `sy-shamir.sh` | +| `ccd-party.x` | CCD/Shamir | Binary | N | 3 or more | `ccd.sh` | +| `malicious-cdd-party.x` | CCD/Shamir | Binary | Y | 3 or more | `mal-ccd.sh` | We use the "generate random triple optimistically/sacrifice/Beaver" methodology described by [Lindell and @@ -889,9 +916,9 @@ all but one of the other parties are allowed to collude. In our implementation, the dealer is the party with the highest number, so with three parties overall, Party 0 and 1 run the online phase. -| Program | Sharing | Domain | Malicious | \# parties | Script | -| --- | --- | --- | --- | --- | --- | -| `dealer-ring-party.x` | Additive | Mod 2^k | N | 3+ | `dealer-ring.sh` | +| Program | Sharing | Domain | Malicious | \# parties | Script | +| --------------------- | -------- | ------- | --------- | ---------- | ---------------- | +| `dealer-ring-party.x` | Additive | Mod 2^k | N | 3+ | `dealer-ring.sh` | ## BMR @@ -903,14 +930,14 @@ circuits. Our implementation is based on the [SPDZ-BMR-ORAM construction](https://eprint.iacr.org/2017/981). The following table lists the available schemes. -| Program | Protocol | Dishonest Maj. | Malicious | \# parties | Script | -| --- | --- | --- | --- | --- | --- | -| `real-bmr-party.x` | MASCOT | Y | Y | 2 or more | `real-bmr.sh` | -| `semi-bmr-party.x` | Semi | Y | N | 2 or more | `semi-bmr.sh` | -| `shamir-bmr-party.x` | Shamir | N | N | 3 or more | `shamir-bmr.sh` | -| `mal-shamir-bmr-party.x` | Shamir | N | Y | 3 or more | `mal-shamir-bmr.sh` | -| `rep-bmr-party.x` | Replicated | N | N | 3 | `rep-bmr.sh` | -| `mal-rep-bmr-party.x` | Replicated | N | Y | 3 | `mal-rep-bmr.sh` | +| Program | Protocol | Dishonest Maj. | Malicious | \# parties | Script | +| ------------------------ | ---------- | -------------- | --------- | ---------- | ------------------- | +| `real-bmr-party.x` | MASCOT | Y | Y | 2 or more | `real-bmr.sh` | +| `semi-bmr-party.x` | Semi | Y | N | 2 or more | `semi-bmr.sh` | +| `shamir-bmr-party.x` | Shamir | N | N | 3 or more | `shamir-bmr.sh` | +| `mal-shamir-bmr-party.x` | Shamir | N | Y | 3 or more | `mal-shamir-bmr.sh` | +| `rep-bmr-party.x` | Replicated | N | N | 3 | `rep-bmr.sh` | +| `mal-rep-bmr-party.x` | Replicated | N | Y | 3 | `mal-rep-bmr.sh` | In the following, we will walk through running the tutorial with BMR based on MASCOT and two parties. The other programs work similarly. @@ -945,6 +972,7 @@ In this section we show how to benchmark purely the data-dependent (often called online) phase of some protocols. This requires to generate the output of a previous phase. There are two options to do that: + 1. For select protocols, you can run [preprocessing as required](#preprocessing-as-required). 2. You can run insecure preprocessing. For this, you will have to @@ -985,7 +1013,6 @@ Parameters can be customised by running `Scripts/setup-online.sh []` - #### To compile a program To compile for example the program in `./Programs/Source/tutorial.mpc`, run: @@ -1074,12 +1101,12 @@ After compiling the mpc file: You can benchmark the ORAM implementation as follows: -1) Edit `Program/Source/gc_oram.mpc` to change size and to choose -Circuit ORAM or linear scan without ORAM. -2) Run `./compile.py -G -D gc_oram`. The `-D` argument instructs the -compiler to remove dead code. This is useful for more complex programs -such as this one. -3) Run `gc_oram` in the virtual machines as explained above. +1. Edit `Program/Source/gc_oram.mpc` to change size and to choose + Circuit ORAM or linear scan without ORAM. +2. Run `./compile.py -G -D gc_oram`. The `-D` argument instructs the + compiler to remove dead code. This is useful for more complex programs + such as this one. +3. Run `gc_oram` in the virtual machines as explained above. ## Preprocessing as required @@ -1138,7 +1165,7 @@ MASCOT can be run as follows: `host2:$ ./ot-offline.x -p 1 -c` -For SPDZ2k, use `-Z ` to set the computation domain to Z_{2^k}, and +For SPDZ2k, use `-Z ` to set the computation domain to Z\_{2^k}, and `-S` to set the security parameter. The latter defaults to k. At the time of writing, the following combinations are available: 32/32, 64/64, 64/48, and 66/48. @@ -1151,11 +1178,11 @@ loops. We have implemented several protocols to measure the maximal throughput for the [Overdrive paper](https://eprint.iacr.org/2017/1230). As for MASCOT, these implementations are not suited to generate data for the online phase because they only generate one type at a time. -Binary | Protocol ------- | -------- -`simple-offline.x` | SPDZ-1 and High Gear (with command-line argument `-g`) -`pairwise-offline.x` | Low Gear -`cnc-offline.x` | SPDZ-2 with malicious security (covert security with command-line argument `-c`) +| Binary | Protocol | +| -------------------- | -------------------------------------------------------------------------------- | +| `simple-offline.x` | SPDZ-1 and High Gear (with command-line argument `-g`) | +| `pairwise-offline.x` | Low Gear | +| `cnc-offline.x` | SPDZ-2 with malicious security (covert security with command-line argument `-c`) | These programs can be run similarly to `spdz2-offline.x`, for example: From 63c3d65f4a30f69e40a357228ad76e4a58230589 Mon Sep 17 00:00:00 2001 From: JernKunpittaya <61564542+JernKunpittaya@users.noreply.github.com> Date: Thu, 30 Jan 2025 13:07:09 +0700 Subject: [PATCH 2/4] make main example functional --- mpcstats/main.py | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mpcstats/main.py b/mpcstats/main.py index 9183bd88b..54c537d00 100644 --- a/mpcstats/main.py +++ b/mpcstats/main.py @@ -17,7 +17,7 @@ SCRIPTS_DIR = PROJECT_ROOT / "Scripts" # E.g. Scripts/semi.sh testmpc LOCAL_EXECUTION_EXE = SCRIPTS_DIR / f"{MPC_PROTOCOL}.sh" -PLAYER_DATA_DIR = PROJECT_ROOT / "Player-Data" +PLAYER_DATA_DIR = PROJECT_ROOT / "mpcstats/Player-Data" PLAYER_DATA_DIR.mkdir(parents=True, exist_ok=True) @@ -52,6 +52,7 @@ def prepare_data(): """ Save PLAYER_DATA to each party input file. """ + # print_ln("third") for party_index, player_data in enumerate(PLAYER_DATA): player_data_file = PLAYER_DATA_DIR / f"Input-P{party_index}-0" with open(player_data_file, "w") as f: @@ -62,8 +63,9 @@ def prepare_data(): if __name__ == "__main__": # Save PLAYER_DATA to each party input file + # print_ln("FIRST") prepare_data() - + # print_ln("Second") # Computation defined by the user def computation(): # Read all data from all parties @@ -74,7 +76,7 @@ def computation(): print_ln("data_party_0:") print_data(matrix_0) - # 3 0 1 2 + # 3 0 4 5 # 50 60 70 80 data_party_1 = PLAYER_DATA[1] matrix_1 = read_data(1, len(data_party_1), len(data_party_1[0])) @@ -94,8 +96,8 @@ def computation(): # [ # [0, 1, 2, 3], # [170, 160, 152, 180], - # [0, MAGIC_NUMBER, MAGIC_NUMBER, 3, MAGIC_NUMEBER], - # [60, MAGIC_NUMBER, MAGIC_NUMBER, 50, MAGIC_NUMBER], + # [0, MAGIC_NUMBER, MAGIC_NUMBER, 3], + # [60, MAGIC_NUMBER, MAGIC_NUMBER, 50], # ] new_data = join(matrix_0, matrix_1, 0, 0) print_ln("new_data:") From 0ed29d0dcf682ec1246c9fa90adc6ccf458f21ad Mon Sep 17 00:00:00 2001 From: JernKunpittaya <61564542+JernKunpittaya@users.noreply.github.com> Date: Thu, 30 Jan 2025 13:12:33 +0700 Subject: [PATCH 3/4] mpcstats doc --- mpcstats/README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/mpcstats/README.md b/mpcstats/README.md index 8e32a0b87..e719e96d1 100644 --- a/mpcstats/README.md +++ b/mpcstats/README.md @@ -1,6 +1,9 @@ # MPCStats Library +This library allows users to write simple python to calculate stats function using MPC without the need to interact with MP-SPDZ itself. You can see the example in main.py, and all the stats functions implemented in mpcstats_lib.py + ## Installation + Clone the repo. ```bash @@ -28,10 +31,13 @@ If you're on macOS and see the following linker warning, you can safely ignore i ld: warning: search path '/usr/local/opt/openssl/lib' not found ``` -## Run +## Run Example + ```bash +cd mpcstats python main.py ``` ## Implementation + Statistics operations implementation is in [mpcstats_lib.py](./mpcstats_lib.py). From 5d0bacc0449a1e037763484b2e1ea1cfbd84dc88 Mon Sep 17 00:00:00 2001 From: JernKunpittaya <61564542+JernKunpittaya@users.noreply.github.com> Date: Thu, 30 Jan 2025 13:16:10 +0700 Subject: [PATCH 4/4] del debug msg --- mpcstats/main.py | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/mpcstats/main.py b/mpcstats/main.py index 54c537d00..c7a8d76ed 100644 --- a/mpcstats/main.py +++ b/mpcstats/main.py @@ -52,7 +52,7 @@ def prepare_data(): """ Save PLAYER_DATA to each party input file. """ - # print_ln("third") + for party_index, player_data in enumerate(PLAYER_DATA): player_data_file = PLAYER_DATA_DIR / f"Input-P{party_index}-0" with open(player_data_file, "w") as f: @@ -63,9 +63,7 @@ def prepare_data(): if __name__ == "__main__": # Save PLAYER_DATA to each party input file - # print_ln("FIRST") prepare_data() - # print_ln("Second") # Computation defined by the user def computation(): # Read all data from all parties