Skip to content

Run performance tests

Jump to bottom
IReallyLikeYourPants edited this page Mar 10, 2026 · 3 revisions

This Wiki page contains the steps to test CONFETTY performance locally through automatic scripts.

Please ensure that you have completed the steps in the Requirements, installation, and preliminary operations section before proceeding with the tests.

Overview

The CONFETTY Evaluation Tool runs a series of tests that analyze gas costs and execution times for various process configurations. The tests are described in this paper and they include:

  • X-ray tests for the number of writing participants, message size, and process size.
  • Synthesis tests including parallel split, parallel split & join, exclusive split, and exclusive split & join.
  • A test with three state-of-the-art processes.

Initialize the framework

If your CONFETTY container is not running, execute:

docker start confetty_container

Then enter the container:

docker exec -it confetty_container bash

At this stage, you should see a terminal prompt as root inside the container.

Now execute:

./test_init.sh

This script automatically performs the following operations:

  • Starts the local blockchain Ganache with predefined accounts.
  • Starts MongoDB to store server information.
  • Deploys the CONFETTY smart contracts on the blockchain.
  • Starts IPFS.
  • Builds and starts the frontend server Tomcat.

Python tests

To execute the Python scripts, open another console in the Docker container:

docker exec -it confetty_container bash

Then change directory to Evaluation Tool:

cd CONFETTY/Evaluation\ Tool

Here you will find the test_Auto.py script. This script will:

  • Generate the required JSON test inputs.
  • Start the confidentiality API.
  • Execute each test case 5 times.
  • Create two Excel files in the table_Output folder:
    • costs.xlsx for gas cost measurements.
    • timings.xlsx for execution time measurements.

The following section lists the possible tests to execute.

Test cases

  1. Number of writing participants x-ray test (2 to 10 participants):

    python3.7 test_Auto.py -t1

  2. Message size dimension x-ray test (x1 to x9):

    python3.7 test_Auto.py -t2

  3. Process size dimension x-ray test (x1 to x10):

    python3.7 test_Auto.py -t3

  4. Parallel split synth test (x1 to x10):

    python3.7 test_Auto.py -t4

  5. Parallel split and join synth test (x1 to x10):

    python3.7 test_Auto.py -t5

  6. Exclusive split synth test (x1 to x10):

    python3.7 test_Auto.py -t6

  7. Exclusive split and join synth test (x1 to x10):

    python3.7 test_Auto.py -t7

  8. Three state-of-the-art process tests:

    python3.7 test_Auto.py -t8

  9. Default execution:

    Without any -t input, the script will execute 5 times the default X-ray Test:

    python3.7 test_Auto.py

  10. Execute all tests together:

    Run the following command to execute all tests sequentially.
    Note: This process will take approximately 3 hours (i7-13700h, 32 GB RAM).

    python3.7 test_Auto.py; python3.7 test_Auto.py -t1; python3.7 test_Auto.py -t2; python3.7 test_Auto.py -t3; python3.7 test_Auto.py -t4; python3.7 test_Auto.py -t5; python3.7 test_Auto.py -t6; python3.7 test_Auto.py -t7; python3.7 test_Auto.py -t8

Clone this wiki locally