From 63f12bad2ae29ed116977a492badfb6a96b26cdc Mon Sep 17 00:00:00 2001 From: asmataamallah25 Date: Tue, 26 Nov 2024 14:09:17 +0100 Subject: [PATCH 01/16] Added documentation to Canis Major --- docs/Troubleshooting.md | 45 +++++++++++++++++++++++++++++++++++++++++ docs/setup.md | 41 +++++++++++++++++++++++++++++++++++++ 2 files changed, 86 insertions(+) create mode 100644 docs/Troubleshooting.md create mode 100644 docs/setup.md diff --git a/docs/Troubleshooting.md b/docs/Troubleshooting.md new file mode 100644 index 0000000..9eaa364 --- /dev/null +++ b/docs/Troubleshooting.md @@ -0,0 +1,45 @@ +# Troubleshooting Guide: Canis Major Integration Tests + +## Common Issue: Build Failure During Integration Tests + +This guide addresses the most common issue encountered with the Canis Major adapter: build failure when running integration tests. If you experience a build failure while executing the `mvn clean test` command, follow these steps to resolve the issue. + +### Solution: Updating 'NGSI Address' environment variable + +1. **Check the IP Address of the Canis Major Container** + + Run the following command to obtain the IP address of the Canis Major container: + + ```bash + docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' + ``` + + Replace `` with the actual name or ID of your Canis Major container. + +2. **Run Tests with Updated NGSI Address** + + Navigate to the `/it` folder and execute the following command: + + ```bash + NGSI_ADDRESS=:4000 mvn clean test + ``` + + Replace `` with the IP address obtained in step 1. + +### Command Breakdown + +The command used in step 2 consists of two main parts: + +1. **Environment Variable Assignment**: + `NGSI_ADDRESS=:4000` + - Sets the `NGSI_ADDRESS` environment variable. + - Specifies the address of the NGSI (Next Generation Service Interface) server or endpoint for test interaction. + +2. **Maven Command**: + `mvn clean test` + - `clean`: Deletes the `target` directory, ensuring a clean build environment. + - `test`: Compiles the source code and runs all unit tests in the project. + +By using this approach, you ensure that the integration tests are executed against the correct NGSI endpoint, which should resolve the build failure issue. + +If you continue to experience problems after following these steps, please open an issue in the GitHub repository with detailed information about the error you're encountering. \ No newline at end of file diff --git a/docs/setup.md b/docs/setup.md new file mode 100644 index 0000000..fcdccd4 --- /dev/null +++ b/docs/setup.md @@ -0,0 +1,41 @@ +# Prerequisites for Running Canis Major + + +This document outlines the prerequisites to running the Canis Major adapter without issues. The setup has been tested on an Ubuntu 22.04.1 LTS virtual machine running on FIWARE Lab. For more information on [FIWARE lab](https://www.fiware.org/developers/fiware-lab/) check the official documentation. + +#### Java setup + +The Canis Major adaptor is currently functional using Java 21. Here are the specific details of the Java version used: + +- **Version**: OpenJDK 21.0.5 +- **Release Date**: October 15, 2024 +- **Distribution**: OpenJDK (Open Java Development Kit) +To install OpenJDK 21 on Ubuntu 22.04, the following command could be used: + +````bash +sudo apt update +sudo apt install openjdk-21-jdk +```` +To verify the Java installation: +````bash +java -version +```` + +#### Docker Setup + +Docker is required to run Canis Major. The following versions have been tested and confirmed to work: + +- **Docker Version**: 27.3.1, build ce12230 +- **Docker Compose Version**: v2.29.7 + +For a proper docker and docker-compose installation follow the instructions provided in the official [documentation](https://docs.docker.com/engine/install/ubuntu/). + +To verify the docker and docker-compose installations: +```bash +docker --version +docker-compose --version +``` + +Proper setup is crucial for the proper functioning of the Canis Major adapter and for avoiding any issues when testing it. + + From a60e7da16c748595b95bc4f1b9ed4f9f55b32d7e Mon Sep 17 00:00:00 2001 From: asmataamallah25 Date: Tue, 10 Dec 2024 15:19:02 +0100 Subject: [PATCH 02/16] Updated documentation --- docs/Troubleshooting.md | 72 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 69 insertions(+), 3 deletions(-) diff --git a/docs/Troubleshooting.md b/docs/Troubleshooting.md index 9eaa364..736b5b9 100644 --- a/docs/Troubleshooting.md +++ b/docs/Troubleshooting.md @@ -33,13 +33,79 @@ The command used in step 2 consists of two main parts: 1. **Environment Variable Assignment**: `NGSI_ADDRESS=:4000` - Sets the `NGSI_ADDRESS` environment variable. - - Specifies the address of the NGSI (Next Generation Service Interface) server or endpoint for test interaction. + - Specifies the address of the ETSI NGSI-LD API of the Canis Major. 2. **Maven Command**: `mvn clean test` - `clean`: Deletes the `target` directory, ensuring a clean build environment. - `test`: Compiles the source code and runs all unit tests in the project. -By using this approach, you ensure that the integration tests are executed against the correct NGSI endpoint, which should resolve the build failure issue. +By using this approach, you ensure that the integration tests are executed against the correct Canis Major endpoint, which should resolve the build failure issue. -If you continue to experience problems after following these steps, please open an issue in the GitHub repository with detailed information about the error you're encountering. \ No newline at end of file +If you continue to experience problems after following the provided steps, please open an issue in the GitHub repository with detailed information about the error you are encountering. + +## Common Issue: Problem to execute the integration test again. + +If you try to execute twice the integration tests, you will receive an error message like the following: + +```bash +[INFO] ------------------------------------------------------- +[INFO] T E S T S +[INFO] ------------------------------------------------------- +[INFO] Running it.RunCucumberTest + +Scenario: A test-store, created at orion-ld, is available through CanisMajor. + # it/store_transactions_on_entities_in_canis_major.feature:4 + + Given CanisMajor is running and available for requests. + + # it.StepDefinitions.setup_canis_major_in_docker() + + And Vault is configured as a signing endpoint. + + # it.StepDefinitions.configure_ethereum_plugin_vault() + + And Franzi is registered in vault. + + # it.StepDefinitions.register_franzi_in_vault() + + And Mira is registered in vault. + + # it.StepDefinitions.register_mira_in_vault() + + When Franzi creates the test-store. + + # it.StepDefinitions.create_test_store() + + Then Only one transaction should be persisted for the entity. + + # it.StepDefinitions.assert_only_one_transaction() + +org.awaitility.core.ConditionTimeoutException: Condition with it.StepDefinitions was not + + fulfilled within 15 seconds. + +at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:165) + +at org.awaitility.core.CallableCondition.await(CallableCondition.java:78) + +at org.awaitility.core.CallableCondition.await(CallableCondition.java:26) + +at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:895) + +at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:864) + +at it.StepDefinitions.assert_only_one_transaction(StepDefinitions.java:559) + +at ✽.Only one transaction should be persisted for the entity. + +(classpath:it/store_transactions_on_entities_in_canis_major.feature:10) + + And The transaction to persist test-store can be read through CanisMajor. + # it.StepDefinitions.get_test_store() +``` + +This is produced by the intent to save again the information of the users' credentials in the Vault. + +### Command Breakdown +Stop the docker compose and start again to ensure the Vault's content is clean before executing the integration tests again. \ No newline at end of file From d9b530915f944871be956f747e7a094acc7abee1 Mon Sep 17 00:00:00 2001 From: asmataamallah25 Date: Mon, 6 Jan 2025 10:16:23 +0100 Subject: [PATCH 03/16] Address review feedback --- docs/Troubleshooting.md | 26 ++++++++++++++++++++++++-- docs/setup.md | 22 ++++++++++++++-------- 2 files changed, 38 insertions(+), 10 deletions(-) diff --git a/docs/Troubleshooting.md b/docs/Troubleshooting.md index 736b5b9..a98f079 100644 --- a/docs/Troubleshooting.md +++ b/docs/Troubleshooting.md @@ -107,5 +107,27 @@ at ✽.Only one transaction should be persisted for the entity. This is produced by the intent to save again the information of the users' credentials in the Vault. -### Command Breakdown -Stop the docker compose and start again to ensure the Vault's content is clean before executing the integration tests again. \ No newline at end of file +### Solution: Clean the docker compose +Stop the docker compose and start again to ensure the Vault's content is clean before executing the integration tests again. + +For a docker compose clean follow these steps for a complete Docker cleanup: +- Stop all running containers + ```shell + sudo docker stop $(sudo docker ps -aq) + ``` +- Remove all containers + ```shell + sudo docker rm $(sudo docker ps -aq) + ``` +- Remove all volumes + ```shell + sudo docker volume rm $(sudo docker volume ls -q) + ``` +- Remove all custom networks + ```shell + sudo docker network prune + ``` + +### Notes +- You can add `-f` flag to skip confirmation prompts. +- Root privileges (sudo) may be required depending on your Docker setup. diff --git a/docs/setup.md b/docs/setup.md index fcdccd4..5cf5b2a 100644 --- a/docs/setup.md +++ b/docs/setup.md @@ -3,7 +3,7 @@ This document outlines the prerequisites to running the Canis Major adapter without issues. The setup has been tested on an Ubuntu 22.04.1 LTS virtual machine running on FIWARE Lab. For more information on [FIWARE lab](https://www.fiware.org/developers/fiware-lab/) check the official documentation. -#### Java setup +## Java setup The Canis Major adaptor is currently functional using Java 21. Here are the specific details of the Java version used: @@ -21,19 +21,25 @@ To verify the Java installation: java -version ```` -#### Docker Setup +## Docker Setup -Docker is required to run Canis Major. The following versions have been tested and confirmed to work: +Docker is required to run Canis Major. The following version have been tested and confirmed to work: -- **Docker Version**: 27.3.1, build ce12230 -- **Docker Compose Version**: v2.29.7 +**Docker Version**: 27.3.1, build ce12230 -For a proper docker and docker-compose installation follow the instructions provided in the official [documentation](https://docs.docker.com/engine/install/ubuntu/). -To verify the docker and docker-compose installations: +For a proper docker installation follow the instructions provided in the official [documentation](https://docs.docker.com/engine/install/ubuntu/). + +To verify the installation follow these commands: ```bash docker --version -docker-compose --version +docker compose version +``` +The output will be similar to this: + +```bash +Docker version 27.3.1, build ce12230 +Docker Compose version v2.29.7 ``` Proper setup is crucial for the proper functioning of the Canis Major adapter and for avoiding any issues when testing it. From 47c4f20bad809eee6ca14228e23d67b8fa57861e Mon Sep 17 00:00:00 2001 From: asmataamallah25 <168739895+asmataamallah25@users.noreply.github.com> Date: Tue, 14 Jan 2025 14:05:35 +0100 Subject: [PATCH 04/16] Create canis-major-integration-guide.md --- docs/canis-major-integration-guide.md | 213 ++++++++++++++++++++++++++ 1 file changed, 213 insertions(+) create mode 100644 docs/canis-major-integration-guide.md diff --git a/docs/canis-major-integration-guide.md b/docs/canis-major-integration-guide.md new file mode 100644 index 0000000..93bf9ba --- /dev/null +++ b/docs/canis-major-integration-guide.md @@ -0,0 +1,213 @@ +# Canis Major test + +Canis Major serves as a blockchain adaptor within the FIWARE ecosystem, providing secure data persistence across blockchain networks and the Context Broker. The workflow is straightforward: + - clients submit transactions containing payload data and wallet credentials. + - Once validated, the data is stored in an ETSI Broker (e.g., Orion-LD Broker) and simultaneously processed into a Merkle tree structure for blockchain integration. + - The system then signs the transaction using the provided wallet credentials and submits it to an Oketh-compatible blockchain. + + ## Integration tests + Let's explore the practical implementation through a series of test commands. First, it is needed to execute the integration tests, by running the following commands: + + ```shell + cd it + docker-compose -f docker-compose/docker-compose-env.yaml -f docker-compose/docker-compose-java.yaml up + NGSI_ADDRESS=localhost:4000 mvn clean test + ``` + +## Entity creation +After running the integartion tests, create an entity by sending this POST request to Canis Major: + +```shell +curl --location 'http://localhost:4000/ngsi-ld/v1/entities/' \ +--header 'Link: ; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \ +--header 'Wallet-Type: vault' \ +--header 'Wallet-Token: vault-plaintext-root-token' \ +--header 'Wallet-Address: http://vault:8200/v1/ethereum/accounts/mira' \ +--header 'Content-Type: application/json' \ +--header 'Accept: application/json' \ +--header 'NGSILD-TENANT: orion' \ +--data '{ + "id": "urn:ngsi-ld:Building:warehouse001", + "type": "Building", + "category": { + "type": "Property", + "value": ["warehouse"] + }, + "address": { + "type": "Property", + "value": { + "streetAddress": "Alexanderplatz 2", + "addressRegion": "Berlin", + "addressLocality": "Mitte", + "postalCode": "10178" + } + } +}' +``` + +### Wallet HTTP Headers +The headers included in the HTTP request are crucial for interacting with the blockchain. Here's a breakdown of each component and its significance: + +#### Wallet-Type +This header specifies the type of wallet service being used, in this case, "vault". By specifying the wallet type, the system can ensure that it uses the appropriate methods and protocols for that specific wallet service. + +#### Wallet-Token +This header provides an authentication token for accessing the vault service. This token ensures that only authorized users can access the wallet's functionalities, such as retrieving private keys or signing transactions. Without proper authentication, the system would be vulnerable to unauthorized access. + +#### Wallet-Address +This header points to the specific Ethereum account associated with the wallet. The wallet address is a unique identifier for an Ethereum account. It is used to send and receive transactions on the Ethereum blockchain. By specifying the wallet address, the system knows which account to interact with for operations such as sending tokens, signing transactions, or querying account balances. + +### Content Headers +- Content-Type: Declares the request body format as JSON +- Accept: Indicates the expected response format as JSON + +## Retrieve the entity types in the Context broker +To retrieve the available entity types in the context broker run the following command: + +```shell +curl -L 'http://localhost:1026/ngsi-ld/v1/types' \ +-H 'Link: ; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \ +-H 'NGSILD-Tenant: orion' +``` +The command returns the following response, demonstrating the available entity types in the Context Broker: +```json +{ + "@context":"https://raw.githubusercontent.com/smart-data-models/dataModel.DistributedLedgerTech/master/context.jsonld","id":"urn:ngsi-ld:EntityTypeList:d77ccfa0-b3cd-11ef-ae1b-0242ac120005","type":"EntityTypeList","typeList":["DLTtxReceipt"] +} +``` +The @context is using a definition on smart data models for a DLTtxReceipt and the NGSILD-Tenant is defined in the start-up of the tests. + +## Retrieve data from Canis Major +To retrieve detailed receipt information for a specific building entity from the Canis Major, use the following HTTP request: + +```shell +curl -L 'http://localhost:4000/ngsi-ld/v1/entities/urn:ngsi-ld:Building:warehouse001' \ +-H 'Link: ; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \ +-H 'Accept: application/json' | jq '.' +``` + +The output will be similar to following response: + +```json +{ + "blockHash": "0x619433204be327dda0d4722482e44310d2f3807e1a23b1fc097ca809f7afb421", + "blockNumber": 193, + "blockNumberRaw": "0xc1", + "cumulativeGasUsed": 23866, + "cumulativeGasUsedRaw": "0x5d3a", + "from": "0xd9fe663797b75d0b3897d55d35e0b4e72307a63f", + "gasUsed": 23866, + "gasUsedRaw": "0x5d3a", + "logs": [ + { + "address": "0x476059cd57800db8eb88f67c2aa38a6fcf8251e0", + "blockHash": "0x619433204be327dda0d4722482e44310d2f3807e1a23b1fc097ca809f7afb421", + "blockNumber": 193, + "blockNumberRaw": "0xc1", + "data": "0x", + "logIndex": 0, + "logIndexRaw": "0x0", + "removed": false, + "topics": [ + "0xa3865c00e01495fc2b86502cae36a4edb139f748682e7d80725a3d6571a482fa", + "0xf85c55023889d6ec0b723c8220471171435040e275059f8e222dfa57a50f0dd5", + "0x33868c71f7186ea974685b553d8eee61eb1e95e0a2c70ed54fcd13db67920f74" + ], + "transactionHash": "0x83f05282a30f77dcfe52ca029b10ba80aac5dad5cc46f5dc437b79e860e4dd65", + "transactionIndex": 0, + "transactionIndexRaw": "0x0", + "type": "mined" + } + ], + "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000000000020000000000000008000200000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000004000000000000000000020000000000000000000000000000000000000000000000000010000100000000000000000000000000000000000000000000000000008000000000000000000000420000000000000000000000000000000000000000000000000000000000000000000000000", + "status": "0x1", + "statusOK": true, + "to": "0x476059cd57800db8eb88f67c2aa38a6fcf8251e0", + "transactionHash": "0x83f05282a30f77dcfe52ca029b10ba80aac5dad5cc46f5dc437b79e860e4dd65", + "transactionIndexRaw": "0x0" +} +``` + +The JSON structure shows a blockchain transaction receipt with the following details: +### Transaction Information +- Block Number: 193 (0xc1) +- Transaction Status: Successful (statusOK: true) +- Transaction Hash: 0x83f05282a30f77dcfe52ca029b10ba80aac5dad5cc46f5dc437b79e860e4dd65 +### Transaction Participants +- From Address: 0xd9fe663797b75d0b3897d55d35e0b4e72307a63f +- To Address: 0x476059cd57800db8eb88f67c2aa38a6fcf8251e0 + +## Retrieve data from the context broker +To retrieve specific DLT transaction receipts from the the context broker, we'll use an NGSI-LD query that filters entities by type and property values. The query targets entities of type DLTtxReceipt and filters them based on the refEntity property matching the Building entity "urn:ngsi-ld:Building:warehouse001". The attrs=TxReceipts parameter in the NGSI-LD query acts as a data filter to limit the response to only include the TxReceipts property, excluding all other properties of the entity and reducing response payload size. + +```shell +curl -L 'http://localhost:1026/ngsi-ld/v1/entities/?type=DLTtxReceipt&q=refEntity%3D%3D%22urn%3Angsi-ld%3ABuilding%3Awarehouse001%22&attrs=TxReceipts' \ +-H 'Link: ; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \ +-H 'NGSILD-Tenant: orion' +``` + +The outpot will be similar to the following json response: + +```json +{ + "@context": "https://raw.githubusercontent.com/smart-data-models/dataModel.DistributedLedgerTech/master/context.jsonld", + "id": "urn:ngsi-ld:dlttxreceipt:0xa46d2e3b190d36fbb8af5d0a1c212d8036cf007e6ec4d1309904e052d25e5499", + "type": "DLTtxReceipt", + "TxReceipts": { + "type": "Property", + "value": { + "blockHash": "0x2022f801a1c094bd3a893ac6f3087c17bb9c673a6b5663f353eded030e1e7161", + "blockNumber": 192, + "blockNumberRaw": "0xc0", + "cumulativeGasUsed": 23866, + "cumulativeGasUsedRaw": "0x5d3a", + "from": "0x34e5b3f990e55d0651b35c817bafb89d2877cb95", + "gasUsed": 23866, + "gasUsedRaw": "0x5d3a", + "logs": [ + { + "address": "0x476059cd57800db8eb88f67c2aa38a6fcf8251e0", + "blockHash": "0x2022f801a1c094bd3a893ac6f3087c17bb9c673a6b5663f353eded030e1e7161", + "blockNumber": 192, + "blockNumberRaw": "0xc0", + "data": "0x", + "logIndex": 0, + "logIndexRaw": "0x0", + "removed": false, + "topics": [ + "0xa3865c00e01495fc2b86502cae36a4edb139f748682e7d80725a3d6571a482fa", + "0xf85c55023889d6ec0b723c8220471171435040e275059f8e222dfa57a50f0dd5", + "0xefc7a4d4c2393e9b62ac4b93b7d199c71e0bb103fdb00fc1b37d7949ad886ddd" + ], + "transactionHash": "0xa46d2e3b190d36fbb8af5d0a1c212d8036cf007e6ec4d1309904e052d25e5499", + "transactionIndex": 0, + "transactionIndexRaw": "0x0", + "type": "mined" + } + ], + "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000000000020000000000000008000200000000000000000000000000000000000000000000000100000000000000100000000000000000000000000000000000000000000000000000000000000020000000000000000000000000008000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000420000000000000000000000000000000000000000200000000000000000000000000000000", + "status": "0x1", + "statusOK": true, + "to": "0x476059cd57800db8eb88f67c2aa38a6fcf8251e0", + "transactionHash": "0xa46d2e3b190d36fbb8af5d0a1c212d8036cf007e6ec4d1309904e052d25e5499", + "transactionIndex": 0, + "transactionIndexRaw": "0x0" + } + } +} +``` + +This JSON response shows a DLTtxReceipt (Distributed Ledger Technology Transaction Receipt) entity in NGSI-LD format: + +### Transaction Details + +- Entity ID: `urn:ngsi-ld:dlttxreceipt:0xa46d2e3b190d36fbb8af5d0a1c212d8036cf007e6ec4d1309904e052d25e5499` +- Block Number: 192 (0xc0) +- Transaction Status: Successful (statusOK: true) +- Block Hash: `0x2022f801a1c094bd3a893ac6f3087c17bb9c673a6b5663f353eded030e1e7161` +- Transaction Hash: `0xa46d2e3b190d36fbb8af5d0a1c212d8036cf007e6ec4d1309904e052d25e5499` + +### Transaction Participants +- From Address: `0x34e5b3f990e55d0651b35c817bafb89d2877cb95` +- To Address: `0x476059cd57800db8eb88f67c2aa38a6fcf8251e0` + From c96b84caaaad70dcef605527cd3dfeb8e915096e Mon Sep 17 00:00:00 2001 From: asmataamallah25 Date: Tue, 21 Jan 2025 14:57:28 +0100 Subject: [PATCH 05/16] Update Docker compose cleanup solutions in Troubleshooting.md --- docs/Troubleshooting.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/docs/Troubleshooting.md b/docs/Troubleshooting.md index a98f079..28d2f06 100644 --- a/docs/Troubleshooting.md +++ b/docs/Troubleshooting.md @@ -107,10 +107,17 @@ at ✽.Only one transaction should be persisted for the entity. This is produced by the intent to save again the information of the users' credentials in the Vault. -### Solution: Clean the docker compose -Stop the docker compose and start again to ensure the Vault's content is clean before executing the integration tests again. +### Solution 1: Clean the docker compose +For a docker compose clean, execute the following command: + ```shell + sudo docker compose -f docker-compose-env.yaml -f docker-compose-java.yaml down -v + ``` +This command deletes all containers, networks, and volumes created on the corresponding compose. It is the normal way to remove resources in a compose. + +### Solution 2: Remove all docker resources from the machine +Delete all the containers, volumes, and networks you have created in your machine, **including others not related to the deployment of Canis Major.** -For a docker compose clean follow these steps for a complete Docker cleanup: +For a complete Docker cleanup of all resources: - Stop all running containers ```shell sudo docker stop $(sudo docker ps -aq) From a388b44e2c37458f50b8decece4dd0a3c517a3d9 Mon Sep 17 00:00:00 2001 From: asmataamallah25 <168739895+asmataamallah25@users.noreply.github.com> Date: Thu, 13 Feb 2025 11:41:06 +0100 Subject: [PATCH 06/16] Update setup.md --- docs/setup.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/setup.md b/docs/setup.md index 5cf5b2a..ae6fa5f 100644 --- a/docs/setup.md +++ b/docs/setup.md @@ -1,7 +1,7 @@ # Prerequisites for Running Canis Major -This document outlines the prerequisites to running the Canis Major adapter without issues. The setup has been tested on an Ubuntu 22.04.1 LTS virtual machine running on FIWARE Lab. For more information on [FIWARE lab](https://www.fiware.org/developers/fiware-lab/) check the official documentation. +This document outlines the prerequisites for running the Canis Major adapter without issues. The setup has been tested on an Ubuntu 22.04.1 LTS virtual machine running on FIWARE Lab. For more information on [FIWARE lab](https://www.fiware.org/developers/fiware-lab/) check the official documentation. ## Java setup @@ -30,7 +30,7 @@ Docker is required to run Canis Major. The following version have been tested an For a proper docker installation follow the instructions provided in the official [documentation](https://docs.docker.com/engine/install/ubuntu/). -To verify the installation follow these commands: +To verify the installation use these commands: ```bash docker --version docker compose version From 6b2a80e4735f418b1b513c480fae29052b1065ae Mon Sep 17 00:00:00 2001 From: asmataamallah25 <168739895+asmataamallah25@users.noreply.github.com> Date: Thu, 13 Feb 2025 11:52:39 +0100 Subject: [PATCH 07/16] Update canis-major-integration-guide.md --- docs/canis-major-integration-guide.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/canis-major-integration-guide.md b/docs/canis-major-integration-guide.md index 93bf9ba..7f1d2e5 100644 --- a/docs/canis-major-integration-guide.md +++ b/docs/canis-major-integration-guide.md @@ -1,7 +1,7 @@ # Canis Major test Canis Major serves as a blockchain adaptor within the FIWARE ecosystem, providing secure data persistence across blockchain networks and the Context Broker. The workflow is straightforward: - - clients submit transactions containing payload data and wallet credentials. + - Clients submit transactions containing payload data and wallet credentials. - Once validated, the data is stored in an ETSI Broker (e.g., Orion-LD Broker) and simultaneously processed into a Merkle tree structure for blockchain integration. - The system then signs the transaction using the provided wallet credentials and submits it to an Oketh-compatible blockchain. @@ -72,7 +72,10 @@ curl -L 'http://localhost:1026/ngsi-ld/v1/types' \ The command returns the following response, demonstrating the available entity types in the Context Broker: ```json { - "@context":"https://raw.githubusercontent.com/smart-data-models/dataModel.DistributedLedgerTech/master/context.jsonld","id":"urn:ngsi-ld:EntityTypeList:d77ccfa0-b3cd-11ef-ae1b-0242ac120005","type":"EntityTypeList","typeList":["DLTtxReceipt"] + "@context": "https://raw.githubusercontent.com/smart-data-models/dataModel.DistributedLedgerTech/master/context.jsonld", + "id": "urn:ngsi-ld:EntityTypeList:d77ccfa0-b3cd-11ef-ae1b-0242ac120005", + "type": "EntityTypeList", + "typeList": ["DLTtxReceipt"] } ``` The @context is using a definition on smart data models for a DLTtxReceipt and the NGSILD-Tenant is defined in the start-up of the tests. From 7e2c288b55246f9bdd43da505c02fd825a014af7 Mon Sep 17 00:00:00 2001 From: asmataamallah25 <168739895+asmataamallah25@users.noreply.github.com> Date: Fri, 14 Feb 2025 09:18:05 +0100 Subject: [PATCH 08/16] Update Troubleshooting.md --- docs/Troubleshooting.md | 76 ++++++++++++++--------------------------- 1 file changed, 26 insertions(+), 50 deletions(-) diff --git a/docs/Troubleshooting.md b/docs/Troubleshooting.md index 28d2f06..d256eac 100644 --- a/docs/Troubleshooting.md +++ b/docs/Troubleshooting.md @@ -11,7 +11,8 @@ This guide addresses the most common issue encountered with the Canis Major adap Run the following command to obtain the IP address of the Canis Major container: ```bash - docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' + docker inspect -f '{{range .NetworkSettings.Networks}}\ + {{.IPAddress}}{{end}}' ``` Replace `` with the actual name or ID of your Canis Major container. @@ -21,6 +22,7 @@ This guide addresses the most common issue encountered with the Canis Major adap Navigate to the `/it` folder and execute the following command: ```bash + cd it NGSI_ADDRESS=:4000 mvn clean test ``` @@ -56,85 +58,59 @@ If you try to execute twice the integration tests, you will receive an error mes Scenario: A test-store, created at orion-ld, is available through CanisMajor. # it/store_transactions_on_entities_in_canis_major.feature:4 - Given CanisMajor is running and available for requests. - # it.StepDefinitions.setup_canis_major_in_docker() - And Vault is configured as a signing endpoint. - # it.StepDefinitions.configure_ethereum_plugin_vault() - And Franzi is registered in vault. - # it.StepDefinitions.register_franzi_in_vault() - And Mira is registered in vault. - # it.StepDefinitions.register_mira_in_vault() - When Franzi creates the test-store. - # it.StepDefinitions.create_test_store() - Then Only one transaction should be persisted for the entity. - # it.StepDefinitions.assert_only_one_transaction() - org.awaitility.core.ConditionTimeoutException: Condition with it.StepDefinitions was not - fulfilled within 15 seconds. - at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:165) - at org.awaitility.core.CallableCondition.await(CallableCondition.java:78) - at org.awaitility.core.CallableCondition.await(CallableCondition.java:26) - at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:895) - at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:864) - at it.StepDefinitions.assert_only_one_transaction(StepDefinitions.java:559) - at ✽.Only one transaction should be persisted for the entity. - (classpath:it/store_transactions_on_entities_in_canis_major.feature:10) - And The transaction to persist test-store can be read through CanisMajor. # it.StepDefinitions.get_test_store() ``` This is produced by the intent to save again the information of the users' credentials in the Vault. -### Solution 1: Clean the docker compose +### Solution: Clean the docker compose For a docker compose clean, execute the following command: ```shell sudo docker compose -f docker-compose-env.yaml -f docker-compose-java.yaml down -v ``` This command deletes all containers, networks, and volumes created on the corresponding compose. It is the normal way to remove resources in a compose. - -### Solution 2: Remove all docker resources from the machine -Delete all the containers, volumes, and networks you have created in your machine, **including others not related to the deployment of Canis Major.** - -For a complete Docker cleanup of all resources: -- Stop all running containers - ```shell - sudo docker stop $(sudo docker ps -aq) - ``` -- Remove all containers - ```shell - sudo docker rm $(sudo docker ps -aq) - ``` -- Remove all volumes - ```shell - sudo docker volume rm $(sudo docker volume ls -q) - ``` -- Remove all custom networks - ```shell - sudo docker network prune - ``` - -### Notes -- You can add `-f` flag to skip confirmation prompts. -- Root privileges (sudo) may be required depending on your Docker setup. +> [!NOTE] +> If you want to remove all the containers, volumes, and networks you have created in your machine, **including others not related to the deployment of Canis Major.** follow these commands for a complete Docker cleanup of all resources: +> - Stop all running containers +> - ```shell +> sudo docker stop $(sudo docker ps -aq) +> ``` +> - Remove all containers +> ```shell +> sudo docker rm $(sudo docker ps -aq) +> ``` +> - Remove all volumes +> ```shell +> sudo docker volume rm $(sudo docker volume ls -q) +> ``` +> - Remove all custom networks +> ```shell +> sudo docker network prune +> ``` + +> [!TIP] +> - You can add `-f` flag to skip confirmation prompts. +> - Root privileges (sudo) may be required depending on your Docker setup. From f61459e163254f650ca22b3ec43c1abc9e81a6b2 Mon Sep 17 00:00:00 2001 From: asmataamallah25 <168739895+asmataamallah25@users.noreply.github.com> Date: Fri, 14 Feb 2025 09:19:44 +0100 Subject: [PATCH 09/16] Update Troubleshooting.md --- docs/Troubleshooting.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/docs/Troubleshooting.md b/docs/Troubleshooting.md index d256eac..8a31f74 100644 --- a/docs/Troubleshooting.md +++ b/docs/Troubleshooting.md @@ -95,21 +95,21 @@ This command deletes all containers, networks, and volumes created on the corres > [!NOTE] > If you want to remove all the containers, volumes, and networks you have created in your machine, **including others not related to the deployment of Canis Major.** follow these commands for a complete Docker cleanup of all resources: > - Stop all running containers -> - ```shell -> sudo docker stop $(sudo docker ps -aq) -> ``` -> - Remove all containers -> ```shell -> sudo docker rm $(sudo docker ps -aq) -> ``` -> - Remove all volumes -> ```shell -> sudo docker volume rm $(sudo docker volume ls -q) -> ``` -> - Remove all custom networks -> ```shell -> sudo docker network prune -> ``` +> ```shell +> sudo docker stop $(sudo docker ps -aq) +> ``` +> - Remove all containers +> ```shell +> sudo docker rm $(sudo docker ps -aq) +> ``` +> - Remove all volumes +> ```shell +> sudo docker volume rm $(sudo docker volume ls -q) +> ``` +> - Remove all custom networks +> ```shell +> sudo docker network prune +> ``` > [!TIP] > - You can add `-f` flag to skip confirmation prompts. From 3aef1bdec011841640507a39e75f98186049e9bc Mon Sep 17 00:00:00 2001 From: asmataamallah25 <168739895+asmataamallah25@users.noreply.github.com> Date: Fri, 14 Feb 2025 09:20:52 +0100 Subject: [PATCH 10/16] Update Troubleshooting.md --- docs/Troubleshooting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Troubleshooting.md b/docs/Troubleshooting.md index 8a31f74..6cbff25 100644 --- a/docs/Troubleshooting.md +++ b/docs/Troubleshooting.md @@ -113,4 +113,4 @@ This command deletes all containers, networks, and volumes created on the corres > [!TIP] > - You can add `-f` flag to skip confirmation prompts. -> - Root privileges (sudo) may be required depending on your Docker setup. +> - Root privileges `sudo` may be required depending on your Docker setup. From 456d0adbe39f181c0a4d6a952422ca2534471cc9 Mon Sep 17 00:00:00 2001 From: asmataamallah25 <168739895+asmataamallah25@users.noreply.github.com> Date: Fri, 14 Feb 2025 10:06:49 +0100 Subject: [PATCH 11/16] Update canis-major-integration-guide.md --- docs/canis-major-integration-guide.md | 79 +++++++++++++++------------ 1 file changed, 45 insertions(+), 34 deletions(-) diff --git a/docs/canis-major-integration-guide.md b/docs/canis-major-integration-guide.md index 7f1d2e5..dbc236c 100644 --- a/docs/canis-major-integration-guide.md +++ b/docs/canis-major-integration-guide.md @@ -9,24 +9,26 @@ Canis Major serves as a blockchain adaptor within the FIWARE ecosystem, providin Let's explore the practical implementation through a series of test commands. First, it is needed to execute the integration tests, by running the following commands: ```shell - cd it - docker-compose -f docker-compose/docker-compose-env.yaml -f docker-compose/docker-compose-java.yaml up - NGSI_ADDRESS=localhost:4000 mvn clean test +cd it +docker-compose -f docker-compose/docker-compose-env.yaml -f docker-compose/docker-compose-java.yaml up +NGXI_ADDRESS=localhost:4000 mvn clean test ``` ## Entity creation -After running the integartion tests, create an entity by sending this POST request to Canis Major: +After running the integration tests, create an entity by sending this POST request to Canis Major: ```shell -curl --location 'http://localhost:4000/ngsi-ld/v1/entities/' \ ---header 'Link: ; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \ ---header 'Wallet-Type: vault' \ ---header 'Wallet-Token: vault-plaintext-root-token' \ ---header 'Wallet-Address: http://vault:8200/v1/ethereum/accounts/mira' \ ---header 'Content-Type: application/json' \ ---header 'Accept: application/json' \ ---header 'NGSILD-TENANT: orion' \ ---data '{ +curl -iX POST 'http://localhost:4000/ngsi-ld/v1/entities/' \ + --H 'Link: ; rel="http://www.w3.org/ns/json-ld#context";\ + type="application/ld+json"' \ + --H 'Wallet-Type: vault' \ + --H 'Wallet-Token: vault-plaintext-root-token' \ + --H 'Wallet-Address: http://vault:8200/v1/ethereum/accounts/mira' \ + --H 'Content-Type: application/json' \ + --H 'Accept: application/json' \ + --H 'NGSILD-TENANT: orion' \ + --data '{ "id": "urn:ngsi-ld:Building:warehouse001", "type": "Building", "category": { @@ -49,7 +51,7 @@ curl --location 'http://localhost:4000/ngsi-ld/v1/entities/' \ The headers included in the HTTP request are crucial for interacting with the blockchain. Here's a breakdown of each component and its significance: #### Wallet-Type -This header specifies the type of wallet service being used, in this case, "vault". By specifying the wallet type, the system can ensure that it uses the appropriate methods and protocols for that specific wallet service. +This header specifies the type of wallet service being used, in this case, `vault`. By specifying the wallet type, the system can ensure that it uses the appropriate methods and protocols for that specific wallet service. #### Wallet-Token This header provides an authentication token for accessing the vault service. This token ensures that only authorized users can access the wallet's functionalities, such as retrieving private keys or signing transactions. Without proper authentication, the system would be vulnerable to unauthorized access. @@ -57,36 +59,45 @@ This header provides an authentication token for accessing the vault service. Th #### Wallet-Address This header points to the specific Ethereum account associated with the wallet. The wallet address is a unique identifier for an Ethereum account. It is used to send and receive transactions on the Ethereum blockchain. By specifying the wallet address, the system knows which account to interact with for operations such as sending tokens, signing transactions, or querying account balances. +### Link Header +The Link header specifies the JSON-LD @context to be used, defining the semantic meaning of the terms used in the entity. The used link header contains three main parts: +- The URL of the context file +- The relationship type `rel="http://www.w3.org/ns/json-ld#context"` +- The content type `type="application/ld+json"` + +This context helps in standardizing the data model and ensures interoperability and in our case, it's referencing a context file from the Smart Data Models initiative for Building entities. The Link Header used here is an HTTP Link Header using the `http://www.w3.org/ns/json-ld#context` link relation pointing to the `@context` file defining the data model for [Building](https://smart-data-models.github.io/dataModel.Building/context.jsonld). + +### NGSILD-TENANT Header +This header is used for multi-tenancy support in NGSI-LD implementations. In our case, "orion" is the tenant identifier. However, different tenants can have their own isolated set of entities, even if they have the same entity IDs enabling multiple organizations to use the same NGSI-LD broker without data interference. + ### Content Headers -- Content-Type: Declares the request body format as JSON -- Accept: Indicates the expected response format as JSON +- `Content-Type`: Declares the request body format as JSON +- `Accept`: Indicates the expected response format as JSON ## Retrieve the entity types in the Context broker To retrieve the available entity types in the context broker run the following command: ```shell -curl -L 'http://localhost:1026/ngsi-ld/v1/types' \ --H 'Link: ; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \ --H 'NGSILD-Tenant: orion' +curl -iX GET 'http://localhost:1026/ngsi-ld/v1/types' \ + -H 'Link: ; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \ + -H 'NGSILD-Tenant: orion' ``` The command returns the following response, demonstrating the available entity types in the Context Broker: ```json -{ - "@context": "https://raw.githubusercontent.com/smart-data-models/dataModel.DistributedLedgerTech/master/context.jsonld", - "id": "urn:ngsi-ld:EntityTypeList:d77ccfa0-b3cd-11ef-ae1b-0242ac120005", - "type": "EntityTypeList", - "typeList": ["DLTtxReceipt"] -} +"@context": "https://raw.githubusercontent.com/smart-data-models/dataModel.DistributedLedgerTech/master/context.jsonld", +"id": "urn:ngsi-ld:EntityTypeList:d77ccfa0-b3cd-11ef-ae1b-0242ac120005", +"type": "EntityTypeList", +"typeList": ["DLTtxReceipt"] ``` -The @context is using a definition on smart data models for a DLTtxReceipt and the NGSILD-Tenant is defined in the start-up of the tests. +The `@context` is using a definition on smart data models for a `DLTtxReceipt` and the `NGSILD-Tenant` is defined in the start-up of the tests. ## Retrieve data from Canis Major To retrieve detailed receipt information for a specific building entity from the Canis Major, use the following HTTP request: ```shell -curl -L 'http://localhost:4000/ngsi-ld/v1/entities/urn:ngsi-ld:Building:warehouse001' \ --H 'Link: ; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \ --H 'Accept: application/json' | jq '.' +curl -iX GET 'http://localhost:4000/ngsi-ld/v1/entities/urn:ngsi-ld:Building:warehouse001' \ + -H 'Link: ; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \ + -H 'Accept: application/json' ``` The output will be similar to following response: @@ -144,12 +155,12 @@ The JSON structure shows a blockchain transaction receipt with the following det To retrieve specific DLT transaction receipts from the the context broker, we'll use an NGSI-LD query that filters entities by type and property values. The query targets entities of type DLTtxReceipt and filters them based on the refEntity property matching the Building entity "urn:ngsi-ld:Building:warehouse001". The attrs=TxReceipts parameter in the NGSI-LD query acts as a data filter to limit the response to only include the TxReceipts property, excluding all other properties of the entity and reducing response payload size. ```shell -curl -L 'http://localhost:1026/ngsi-ld/v1/entities/?type=DLTtxReceipt&q=refEntity%3D%3D%22urn%3Angsi-ld%3ABuilding%3Awarehouse001%22&attrs=TxReceipts' \ --H 'Link: ; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \ --H 'NGSILD-Tenant: orion' +curl -iX GET 'http://localhost:1026/ngsi-ld/v1/entities/?type=DLTtxReceipt&q=refEntity%3D%3D%22urn%3Angsi-ld%3ABuilding%3Awarehouse001%22&attrs=TxReceipts' \ + -H 'Link: ; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \ + -H 'NGSILD-Tenant: orion' ``` -The outpot will be similar to the following json response: +The output will be similar to the following json response: ```json { @@ -200,7 +211,7 @@ The outpot will be similar to the following json response: } ``` -This JSON response shows a DLTtxReceipt (Distributed Ledger Technology Transaction Receipt) entity in NGSI-LD format: +This JSON response shows a `DLTtxReceipt` (Distributed Ledger Technology Transaction Receipt based on the following [data model](https://github.com/smart-data-models/dataModel.DistributedLedgerTec)) entity in NGSI-LD format. ### Transaction Details From 1bd556888de5b447e5a39035643efc10bad9a99e Mon Sep 17 00:00:00 2001 From: Jason Fox Date: Fri, 14 Feb 2025 13:50:24 +0100 Subject: [PATCH 12/16] Update canis-major-integration-guide.md --- docs/canis-major-integration-guide.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/canis-major-integration-guide.md b/docs/canis-major-integration-guide.md index dbc236c..222c41e 100644 --- a/docs/canis-major-integration-guide.md +++ b/docs/canis-major-integration-guide.md @@ -11,6 +11,9 @@ Canis Major serves as a blockchain adaptor within the FIWARE ecosystem, providin ```shell cd it docker-compose -f docker-compose/docker-compose-env.yaml -f docker-compose/docker-compose-java.yaml up +``` + +```shell NGXI_ADDRESS=localhost:4000 mvn clean test ``` From 201a08f0c3954eeebaada06def39ffe32994dbf1 Mon Sep 17 00:00:00 2001 From: Jason Fox Date: Fri, 14 Feb 2025 13:52:25 +0100 Subject: [PATCH 13/16] Update Troubleshooting.md --- docs/Troubleshooting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Troubleshooting.md b/docs/Troubleshooting.md index 6cbff25..7c29226 100644 --- a/docs/Troubleshooting.md +++ b/docs/Troubleshooting.md @@ -11,7 +11,7 @@ This guide addresses the most common issue encountered with the Canis Major adap Run the following command to obtain the IP address of the Canis Major container: ```bash - docker inspect -f '{{range .NetworkSettings.Networks}}\ + docker inspect -f '{{range .NetworkSettings.Networks}} \ {{.IPAddress}}{{end}}' ``` From d39bf92bb37e4e93ffb12e4627470d12a78ad2d6 Mon Sep 17 00:00:00 2001 From: Jason Fox Date: Fri, 14 Feb 2025 14:00:33 +0100 Subject: [PATCH 14/16] Update Troubleshooting.md --- docs/Troubleshooting.md | 50 +++++++++++++++++++++++++---------------- 1 file changed, 31 insertions(+), 19 deletions(-) diff --git a/docs/Troubleshooting.md b/docs/Troubleshooting.md index 7c29226..9cdaef1 100644 --- a/docs/Troubleshooting.md +++ b/docs/Troubleshooting.md @@ -86,30 +86,42 @@ at ✽.Only one transaction should be persisted for the entity. This is produced by the intent to save again the information of the users' credentials in the Vault. -### Solution: Clean the docker compose +### Solution 1: Clean the docker compose + +> [!NOTE] +> This is the preferred non-invasive solution + For a docker compose clean, execute the following command: ```shell sudo docker compose -f docker-compose-env.yaml -f docker-compose-java.yaml down -v ``` This command deletes all containers, networks, and volumes created on the corresponding compose. It is the normal way to remove resources in a compose. -> [!NOTE] -> If you want to remove all the containers, volumes, and networks you have created in your machine, **including others not related to the deployment of Canis Major.** follow these commands for a complete Docker cleanup of all resources: -> - Stop all running containers -> ```shell -> sudo docker stop $(sudo docker ps -aq) -> ``` -> - Remove all containers -> ```shell -> sudo docker rm $(sudo docker ps -aq) -> ``` -> - Remove all volumes -> ```shell -> sudo docker volume rm $(sudo docker volume ls -q) -> ``` -> - Remove all custom networks -> ```shell -> sudo docker network prune -> ``` + + +### Solution 2: Remove all running containers and resources + +> [!WARNING] +> The following clean up will remove multiple containers **including others not related to the deployment of Canis Major.** + +In the case that you are experiencing a conflict and you want to remove all the containers, volumes, and networks you have created in your machine, +follow these commands for a complete Docker cleanup of all resources. + +- Stop all running containers +```shell +sudo docker stop $(sudo docker ps -aq) +``` +- Remove all containers +```shell +sudo docker rm $(sudo docker ps -aq) +``` +- Remove all volumes +```shell +sudo docker volume rm $(sudo docker volume ls -q) +``` +- Remove all custom networks +```shell +sudo docker network prune +``` > [!TIP] > - You can add `-f` flag to skip confirmation prompts. From 02ab8aaf90f036f7ba0a573ea13c8045d61d49bd Mon Sep 17 00:00:00 2001 From: asmataamallah25 Date: Wed, 11 Jun 2025 20:22:52 +0200 Subject: [PATCH 15/16] Comment source code and add more documentation --- docs/canis_major_technical_overview.md | 309 ++++++++++++++++++ it/src/main/java/it/pojo/Address.java | 2 +- .../main/java/it/pojo/CMEntitesResponse.java | 2 +- .../main/java/it/pojo/CMEntityResponse.java | 2 + it/src/main/java/it/pojo/Entity.java | 2 +- .../main/java/it/pojo/EntityTransactions.java | 6 +- it/src/main/java/it/pojo/ErrorMessage.java | 1 + .../java/it/pojo/EthereumPluginMount.java | 2 + it/src/main/java/it/pojo/Oauth2Response.java | 1 + it/src/main/java/it/pojo/PluginConfig.java | 1 + it/src/main/java/it/pojo/Property.java | 1 + it/src/main/java/it/pojo/TestAccount.java | 2 + it/src/main/java/it/pojo/TxDetails.java | 2 +- it/src/main/java/it/pojo/VaultAccount.java | 1 + it/src/test/java/it/RunCucumberTest.java | 1 + it/src/test/java/it/StepDefinitions.java | 15 +- .../aeicontract/AlaDIDPubkeyResolver.java | 2 + .../org/fiware/canismajor/Application.java | 8 +- .../DefaultAccountProperties.java | 1 + .../configuration/DefaultVaultProperties.java | 1 + .../configuration/EthereumProperties.java | 1 + .../configuration/GeneralProperties.java | 4 +- .../exception/AccountException.java | 1 + .../exception/NGSIConnectException.java | 1 + .../exception/PersistenceException.java | 1 + .../exception/SigningException.java | 1 + .../exception/TransactionException.java | 1 + .../canismajor/exception/VaultException.java | 1 + .../canismajor/mapping/TransactionMapper.java | 1 + .../canismajor/mapping/TxReceiptMapper.java | 4 + .../repository/EntityRepository.java | 3 + .../canismajor/rest/EntitiesController.java | 3 + .../canismajor/rest/NGSILDController.java | 7 + 33 files changed, 371 insertions(+), 20 deletions(-) create mode 100644 docs/canis_major_technical_overview.md diff --git a/docs/canis_major_technical_overview.md b/docs/canis_major_technical_overview.md new file mode 100644 index 0000000..4f194b6 --- /dev/null +++ b/docs/canis_major_technical_overview.md @@ -0,0 +1,309 @@ +# Canis Major: Technical Overview + + +This document is intended to supplement existing documentation within the Canis Major component by providing additional explanations and context for the following key technologies: + +- Ethereum Blockchain +- HashiCorp Vault +- AEI contract +- Ganache + +## Ethereum Blockchain Overview + +Blockchain is s distributed ledger that is decentralized, peer-to-peer, and immutable. It is a chain of blocks containing a set of transactions each linked to the previous one through a cryptographic hash. + + +* Decentralized: Blockchain utilizes a distributed ledger architecture, eliminating a single point of control. + +* Peer-to-Peer: Blockchain operates over a P2P network topology, where nodes communicate directly for transaction propagation and block validation, removing intermediaries and enhancing fault tolerance. + +* Immutable: Data appended to the blockchain is cryptographically linked via hash pointers. Once confirmed, blocks are tamper-evident and irreversible, ensuring data integrity. + +### Key Components of Ethereum + +**1. Nodes** + +A node in Ethereum is an individual instance of client software that participates in the network by maintaining a local copy of the blockchain, and validating transactions. The Ethereum network consists of multiple interconnected nodes, forming a peer-to-peer distributed ledger system. + +The Ethereum Client is a software implementation of the Ethereum protocol that allows users to interact with the blockachin. + +**2. Ethereum Virtual Machine (EVM)** + +The EVM is the runtime environment for smart contracts on Ethereum. It is a sandbox that is isolated from the other parts of the system and responsible for the execution of smart contracts and other decentralized applications. + + +**3. Smart Contracts** + +Smart contracts are self-executing contracts with the terms of the agreement written directly into lines of code. They automate processes, and enforce agreements without the need for intermediaries. + + +**4. Ether (ETH)** +Ether is the native cryptocurrency of Ethereum, used for transaction fees, staking, and as a digital asset. + + +Functionality: It incentivizes network participants, such as validators, and facilitates the execution of smart contracts. + + +**5.Gas** + +Gas is a unit of measurement that represents the computational effort required to execute transactions or smart contracts on the Ethereum network It prevents spam, and ensures the network runs smoothly by charging for computational resources used. + +**Gas Calculation** + +Gas fees are calculated using the following formula: + +Gas fee = Amount of gas used for an operation x cost per unit + +> [!NOTE] +> These additional terms are essential for understanding Ethereum transaction fees: +> The Gas Limit is the maximum amount of gas a user is willing to spend on a transaction. +> The Base Fee is the minimum fee required for a transaction to be included in a block, which adjusts based on network congestion. +> The Priority Fee (Tip) is an optional additional fee to incentivize validators to process a transaction more quickly. + + +**6.Accounts** + +An Ethereum account is an entity identified with an addreess. It has an Ether balance, interact with smart contracts and send transactions on Ethereum. + +There are two types of accounts: +- Externally Owned Accounts (EOA) +- Contract Accounts + +**7.Wallets** + +Wallets are used for storing, managing, and interact with the Ether cryptocurrency and tokens. Ethereum wallet stores as well the cryptographic keys (public and private) that enable users to access their blockchain-based assets, sign transactions, and interact with decentralized applications and smart contracts. + +There are several types of wallets including: +- Physical wallets +- Mobile applications +- Browser wallets / web applications +- Browser extension wallets +- Desktop wallets + +> [!Note] +> You should distinguish between an account and a wallet: +> An account is the actual entity on the blockchain (with an address and balance), while a wallet is the tool used to access, manage, and interact with the accounts + +**8.Transactions in the scope of Canis Major** + +Transactions are fundamental concepts of any Blockchain technology, as they enable the movement of Digital assets between the participants in the Ethereum network. + +In this context of Canis Major we consider an example for retrieving detailed transaction receipt information for a specific building entity. This transaction receipt was persisted in Orion-LD context broker. Find more details [here](https://github.com/asmataamallah25/CanisMajor/blob/documentation/docs/canis-major-integration-guide.md) + +```shell + { + "@context": "https://raw.githubusercontent.com/smart-data-models/dataModel.DistributedLedgerTech/master/context.jsonld", + "id": "urn:ngsi-ld:dlttxreceipt:0xa46d2e3b190d36fbb8af5d0a1c212d8036cf007e6ec4d1309904e052d25e5499", + "type": "DLTtxReceipt", + "TxReceipts": { + "type": "Property", + "value": { + "blockHash": "0x2022f801a1c094bd3a893ac6f3087c17bb9c673a6b5663f353eded030e1e7161", + "blockNumber": 192, + "blockNumberRaw": "0xc0", + "cumulativeGasUsed": 23866, + "cumulativeGasUsedRaw": "0x5d3a", + "from": "0x34e5b3f990e55d0651b35c817bafb89d2877cb95", + "gasUsed": 23866, + "gasUsedRaw": "0x5d3a", + "logs": [ + { + "address": "0x476059cd57800db8eb88f67c2aa38a6fcf8251e0", + "blockHash": "0x2022f801a1c094bd3a893ac6f3087c17bb9c673a6b5663f353eded030e1e7161", + "blockNumber": 192, + "blockNumberRaw": "0xc0", + "data": "0x", + "logIndex": 0, + "logIndexRaw": "0x0", + "removed": false, + "topics": [ + "0xa3865c00e01495fc2b86502cae36a4edb139f748682e7d80725a3d6571a482fa", + "0xf85c55023889d6ec0b723c8220471171435040e275059f8e222dfa57a50f0dd5", + "0xefc7a4d4c2393e9b62ac4b93b7d199c71e0bb103fdb00fc1b37d7949ad886ddd" + ], + "transactionHash": "0xa46d2e3b190d36fbb8af5d0a1c212d8036cf007e6ec4d1309904e052d25e5499", + "transactionIndex": 0, + "transactionIndexRaw": "0x0", + "type": "mined" + } + ], + "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000000000020000000000000008000200000000000000000000000000000000000000000000000100000000000000100000000000000000000000000000000000000000000000000000000000000020000000000000000000000000008000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000420000000000000000000000000000000000000000200000000000000000000000000000000", + "status": "0x1", + "statusOK": true, + "to": "0x476059cd57800db8eb88f67c2aa38a6fcf8251e0", + "transactionHash": "0xa46d2e3b190d36fbb8af5d0a1c212d8036cf007e6ec4d1309904e052d25e5499", + "transactionIndex": 0, + "transactionIndexRaw": "0x0" + } + } +} +``` +To assist with understanding the structure and significance of the transcation receipt, concise explanations as well as the actual value from for each field is provided: + +1. **blockHash** + - Type: String (hexadecimal) + - Description: The unique hash of the block in which this transaction was included. + - Value: + "0x2022f801a1c094bd3a893ac6f3087c17bb9c673a6b5663f353eded030e1e7161" + + 2. **blockNumber** + - Type: Integer + - Description: The number of the block in which the transaction was included, in decimal format. + - Value: 192 + +3. **blockNumberRaw** + - Type: String (hexadecimal) + - Description: The block number in hexadecimal format. + - Value: "0xc0" (which is 192 in decimal) + +4. **cumulativeGasUsed** + - Type: Integer + - Description: The total amount of gas used in the block up to and including this transaction, in decimal format. + - Value: 23866 + +5. **cumulativeGasUsedRaw** + - Type: String (hexadecimal) + - Description:The cumulative gas used in hexadecimal format. + - Value: "0x5d3a" (which is 23866 in decimal) + +6. **from** + - Type: String (hexadecimal) + -Description: The address of the account that initiated the transaction. + - Value: "0x34e5b3f990e55d0651b35c817bafb89d2877cb95" + + 7. **to** + - Type: String (hexadecimal) + - Description: The address of the contract or account that received the transaction. + - Value:"0x476059cd57800db8eb88f67c2aa38a6fcf8251e0" +> [!NOTE] +> The "from" and "to" in Ethereum transactions are in hexadecimal format (hexdec) for several important technical and practical reasons: +> - Hexadecimal is a compact way to represent binary data. +> - Ethereum addresses are derived from hashes of public keys. Storing and transmitting addresses in hexadecimal strings is a common standard that compatibility across tools and systems. +> - Ethereum addresses always start with 0x to indicate they are in hexadecimal format, which helps distinguish them from other fields and prevents confusion. + +8. **gasUsed** + - Type: Integer + - Description: The amount of gas used by this specific transaction, in decimal format. + - Value: 23866 + +9. **gasUsedRaw** + - Type: String (hexadecimal) + - Description: The gas used by this transaction in hexadecimal format. + - Value: "0x5d3a" (which is 23866 in decimal) + +10. **logs** + - Type: Array of Objects + - Description: An array of log objects generated by this transaction. Each log represents an event emitted by a smart contract. + - Log Object Fields + - address: The address of the contract that emitted the log. + - blockHash: The hash of the block containing the log. + - blockNumber: The block number containing the log. + - blockNumberRaw: The block number in hexadecimal. + - data: Additional data associated with the event (empty in this example). + - logIndex: The position of the log in the block’s logs array (decimal). + - logIndexRaw: The log index in hexadecimal. + - removed: Indicates if the log was removed due to a chain reorganization (false if not removed). + - topics: Array of indexed event parameters. The first topic is usually the event signature. + - transactionHash: The hash of the transaction that generated this log. + - transactionIndex: The position of the transaction in the block (decimal). + - transactionIndexRaw: The transaction index in hexadecimal. + - type: The type of log (e.g., "mined" for a log from a mined transaction). + +11. **logsBloom** + - Type: String (hexadecimal) + - Description: A bloom filter for logs generated by this transaction. Used for efficient log filtering. + - Value: "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000000000020000000000000008000200000000000000000000000000000000000000000000000100000000000000100000000000000000000000000000000000000000000000000000000000000020000000000000000000000000008000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000420000000000000000000000000000000000000000200000000000000000000000000000000 + - Additional explanation: The logsBloom field is a special kind of filter that is represented as a long hexadecimal string. It is generated for each transaction and contains information about all the logs (events) produced by that transaction on the Ethereum blockchain. The filter is a bit pattern, which means that when an event is logged, it sets certain bits based on the event’s properties. + +12. **status** + - Type: String (hexadecimal) + - Description: Indicates whether the transaction was successful. "0x1" means success, "0x0" means failure. + - Value: "0x1" + +13. **statusOK** + - Type: Boolean + - Description: A boolean indicating if the transaction was successful. + - Value: true + +14. **transactionHash** + - Type: String (hexadecimal) + - Description: The unique hash of the transaction. + - Value: "0xa46d2e3b190d36fbb8af5d0a1c212d8036cf007e6ec4d1309904e052d25e5499" + +15. **transactionIndexRaw** + - Type: String (hexadecimal) + - Description: The position of the transaction in the block, in hexadecimal format. + - Value: "0x0" (which is 0 in decimal) + +## HashiCorp Vault + +As part of Canis Major commitment to security and efficient secrets management, HashiCorp Vault is integrated to provide robust protection for sensitive data. This section explains how Vault functions within Canis Major. + +### Vault Overview + +HashiCorp Vault is a tool designed for securely storing and managing sensitive data such as passwords, tokens, cryptographic key, and digital wallet credentials, providing a centralized way to manage access to secrets. +In Canis Major, Vault serves as a secure wallet for managing digital wallet information and performing the cryptographic operation of signing transactions. + +### Configuration of Vault service + +The Docker Compose configuration sets up a vault-server service with the following key points: +Key Points: + +- Image + Uses a customized image (vault-ethereum) for Ethereum-related use cases. +- Environment Variables: + VAULT_ADDR: Specifies where Vault listens (http://0.0.0.0:8200). + VAULT_DEV_ROOT_TOKEN_ID: Sets a development root token (vault-plaintext-root-token) for authentication. +- Security: + Adds IPC_LOCK capability for enhanced security by preventing memory swapping. +- Networking: + Connects to test-network for communication with other services. + +### Vault Backed Signing Service + +The Canis Major source code implements VaultBackedSigningService that interacts with Vault's API to: + +- Retrieve wallet account addresses. +- Sign transactions securely using Vault. + + +## AEI contract + +## AEI contract overview + +The AEI Contract is a smart contract written in Solidity that implements the ERC721 standard. This smart contract is designed to be used with Ethereum +compatible clients and particularly designed to be integrated with Canis Major adaptor, enabling the storage and management of digital assets and their +relationships on the blockchain. + +The AEI Contract allows for the creation, updating, and deletion of digital assets (entities). Each entity can be associated with multiple metadata entries. +These entities can have multiple relationships with other assets. +The contract is specifically designed to store the NGSI-LD data model. + +> [!Note] +> The AEI Contract supports several key methods for managing assets and their metadata: +> - createAsset: Creates a new asset with a unique identifier. +> - getAsset: Retrieves information about a specific asset. +> - updateAsset: Updates the metadata associated with an asset. +> - removeAsset: Deletes an asset. +> - addMetadata: Adds new metadata to an asset. +> - getMetadatas: Retrieves all metadata associated with an asset. +> - removeMetadata: Deletes specific metadata from an asset. +> - addRelation: Establishes a relationship between two assets. +> - getRelations: Lists all relationships of an asset. +> - removeRelation: Deletes a relationship between assets + +### Integration with Canis Major + + + + +## Ganache + +Ganache CLI is used in the Canis Major project as a local Ethereum blockchain emulator that provides: +- A fast and customizable local blockchain environment for testing and development +- Instant mining of transactions without network overheads +- Zero transaction costs for testing +- Pre-configured accounts with test Ether +- Customizable gas price and mining speed + + diff --git a/it/src/main/java/it/pojo/Address.java b/it/src/main/java/it/pojo/Address.java index c27e685..f3e071f 100644 --- a/it/src/main/java/it/pojo/Address.java +++ b/it/src/main/java/it/pojo/Address.java @@ -2,9 +2,9 @@ import lombok.Data; +// This class contains the field of an address @Data public class Address { - private String streetAddress; private String addressRegion; private String adressLocality; diff --git a/it/src/main/java/it/pojo/CMEntitesResponse.java b/it/src/main/java/it/pojo/CMEntitesResponse.java index c253ef0..29c2922 100644 --- a/it/src/main/java/it/pojo/CMEntitesResponse.java +++ b/it/src/main/java/it/pojo/CMEntitesResponse.java @@ -7,7 +7,7 @@ @Data public class CMEntitesResponse { - + // Represent private long offset; private long limit; private long count; diff --git a/it/src/main/java/it/pojo/CMEntityResponse.java b/it/src/main/java/it/pojo/CMEntityResponse.java index 659a210..6105e15 100644 --- a/it/src/main/java/it/pojo/CMEntityResponse.java +++ b/it/src/main/java/it/pojo/CMEntityResponse.java @@ -2,8 +2,10 @@ import lombok.Data; +// This class Represent a response entity @Data public class CMEntityResponse { + private long id; private String entityId; private Object txDetails; diff --git a/it/src/main/java/it/pojo/Entity.java b/it/src/main/java/it/pojo/Entity.java index 9db56f8..3d6ee4c 100644 --- a/it/src/main/java/it/pojo/Entity.java +++ b/it/src/main/java/it/pojo/Entity.java @@ -8,9 +8,9 @@ import java.net.URI; import java.util.Map; +// This class represents a generic entity with flexible properties @Data public class Entity { - @JsonProperty("@context") private Object atContext = null; private URI id; diff --git a/it/src/main/java/it/pojo/EntityTransactions.java b/it/src/main/java/it/pojo/EntityTransactions.java index a02bc6c..c96a089 100644 --- a/it/src/main/java/it/pojo/EntityTransactions.java +++ b/it/src/main/java/it/pojo/EntityTransactions.java @@ -5,10 +5,10 @@ import java.util.List; +// This class encapsulates transaction data related to a specific entity, identified by entityId. @Getter @RequiredArgsConstructor public class EntityTransactions { - - private final String entityId; - private final List txDetails; + private final String entityId; // Unique identifier for the entity + private final List txDetails; // List of transaction details } diff --git a/it/src/main/java/it/pojo/ErrorMessage.java b/it/src/main/java/it/pojo/ErrorMessage.java index 88c5728..d525ce1 100644 --- a/it/src/main/java/it/pojo/ErrorMessage.java +++ b/it/src/main/java/it/pojo/ErrorMessage.java @@ -4,6 +4,7 @@ import java.util.List; +// This class is designed to encapsulate error messages @Data public class ErrorMessage { diff --git a/it/src/main/java/it/pojo/EthereumPluginMount.java b/it/src/main/java/it/pojo/EthereumPluginMount.java index 0943085..77d2653 100644 --- a/it/src/main/java/it/pojo/EthereumPluginMount.java +++ b/it/src/main/java/it/pojo/EthereumPluginMount.java @@ -2,6 +2,8 @@ import lombok.Data; +/* This class represents the configuration for mounting an Ethereum plugin. +It encapsulates the type of plugin being mounted, which defaults to "vault-ethereum*/ @Data public class EthereumPluginMount { private String type = "vault-ethereum"; diff --git a/it/src/main/java/it/pojo/Oauth2Response.java b/it/src/main/java/it/pojo/Oauth2Response.java index 9b3925b..1c1234b 100644 --- a/it/src/main/java/it/pojo/Oauth2Response.java +++ b/it/src/main/java/it/pojo/Oauth2Response.java @@ -5,6 +5,7 @@ import java.util.ArrayList; import java.util.List; +// This class models the standard structure of an OAuth2 token response,. @Data public class Oauth2Response { diff --git a/it/src/main/java/it/pojo/PluginConfig.java b/it/src/main/java/it/pojo/PluginConfig.java index 4c6dbcf..2a50d43 100644 --- a/it/src/main/java/it/pojo/PluginConfig.java +++ b/it/src/main/java/it/pojo/PluginConfig.java @@ -1,5 +1,6 @@ package it.pojo; +// This class encapsulates configuration parameters for connecting to the local Ganache instance. public class PluginConfig { private String chain_id = "5777"; public String rpc_url = "http://ganache-cli:8545"; diff --git a/it/src/main/java/it/pojo/Property.java b/it/src/main/java/it/pojo/Property.java index 0547bbd..893e689 100644 --- a/it/src/main/java/it/pojo/Property.java +++ b/it/src/main/java/it/pojo/Property.java @@ -2,6 +2,7 @@ import lombok.Data; +// This class defines a generic data model for a property. @Data public class Property { diff --git a/it/src/main/java/it/pojo/TestAccount.java b/it/src/main/java/it/pojo/TestAccount.java index ea7dc6c..b8d4502 100644 --- a/it/src/main/java/it/pojo/TestAccount.java +++ b/it/src/main/java/it/pojo/TestAccount.java @@ -3,6 +3,8 @@ import lombok.Getter; import lombok.RequiredArgsConstructor; +/* This class models a test account with immutable fields + for the account’s name, mnemonic, and public key.*/ @Getter @RequiredArgsConstructor public class TestAccount { diff --git a/it/src/main/java/it/pojo/TxDetails.java b/it/src/main/java/it/pojo/TxDetails.java index cbc1674..c92d6dc 100644 --- a/it/src/main/java/it/pojo/TxDetails.java +++ b/it/src/main/java/it/pojo/TxDetails.java @@ -2,7 +2,7 @@ import lombok.Data; -// only stuff that we are interested in for the tests +// This class encapsulates essential information about a blockchain transaction. @Data public class TxDetails { diff --git a/it/src/main/java/it/pojo/VaultAccount.java b/it/src/main/java/it/pojo/VaultAccount.java index 1dd70b3..9f78d9c 100644 --- a/it/src/main/java/it/pojo/VaultAccount.java +++ b/it/src/main/java/it/pojo/VaultAccount.java @@ -4,6 +4,7 @@ import lombok.Data; import lombok.NoArgsConstructor; +// This class models an account with a mnemonic phrase @Data @AllArgsConstructor @NoArgsConstructor diff --git a/it/src/test/java/it/RunCucumberTest.java b/it/src/test/java/it/RunCucumberTest.java index a662254..92f4973 100644 --- a/it/src/test/java/it/RunCucumberTest.java +++ b/it/src/test/java/it/RunCucumberTest.java @@ -10,6 +10,7 @@ import static io.cucumber.junit.platform.engine.Constants.PLUGIN_PROPERTY_NAME; +// This class models a Vault plugin’s configuration @Suite @IncludeEngines("cucumber") @SelectClasspathResource("it") diff --git a/it/src/test/java/it/StepDefinitions.java b/it/src/test/java/it/StepDefinitions.java index fd8face..a294406 100644 --- a/it/src/test/java/it/StepDefinitions.java +++ b/it/src/test/java/it/StepDefinitions.java @@ -47,29 +47,30 @@ import static org.junit.jupiter.api.Assertions.fail; public class StepDefinitions { - - private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper(); - public static final String NGSILD_TENANT = "orion"; +/* This class is a comprehensive Cucumber step definition set for integration testing Canis Major, + and vault-based Ethereum signing in.*/ + private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper(); // JSON serialization/deserialization. + public static final String NGSILD_TENANT = "orion"; // NGSI-LD tenant header value. { OBJECT_MAPPER.setSerializationInclusion(JsonInclude.Include.NON_NULL); OBJECT_MAPPER.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false); } + // Canis Major and Vault service endpoints and the Vault root toke private static final String CANIS_MAJOR_ADDRESS = "127.0.0.1:4000"; // depending on the test setup, this can either go directly to canis-major or to a broker(and being intercepted) private static final String VAULT_ADDRESS = "127.0.0.1:8200"; - - private static final String VAULT_ROOT_TOKEN = "vault-plaintext-root-token"; + // Define test accounts with names, mnemonics, and public keys for use in test scenarios. private static final Map TEST_ACCOUNT_MAP = Map.of( "Default", new TestAccount("default", "label butter chaos blush mind north kit drill position phone decline urge claw mammal risk", "0xd9fe663797b75d0b3897d55d35e0b4e72307a63f"), "Franzi", new TestAccount("franzi", "minimum symptom minute gloom tragic situate silver mechanic salad amused elite beef", "0xa508dD875f10C33C52a8abb20E16fc68E981F186"), "Mira", new TestAccount("mira", "ridge bargain sight table never risk isolate hold jaguar reflect curve globe awake witness reveal", "0x34E5b3f990e55D0651B35c817bAfb89d2877cb95") ); - private static final int TX_AWAIT_MAX_S = 15; + private static final int TX_AWAIT_MAX_S = 15; // maximum wait time for transaction confirmatio // address to be used for accessing the broker. Practically a switch between the proxy-mode and direct canis-major access. private String ngsiAddress = "10.5.0.5:1026"; @@ -102,7 +103,7 @@ public void setup_canis_major_in_docker() throws Exception { .atMost(Duration.of(60, ChronoUnit.SECONDS)) .until(this::assertSystemIsRunning); } - + // Health check and plugin setup methods private boolean assertSuccess(Request request) { OkHttpClient okHttpClient = new OkHttpClient(); try { diff --git a/src/main/java/org/fiware/aeicontract/AlaDIDPubkeyResolver.java b/src/main/java/org/fiware/aeicontract/AlaDIDPubkeyResolver.java index 05852b5..6ac1c97 100644 --- a/src/main/java/org/fiware/aeicontract/AlaDIDPubkeyResolver.java +++ b/src/main/java/org/fiware/aeicontract/AlaDIDPubkeyResolver.java @@ -40,6 +40,7 @@ * *

Generated with web3j version 1.4.1. */ + @SuppressWarnings("rawtypes") public class AlaDIDPubkeyResolver extends Contract { public static final String BINARY = "Bin file was not provided"; @@ -71,6 +72,7 @@ public class AlaDIDPubkeyResolver extends Contract { ; @Deprecated + protected AlaDIDPubkeyResolver(String contractAddress, Web3j web3j, Credentials credentials, BigInteger gasPrice, BigInteger gasLimit) { super(BINARY, contractAddress, web3j, credentials, gasPrice, gasLimit); } diff --git a/src/main/java/org/fiware/canismajor/Application.java b/src/main/java/org/fiware/canismajor/Application.java index 7642c46..aea35f9 100644 --- a/src/main/java/org/fiware/canismajor/Application.java +++ b/src/main/java/org/fiware/canismajor/Application.java @@ -16,11 +16,10 @@ import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; -/** - * Base application as starting point - */ + @RequiredArgsConstructor @Factory +// This class is the main application class public class Application { public static void main(String[] args) { @@ -30,11 +29,13 @@ public static void main(String[] args) { private final EthereumProperties ethereumProperties; @Bean + // This method creates a blocking HTTP client public BlockingHttpClient blockingHttpClient() { return new DefaultHttpClient().toBlocking(); } @Bean + // This method creates an Ethereum client @Requires(property = "ethereum.enabled", value = "true") public Web3j ethereumClient() { return Web3j.build(new HttpService(ethereumProperties.getDltAddress().toString())); @@ -42,6 +43,7 @@ public Web3j ethereumClient() { @Bean + // This method creates a contract gas provider @Requires(property = "ethereum.enabled", value = "true") public ContractGasProvider contractGasProvider() { return new StaticGasProvider(ethereumProperties.getGasPrice(), ethereumProperties.getGas()); diff --git a/src/main/java/org/fiware/canismajor/configuration/DefaultAccountProperties.java b/src/main/java/org/fiware/canismajor/configuration/DefaultAccountProperties.java index 5802726..eefe8c9 100644 --- a/src/main/java/org/fiware/canismajor/configuration/DefaultAccountProperties.java +++ b/src/main/java/org/fiware/canismajor/configuration/DefaultAccountProperties.java @@ -3,6 +3,7 @@ import io.micronaut.context.annotation.ConfigurationProperties; import lombok.Data; +// This class holds configuration for enabling and supplying a default blockchain account @ConfigurationProperties("defaultAccount") @Data public class DefaultAccountProperties { diff --git a/src/main/java/org/fiware/canismajor/configuration/DefaultVaultProperties.java b/src/main/java/org/fiware/canismajor/configuration/DefaultVaultProperties.java index f65d6dc..2affc7c 100644 --- a/src/main/java/org/fiware/canismajor/configuration/DefaultVaultProperties.java +++ b/src/main/java/org/fiware/canismajor/configuration/DefaultVaultProperties.java @@ -5,6 +5,7 @@ import java.net.URL; +// This class holds configuration for enabling and supplying a default vault account @ConfigurationProperties("defaultVaultAccount") @Data public class DefaultVaultProperties { diff --git a/src/main/java/org/fiware/canismajor/configuration/EthereumProperties.java b/src/main/java/org/fiware/canismajor/configuration/EthereumProperties.java index f759520..fa1bf96 100644 --- a/src/main/java/org/fiware/canismajor/configuration/EthereumProperties.java +++ b/src/main/java/org/fiware/canismajor/configuration/EthereumProperties.java @@ -6,6 +6,7 @@ import java.math.BigInteger; import java.net.URL; +// This class holds configuration for Ethereum blockchain connection @ConfigurationProperties("ethereum") @Data public class EthereumProperties { diff --git a/src/main/java/org/fiware/canismajor/configuration/GeneralProperties.java b/src/main/java/org/fiware/canismajor/configuration/GeneralProperties.java index e05d4b7..0787c75 100644 --- a/src/main/java/org/fiware/canismajor/configuration/GeneralProperties.java +++ b/src/main/java/org/fiware/canismajor/configuration/GeneralProperties.java @@ -3,9 +3,7 @@ import io.micronaut.context.annotation.ConfigurationProperties; import lombok.Data; -/** - * Configuration of general properties - */ +// This class holds configuration for general properties @ConfigurationProperties("general") @Data public class GeneralProperties { diff --git a/src/main/java/org/fiware/canismajor/exception/AccountException.java b/src/main/java/org/fiware/canismajor/exception/AccountException.java index a0ce546..fcc4495 100644 --- a/src/main/java/org/fiware/canismajor/exception/AccountException.java +++ b/src/main/java/org/fiware/canismajor/exception/AccountException.java @@ -1,5 +1,6 @@ package org.fiware.canismajor.exception; +// This class is used to throw exceptions related to account operations public class AccountException extends RuntimeException { public AccountException(String message) { diff --git a/src/main/java/org/fiware/canismajor/exception/NGSIConnectException.java b/src/main/java/org/fiware/canismajor/exception/NGSIConnectException.java index 8e734ae..dad5de7 100644 --- a/src/main/java/org/fiware/canismajor/exception/NGSIConnectException.java +++ b/src/main/java/org/fiware/canismajor/exception/NGSIConnectException.java @@ -1,5 +1,6 @@ package org.fiware.canismajor.exception; +// This class is used to throw exceptions related to NGSI connection public class NGSIConnectException extends Exception { public NGSIConnectException(String message) { super(message); diff --git a/src/main/java/org/fiware/canismajor/exception/PersistenceException.java b/src/main/java/org/fiware/canismajor/exception/PersistenceException.java index 5029f2c..0ceac54 100644 --- a/src/main/java/org/fiware/canismajor/exception/PersistenceException.java +++ b/src/main/java/org/fiware/canismajor/exception/PersistenceException.java @@ -1,5 +1,6 @@ package org.fiware.canismajor.exception; +// This class is used to throw exceptions related to persistence operations public class PersistenceException extends RuntimeException { public PersistenceException(String message) { super(message); diff --git a/src/main/java/org/fiware/canismajor/exception/SigningException.java b/src/main/java/org/fiware/canismajor/exception/SigningException.java index 5ce8046..f3ca386 100644 --- a/src/main/java/org/fiware/canismajor/exception/SigningException.java +++ b/src/main/java/org/fiware/canismajor/exception/SigningException.java @@ -1,5 +1,6 @@ package org.fiware.canismajor.exception; +// This class is used to throw exceptions related to signing operations public class SigningException extends RuntimeException { public SigningException(String message) { diff --git a/src/main/java/org/fiware/canismajor/exception/TransactionException.java b/src/main/java/org/fiware/canismajor/exception/TransactionException.java index 3ff3050..0affdb7 100644 --- a/src/main/java/org/fiware/canismajor/exception/TransactionException.java +++ b/src/main/java/org/fiware/canismajor/exception/TransactionException.java @@ -1,5 +1,6 @@ package org.fiware.canismajor.exception; +// This class is used to throw exceptions related to transaction operations public class TransactionException extends Exception{ public TransactionException(String message) { super(message); diff --git a/src/main/java/org/fiware/canismajor/exception/VaultException.java b/src/main/java/org/fiware/canismajor/exception/VaultException.java index 56268a3..76d8a4a 100644 --- a/src/main/java/org/fiware/canismajor/exception/VaultException.java +++ b/src/main/java/org/fiware/canismajor/exception/VaultException.java @@ -1,5 +1,6 @@ package org.fiware.canismajor.exception; +// This class holds exceptions related to vault operations public class VaultException extends RuntimeException { public VaultException(String message) { diff --git a/src/main/java/org/fiware/canismajor/mapping/TransactionMapper.java b/src/main/java/org/fiware/canismajor/mapping/TransactionMapper.java index 7cb686e..05615d5 100644 --- a/src/main/java/org/fiware/canismajor/mapping/TransactionMapper.java +++ b/src/main/java/org/fiware/canismajor/mapping/TransactionMapper.java @@ -4,6 +4,7 @@ import org.mapstruct.Mapper; import org.web3j.crypto.RawTransaction; +// converting RawTransaction to a format suitable for Canis Major @Mapper(componentModel = "jsr330") public interface TransactionMapper { diff --git a/src/main/java/org/fiware/canismajor/mapping/TxReceiptMapper.java b/src/main/java/org/fiware/canismajor/mapping/TxReceiptMapper.java index 6535236..d9521f5 100644 --- a/src/main/java/org/fiware/canismajor/mapping/TxReceiptMapper.java +++ b/src/main/java/org/fiware/canismajor/mapping/TxReceiptMapper.java @@ -21,11 +21,13 @@ import java.util.Map; import java.util.Optional; +// Transform an Ethereum transaction receipt object into an NGSI-LD entity object (EntityVO) @Mapper(componentModel = "jsr330") public interface TxReceiptMapper { ObjectMapper OBJECT_MAPPER = new ObjectMapper(); + // NGSI-LD entity configuration // the transaction index will be appended as a id String ID_TEMPLATE = "urn:ngsi-ld:dlttxreceipt:%s"; String ENTITY_TYPE = "DLTtxReceipt"; @@ -34,6 +36,7 @@ public interface TxReceiptMapper { String REF_ENTITY_KEY = "refEntity"; String RETRIEVAL_QUERY_KEY = "retrievalQuery"; + // mapping the transaction receipt to an NGSI-LD entity default EntityVO transactionReceiptToEntityVO(TransactionReceipt transactionReceipt, URI entityId, RetrievalQueryInfo queryInfo) { EntityVO entityVO = transactionReceiptToEntityVO(transactionReceipt, entityId); @@ -45,6 +48,7 @@ default EntityVO transactionReceiptToEntityVO(TransactionReceipt transactionRece return entityVO; } + default EntityVO transactionReceiptToEntityVO(TransactionReceipt transactionReceipt, URI entityId, QueryInfo queryInfo) { EntityVO entityVO = transactionReceiptToEntityVO(transactionReceipt, entityId); diff --git a/src/main/java/org/fiware/canismajor/repository/EntityRepository.java b/src/main/java/org/fiware/canismajor/repository/EntityRepository.java index eca3c38..9dc2f90 100644 --- a/src/main/java/org/fiware/canismajor/repository/EntityRepository.java +++ b/src/main/java/org/fiware/canismajor/repository/EntityRepository.java @@ -28,6 +28,7 @@ @Slf4j @Singleton @RequiredArgsConstructor +// This class is used to retrieve entities with transactions from the NGSI-LD context broker public class EntityRepository { private static final String REF_QUERY_TEMPLATE = TxReceiptMapper.REF_ENTITY_KEY + "==\"%s\""; @@ -68,6 +69,7 @@ public EntityTransactionListVO getEntitiesWithTransaction() throws NGSIConnectEx return entityTransactionVOS; } + // This method extracts the transaction receipts from the entityVO private Map> extractReceipts(EntityVO entityVO) { return txReceiptMapper.getEntityIdsFromTX(entityVO).stream().collect(Collectors.toMap( id -> id, @@ -83,6 +85,7 @@ private Map> extractReceipts(EntityVO entityV )); } + // This method retrieves the transaction details for a specific entity public EntityTransactionVO getEntityTransactions(URI entityId) throws NGSIConnectException { Optional optionalEntityListVO = executeRequest( () -> apiClient.queryEntities(generalProperties.getNgsiTenant(), null, null, TxReceiptMapper.ENTITY_TYPE, null, String.format(REF_QUERY_TEMPLATE, entityId), null, null, null, null, null, null, null, getLinkHeader())); diff --git a/src/main/java/org/fiware/canismajor/rest/EntitiesController.java b/src/main/java/org/fiware/canismajor/rest/EntitiesController.java index dda1283..6dc40f5 100644 --- a/src/main/java/org/fiware/canismajor/rest/EntitiesController.java +++ b/src/main/java/org/fiware/canismajor/rest/EntitiesController.java @@ -18,11 +18,13 @@ @Slf4j @Controller @RequiredArgsConstructor +// This class is used to retrieve entities with transactions from the NGSI-LD context broker public class EntitiesController implements EntityApi { private final EntityRepository entityRepository; @Override + // This method retrieves all entities with transactions from the NGSI-LD context broker public HttpResponse getEntitiesWithTransactions() { try { return HttpResponse.ok(entityRepository.getEntitiesWithTransaction()); @@ -32,6 +34,7 @@ public HttpResponse getEntitiesWithTransactions() { } @Override + // This method retrieves a specific entity with transactions from the NGSI-LD context broker public HttpResponse getEntityWithTransactions(URI entityId) { try { return HttpResponse.ok(entityRepository.getEntityTransactions(entityId)); diff --git a/src/main/java/org/fiware/canismajor/rest/NGSILDController.java b/src/main/java/org/fiware/canismajor/rest/NGSILDController.java index b844329..308be28 100644 --- a/src/main/java/org/fiware/canismajor/rest/NGSILDController.java +++ b/src/main/java/org/fiware/canismajor/rest/NGSILDController.java @@ -32,6 +32,7 @@ @Slf4j @Controller @RequiredArgsConstructor +// This class is used to create, update, query, retrieve and upsert entities in the NGSI-LD context broker public class NGSILDController implements NgsiLdApi { private final EthereumService ethereumService; @@ -42,6 +43,7 @@ public class NGSILDController implements NgsiLdApi { private final GeneralProperties generalProperties; @Override + // This method creates a new NGSI-LD entity public HttpResponse createNgsiLDEntity(@Nullable String link, @Nullable String walletType, @Nullable String walletToken, @Nullable String walletAddress, EntityVO entityVO) { try { TransactionReceipt transactionReceipt = ethereumService.persistEntityCreation(entityVO, toWalletInformation(walletType, walletToken, walletAddress)); @@ -54,6 +56,7 @@ public HttpResponse createNgsiLDEntity(@Nullable String li } @Override + // This method updates an existing NGSI-LD entity public HttpResponse postUpdateNgsiLDEntity(URI entityId, @Nullable String link, @Nullable String walletType, @Nullable String walletToken, @Nullable String walletAddress, EntityFragmentVO entityFragmentVO) { try { TransactionReceipt transactionReceipt = ethereumService.persistEntityUpdate(entityId, entityFragmentVO, toWalletInformation(walletType, walletToken, walletAddress)); @@ -66,6 +69,7 @@ public HttpResponse postUpdateNgsiLDEntity(URI entityId, @ } @Override + // This method queries NGSI-LD entities public HttpResponse queryEntities(@Nullable String id, @Nullable String idPattern, @Nullable String type, @Nullable String attrs, @Nullable String q, @Nullable String georel, @Nullable String geometry, @Nullable String coordinates, @Nullable String geoproperty, @Nullable String csf, @Nullable Integer limit, @Nullable Integer offset, @Nullable String options, @Nullable String link, @Nullable String walletType, @Nullable String walletToken, @Nullable String walletAddress, @Nullable URI relatedEntity) { QueryInfo queryInfo = new QueryInfo(id, idPattern, type, attrs, q, georel, geometry, coordinates, geoproperty, csf, limit, offset, options, link); // if the request does not provide information about an entity to related to, we are using a generic default. @@ -82,6 +86,7 @@ public HttpResponse queryEntities(@Nullable String id, @Nu } @Override + // This method retrieves an NGSI-LD entity by its ID public HttpResponse retrieveEntityById(URI entityId, @Nullable String attrs, @Nullable String type, @Nullable String options, @Nullable String link, @Nullable String walletType, @Nullable String walletToken, @Nullable String walletAddress) { RetrievalQueryInfo retrievalQueryInfo = new RetrievalQueryInfo(entityId, attrs, type, options, link); try { @@ -95,6 +100,7 @@ public HttpResponse retrieveEntityById(URI entityId, @Null } @Override + // This method upserts NGSI-LD entities public HttpResponse upsertEntities(@Nullable String walletType, @Nullable String walletToken, @Nullable String walletAddress, List entityVOs) { try { TransactionReceipt transactionReceipt = ethereumService.persistBatchOperation(entityVOs, toWalletInformation(walletType, walletToken, walletAddress)); @@ -108,6 +114,7 @@ public HttpResponse upsertEntities(@Nullable String wallet } } + // This method converts the wallet information to a WalletInformation object private WalletInformation toWalletInformation(String walletType, String walletToken, String walletAddress) { try { Optional optionalWalletAddress = Optional.ofNullable(walletAddress); From 58c69ad0dd0b35b3c4f13c336a887273f4487d73 Mon Sep 17 00:00:00 2001 From: asmataamallah25 <168739895+asmataamallah25@users.noreply.github.com> Date: Fri, 20 Jun 2025 11:43:47 +0200 Subject: [PATCH 16/16] Update canis_major_technical_overview.md --- docs/canis_major_technical_overview.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/canis_major_technical_overview.md b/docs/canis_major_technical_overview.md index b1fdc63..e66f7a6 100644 --- a/docs/canis_major_technical_overview.md +++ b/docs/canis_major_technical_overview.md @@ -168,7 +168,7 @@ To assist with understanding the structure and significance of the transcation r 6. **from** - Type: String (hexadecimal) - -Description: The address of the account that initiated the transaction. + - Description: The address of the account that initiated the transaction. - Value: "0x34e5b3f990e55d0651b35c817bafb89d2877cb95" 7. **to** @@ -253,6 +253,7 @@ Key Points: Uses a customized image (vault-ethereum) for Ethereum-related use cases. - Environment Variables: VAULT_ADDR: Specifies where Vault listens (http://0.0.0.0:8200). + VAULT_DEV_ROOT_TOKEN_ID: Sets a development root token (vault-plaintext-root-token) for authentication. - Security: Adds IPC_LOCK capability for enhanced security by preventing memory swapping.