diff --git a/.gitbook/assets/applayer_step1.mp4 b/.gitbook/assets/applayer_step1.mp4 deleted file mode 100644 index e163c74..0000000 Binary files a/.gitbook/assets/applayer_step1.mp4 and /dev/null differ diff --git a/.gitbook/assets/applayer_step2.mp4 b/.gitbook/assets/applayer_step2.mp4 deleted file mode 100644 index b9c35f5..0000000 Binary files a/.gitbook/assets/applayer_step2.mp4 and /dev/null differ diff --git a/.gitbook/assets/applayer_step3.mp4 b/.gitbook/assets/applayer_step3.mp4 deleted file mode 100644 index 3507ee4..0000000 Binary files a/.gitbook/assets/applayer_step3.mp4 and /dev/null differ diff --git a/.gitbook/assets/applayer_step4.mp4 b/.gitbook/assets/applayer_step4.mp4 deleted file mode 100644 index 5c3c219..0000000 Binary files a/.gitbook/assets/applayer_step4.mp4 and /dev/null differ diff --git a/.gitbook/assets/applayer_step5.mp4 b/.gitbook/assets/applayer_step5.mp4 deleted file mode 100644 index fabd603..0000000 Binary files a/.gitbook/assets/applayer_step5.mp4 and /dev/null differ diff --git a/.gitbook/assets/mmwallet1.png b/.gitbook/assets/mmwallet1.png new file mode 100644 index 0000000..217cb0b Binary files /dev/null and b/.gitbook/assets/mmwallet1.png differ diff --git a/.gitbook/assets/mmwallet2.png b/.gitbook/assets/mmwallet2.png new file mode 100644 index 0000000..4f91583 Binary files /dev/null and b/.gitbook/assets/mmwallet2.png differ diff --git a/.gitbook/assets/mmwallet3.png b/.gitbook/assets/mmwallet3.png new file mode 100644 index 0000000..5498dfc Binary files /dev/null and b/.gitbook/assets/mmwallet3.png differ diff --git a/.gitbook/assets/mmwallet4.png b/.gitbook/assets/mmwallet4.png new file mode 100644 index 0000000..9078bb5 Binary files /dev/null and b/.gitbook/assets/mmwallet4.png differ diff --git a/.gitbook/assets/mmwallet5.png b/.gitbook/assets/mmwallet5.png new file mode 100644 index 0000000..4746a3d Binary files /dev/null and b/.gitbook/assets/mmwallet5.png differ diff --git a/.gitbook/assets/mmwallet6.png b/.gitbook/assets/mmwallet6.png new file mode 100644 index 0000000..c08f762 Binary files /dev/null and b/.gitbook/assets/mmwallet6.png differ diff --git a/.gitbook/assets/utils-folder.png b/.gitbook/assets/utils-folder.png deleted file mode 100644 index 784d508..0000000 Binary files a/.gitbook/assets/utils-folder.png and /dev/null differ diff --git a/README.md b/README.md index 2038ed8..f40f107 100644 --- a/README.md +++ b/README.md @@ -22,4 +22,4 @@ layout: Welcome to the central hub for AppLayer documentation. This repository contains information on all the core components of the AppLayer network, such as design and architecture details, how the Blockchain Development Kit (BDK) works, a glossary of terms, community resources, documentation, developer guides, code snippets and more. -The reader should be substantially familiar with Unified Modeling Language (UML), as some sections contain diagrams that further elaborate the concepts explained. +The reader should be substantially familiar with Unified Modeling Language (UML), as some sections contain diagrams that further elaborate the concepts explained in them. diff --git a/SUMMARY.md b/SUMMARY.md index 4c40a65..9f0b62b 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -2,57 +2,52 @@ * [Welcome to AppLayer Docs](README.md) * [Introducing AppLayer](introducing-applayer/README.md) - * [A Primer on Smart Contracts](introducing-applayer/a-primer-on-smart-contracts.md) - * [The Problem With EVMs](introducing-applayer/the-problem-with-evms.md) - * [What is AppLayer?](introducing-applayer/what-is-applayer.md) -* [How AppLayer works](how-applayer-works/README.md) - * [Validators](how-applayer-works/validators.md) - * [Sentinels](how-applayer-works/sentinels.md) - * [Application Chains](how-applayer-works/appchains.md) - * [Bridging](how-applayer-works/bridging/README.md) - * [AppLayer-to-AppLayer Data Bridging](how-applayer-works/bridging/applayer-to-applayer-data-bridging.md) - * [AppLayer-to-AppLayer Token Bridging](how-applayer-works/bridging/applayer-to-applayer-token-bridging.md) - * [AppLayer-to-External Bridging (Ethereum, Solana, etc.)](how-applayer-works/bridging/applayer-to-external-bridging.md) -* [Understanding rdPoS](understanding-rdpos/README.md) - * [Blockchains overview](understanding-rdpos/blockchains-overview.md) - * [How rdPoS works](understanding-rdpos/how-rdpos-works.md) - * [Validator implementations](understanding-rdpos/validator-implementations.md) - * [Slashing](understanding-rdpos/slashing.md) + * [A Primer on Smart Contracts and EVMs](introducing-applayer/a-primer-on-smart-contracts-and-evms.md) + * [Enter AppLayer](introducing-applayer/enter-applayer.md) +* [Getting started](getting-started/README.md) + * [Setting up a wallet](getting-started/setting-up-a-wallet.md) + * [AppLayer Testnet](getting-started/applayer-testnet/README.md) + * [Deploying contracts](getting-started/applayer-testnet/deploying-contracts.md) + * [Developing with BDK](getting-started/developing-with-bdk/README.md) + * [Environment setup](getting-started/developing-with-bdk/environment-setup.md) + * [Compilation and deployment](getting-started/developing-with-bdk/compilation-and-deployment.md) + * [Connecting the wallet](getting-started/developing-with-bdk/connecting-the-wallet.md) + * [Contract Tester](getting-started/developing-with-bdk/contract-tester.md) +* [Contracts](contracts/README.md) + * [Solidity ABI](contracts/solidity-abi.md) + * [Internal and external contract calls](contracts/internal-and-external-contract-calls.md) + * [Precompiled contracts](contracts/precompiled-contracts/README.md) + * [Dynamic and Protocol Contracts](contracts/precompiled-contracts/dynamic-and-protocol-contracts.md) + * [Dynamic Contract Templates](contracts/precompiled-contracts/dynamic-contract-templates.md) + * [Managing precompiled contracts](contracts/precompiled-contracts/managing-precompiled-contracts.md) + * [SafeVariables and commit/revert logic](contracts/precompiled-contracts/safevariables-and-commit-revert-logic.md) + * [How to code a precompiled contract](contracts/precompiled-contracts/how-to-code-a-precompiled-contract.md) + * [Creating a Dynamic Contract (Simple)](contracts/precompiled-contracts/creating-a-dynamic-contract-simple/README.md) + * [Simple Contract Header](contracts/precompiled-contracts/creating-a-dynamic-contract-simple/simple-contract-header.md) + * [Simple Contract Source](contracts/precompiled-contracts/creating-a-dynamic-contract-simple/simple-contract-source.md) + * [Deploying and testing](contracts/precompiled-contracts/creating-a-dynamic-contract-simple/deploying-and-testing.md) + * [Creating a Dynamic Contract (Advanced)](contracts/precompiled-contracts/creating-a-dynamic-contract-advanced.md) + * [Creating a Protocol Contract](contracts/precompiled-contracts/creating-a-protocol-contract.md) + * [EVM contracts](contracts/evm-contracts/README.md) + * [Handling contract calls](contracts/evm-contracts/handling-contract-calls.md) + * [Executing contract calls via EVMC](contracts/evm-contracts/executing-contract-calls-via-evmc.md) + * [Calling contracts](contracts/evm-contracts/calling-contracts.md) * [BDK implementation](bdk-implementation/README.md) - * [The utils folder](bdk-implementation/the-utils-folder.md) - * [The contract folder](bdk-implementation/the-contract-folder.md) - * [The core folder](bdk-implementation/the-core-folder.md) - * [Transactions and Blocks](bdk-implementation/transactions-and-blocks.md) + * [Source code overview](bdk-implementation/source-code-overview.md) + * [Transactions and blocks](bdk-implementation/transactions-and-blocks.md) * [Database](bdk-implementation/database.md) * [Contract call handling](bdk-implementation/contract-call-handling.md) * [RLP (Recursive-Length Prefix)](bdk-implementation/rlp-recursive-length-prefix.md) * [P2P Overview](bdk-implementation/p2p-overview.md) * [P2P Encoding](bdk-implementation/p2p-encoding.md) -* [Understanding contracts](understanding-contracts/README.md) - * [Solidity ABI](understanding-contracts/solidity-abi.md) - * [Internal and external contract calls](understanding-contracts/internal-and-external-contract-calls.md) - * [Setting up the development environment](understanding-contracts/setting-up-the-development-environment.md) - * [Contract Tester](understanding-contracts/contract-tester.md) -* [Precompiled contracts](precompiled-contracts/README.md) - * [Types of pre-compiled contracts](precompiled-contracts/types-of-precompiled-contracts.md) - * [Dynamic and Protocol Contracts](precompiled-contracts/dynamic-and-protocol-contracts.md) - * [SafeVariables and commit/revert logic](precompiled-contracts/safevariables-and-commit-revert-logic.md) - * [How to code a precompiled contract](precompiled-contracts/how-to-code-a-precompiled-contract.md) - * [Creating a Dynamic Contract (Simple)](precompiled-contracts/creating-a-dynamic-contract-simple/README.md) - * [Simple Contract Header](precompiled-contracts/creating-a-dynamic-contract-simple/simple-contract-header.md) - * [Simple Contract Source](precompiled-contracts/creating-a-dynamic-contract-simple/simple-contract-source.md) - * [Deploying and testing](precompiled-contracts/creating-a-dynamic-contract-simple/deploying-and-testing.md) - * [Creating a Dynamic Contract (Advanced)](precompiled-contracts/creating-a-dynamic-contract-advanced.md) - * [Creating a Protocol Contract (Advanced)](precompiled-contracts/creating-a-protocol-contract-advanced.md) -* [EVM contracts](evm-contracts/README.md) - * [State management and VM instance creation](evm-contracts/state-management-and-vm-instance-creation.md) - * [Seamless C++/EVM integration](evm-contracts/seamless-cpp-evm-integration.md) - * [C++ to other contract calls](evm-contracts/cpp-to-other-contract-calls.md) - * [EVM to other contract calls](evm-contracts/evm-to-other-contract-calls.md) - * [Executing contract calls via EVMC](evm-contracts/executing-contract-calls-via-evmc.md) - * [Calling EVM contracts from C++](evm-contracts/calling-evm-contracts-from-cpp.md) - * [Calling C++ contracts from EVM](evm-contracts/calling-cpp-contracts-from-evm.md) -* [Getting started with AppLayer Testnet](getting-started-with-applayer-testnet.md) +* [Future Plans](future-plans/README.md) + * [Sentinels](future-plans/sentinels.md) + * [Application Chains](future-plans/appchains.md) + * [Bridging](future-plans/bridging/README.md) + * [AppLayer-to-AppLayer Data Bridging](future-plans/bridging/applayer-to-applayer-data-bridging.md) + * [AppLayer-to-AppLayer Token Bridging](future-plans/bridging/applayer-to-applayer-token-bridging.md) + * [AppLayer-to-External Bridging (Ethereum, Solana, etc.)](future-plans/bridging/applayer-to-external-bridging.md) + * [rdPoS](future-plans/rdpos.md) * [Join our Community](join-our-community.md) * [Get in Touch](get-in-touch.md) * [Glossary](glossary.md) diff --git a/bdk-implementation/README.md b/bdk-implementation/README.md index 2a88614..c19d4f2 100644 --- a/bdk-implementation/README.md +++ b/bdk-implementation/README.md @@ -1,135 +1,13 @@ ---- -description: How the functional elements of AppLayer interact with each other. ---- - -# Blockchain Development Kit (BDK) implementation - -This chapter aims to explain in technical detail how the BDK is implemented, as well as its submodules and how everything comes together to deliver a blazing fast blockchain. - -The first few subchapters paint a more holistic view of the BDK, as most components are pretty straight-forward to understand. Developers are expected to use the [Doxygen](https://doxygen.nl) documentation as a reference to further understand how the project works. The later subchapters show some components that are particularly denser and/or complex enough that they warrant their own separated explanations. - -Looking at a higher level of abstraction, the original C++ implementation of the BDK is structured like this: - -* The `src/bins` folder contains the source files for the project's main executables - the blockchain executable itself, contract ABI generator, network simulator, faucet API and other testing-related executables are all coded here in their respective subfolders -* The `src/bytes` folder contains code related to the `bytes` class, a container that deals with raw bytes - specifically the `bytes::join()` function and the `bytes::View` class, both used extensively across the project -* The `src/contract` folder contains everything related to the logic of smart contracts - from ABI parsing to custom variable types and template contracts -* The `src/core` folder contains the heart of the BDK - the main components of the blockchain and what makes it tick -* The `src/libs` folder contains third-party libraries not inherently tied to the project but used throughout development -* The `src/net` folder contains everything related to networking, communication between nodes and support for protocols such as gRPC, HTTP, P2P, JSON-RPC, etc. -* The `src/utils` folder contains several commonly-used functions, structures, classes and overall logic to support the functioning of the BDK as a whole - -There is also a `tests` folder that contains several unit tests for each of the components described above. - -## Source tree - -For the more visually inclined, here is a source tree (headers only) containing all of the files inside the `src` folder (except `src/bins` as it only contains source files), their respective subfolders and which components are declared in them. Each component is further explained through the following subchapters of this documentation. For more technical details (e.g. API references for developers), please refer to the [Doxygen](https://www.doxygen.nl) documentation on the project's own repository. - -``` -src -├── bytes -│   ├── initializer.h (bytes::Initializer, bytes::SizedInitializer) -│   ├── join.h (bytes::join()) -│   ├── range.h (bytes::Range, bytes::DataRange, bytes::BorrowedDataRange) -│   └── view.h (bytes::View, bytes::Span) -├── contract (Contracts) -│   ├── abi.h (ABI - encoders, decoders, helper structs, etc.) -│   ├── calltracer.h (trace namespace, Call struct, CallTracer class) -│   ├── contractfactory.h (ContractFactory) -│   ├── contract.h (ContractGlobals, ContractLocals, BaseContract) -│   ├── contracthost.h (ContractHost) -│   ├── contractmanager.h (ContractManager) -│   ├── contractstack.h (ContractStack) -│   ├── customcontracts.h (for declaring custom contracts) -│   ├── dynamiccontract.h (DynamicContract) -│   ├── event.h (Event) -│   ├── templates (folder for contract templates) -│   │ ├── dexv2 (subfolder for the DEXV2 contract components) -│   │ │ ├── dexv2factory.h (DEXV2Factory) -│   │ │ ├── dexv2library.h (DEXV2Library) -│   │ │ ├── dexv2pair.h (DEXV2Pair) -│   │ │ ├── dexv2router02.h (DEXV2Router02) -│   │ │ └── uq112x112.h (UQ112x112 - used in DEX contracts for fixed-point fractions) -│   │ ├── erc20.h (ERC20) -│   │ ├── erc20wrapper.h (ERC20Wrapper) -│   │ ├── erc721.h (ERC721) -│   │ ├── erc721test.h (ERC721Test, used solely for testing purposes) -│   │   ├── erc721uristorage.h (ERC721URIStorage, converted from OpenZeppelin) -│   │ ├── nativewrapper.h (NativeWrapper) -│   │   ├── ownable.h (Ownable, converted from OpenZeppelin) -│   │   ├── pebble.h (Pebble) -│   │ ├── randomnesstest.h (RandomnessTest) -│   │ ├── simplecontract.h (SimpleContract) -│   │   ├── snailtracer.h, snailtraceroptimized.h (SnailTracer and SnailTracerOptimized, converted from the original EVM impl) -│   │ ├── testThrowVars.h (TestThrowVars, used solely for testing purposes) -│   │ └── throwtestA.h, throwtestB.h, throwtestC.h (for testing nested contract calls) -│   └── variables (Safe Variables for use within Dynamic Contracts) -│   ├── reentrancyguard.h (ReentrancyGuard) -│   ├── safeaddress.h (SafeAddress) -│   ├── safearray.h (SafeArray) -│   ├── safebase.h (SafeBase - used as base for all other types) -│   ├── safebool.h (SafeBool) -│   ├── safebytes.h (SafeBytes) -│   ├── safeint.h (SafeInt and respective aliases) -│   ├── safestring.h (SafeString) -│   ├── safetuple.h (SafeTuple) -│   ├── safeuint.h (SafeUint and respective aliases) -│   ├── safeunorderedmap.h (SafeUnorderedMap) -│   └── safevector.h (SafeVector) -├── core (Core components) -│   ├── blockchain.h (Blockchain, Syncer) -│   ├── consensus.h (Consensus) -│   ├── dump.h (Dumpable, DumpManager, DumpWorker) -│   ├── rdpos.h (Validator, rdPoS) -│   ├── state.h (BlockValidationStatus, State) -│   └── storage.h (Storage) -├── libs (Third-party libraries) -│ ├── BS_thread_pool_light.hpp (https://github.com/bshoshany/thread-pool) -│ ├── catch2/catch_amalgamated.hpp (https://github.com/catchorg/Catch2) -│ ├── json.hpp (https://github.com/nlohmann/json) -│ ├── wyhash.h (https://github.com/wangyi-fudan/wyhash) -│ └── zpp_bits.h (https://github.com/eyalz800/zpp_bits) -├── net (Networking) -│   ├── http (HTTP part of networking) -│   │   ├── httpclient.h (HTTPClient) -│   │   ├── httplistener.h (HTTPListener) -│   │   ├── httpparser.h (parser functions for HTTP requests) -│   │   ├── httpserver.h (HTTPServer) -│   │   ├── httpsession.h (HTTPQueue, HTTPSession) -│   │   └── jsonrpc (Namespace for handling JSONRPC data) -│   │   ├── blocktag.h (BlockTag, BlockTagOrNumber) -│   │   ├── call.h (call() - a function that processes a JSON RPC call) -│   │   ├── error.h (Error - for abstracting JSON RPC errors) -│   │   ├── methods.h (contains all implemented JSON RPC methods) -│   │   ├── parser.h (Parser and helper tempate functions) -│   │   └── variadicparser.h (VariadicParser and helper template functions) -│   └── p2p (P2P part of networking) -│   ├── broadcaster.h (Broadcaster) -│   ├── discovery.h (DiscoveryWorker - worker thread for ManagerDiscovery) -│   ├── encoding.h (collection of enums, structs, classes, encoders and decoders used in P2P communications) -│   ├── managerbase.h (ManagerBase - used as base for ManagerDiscovery and ManagerNormal) -│   ├── managerdiscovery.h (ManagerDiscovery) -│   ├── managernormal.h (ManagerNormal) -│   ├── nodeconns.h (NodeConns) -│   └── session.h (Session) -└── utils (Utility components) - ├── clargs.h (definitions for helper functions, enums and structs that deal with command-line argument parsing) - ├── contractreflectioninterface.h (ContractReflectionInterface - interface for registering Dynamic Contracts) - ├── db.h (DBPrefix, DBServer, DBEntry, DBBatch, DB) - ├── dynamicexception.h (DynamicException - custom exception class) - ├── ecdsa.h (PrivKey, Pubkey, UPubkey, Secp256k1) - ├── evmcconv.h (EVMCConv - namespace for EVMC-related data conversion functions) - ├── finalizedblock.h (FinalizedBlock) - ├── hex.h (Hex) - ├── intconv.h (IntConv - namespace for signed integer aliases and data conversion functions) - ├── jsonabi.h (JsonAbi - namespace for writing contract ABIs to JSON format) - ├── logger.h (LogType, Log, LogInfo, Logger, LogicalLocationProvider) - ├── merkle.h (Merkle) - ├── options.h (Options singleton - generated by CMake through a .in file) - ├── randomgen.h (RandomGen) - ├── safehash.h (SafeHash, FNVHash) - ├── strconv.h (StrConv - namespace for string-related data conversion and manipulation functions) - ├── strings.h (FixedBytes and its derivatives - Hash, Functor, Signature, Address, StorageKey) - ├── tx.h (TxBlock, TxValidator) - ├── uintconv.h (UintConv - namespace for unsigned integer aliases and data conversion functions) - └── utils.h (Utils namespace and other misc struct and enum definitions) -``` +--- +description: How the functional elements of AppLayer interact with each other +--- + +# BDK implementation + +This chapter aims to explain how the BDK and some of its most important components are implemented from a more conceptual point-of-view, giving an initial idea on how everything comes together to deliver a blazing fast blockchain. + +Some subchapters paint a more holistic view of the BDK, as most components are pretty straight-forward to understand. Other subchapters focus on some components that are particularly dense and/or complex enough that they warrant their own separated explanations. + +Developers are expected to read the [Doxygen](https://doxygen.nl) documentation alongside this one to further understand how the project works from a more technical point-of-view - as it is constantly being iterated upon, we make extensive use of comments and self-documenting code to keep up with the internals. + +We will focus on the original implementation of the BDK (the C++ one, commonly referred to as "bdk-cpp"). Future implementations, if/when they appear, may be handled separately. \ No newline at end of file diff --git a/bdk-implementation/contract-call-handling.md b/bdk-implementation/contract-call-handling.md index 1770804..c17a583 100644 --- a/bdk-implementation/contract-call-handling.md +++ b/bdk-implementation/contract-call-handling.md @@ -1,53 +1,13 @@ --- -description: How Applayer's BDK handles chains of contract execution. +description: How the BDK handles chains of contract execution --- # Contract call handling -Each time a transaction or contract execution alters any state variables—such as creating a new contract, updating a variable, or initiating transfers—those changes are *not* set directly at the state, but rather kept track of in a separate location. This is crucially important, as contracts can call other contracts, and nested calls can prolong themselves to a point you have dozens if not hundreds of variables with changed values, or several transactions that were made that must be reverted because the call failed. +Each time a transaction or contract execution alters any state variables - such as creating a new contract, updating a variable, or initiating transfers for example - those changes are *not* set directly at the state, but rather kept track of in a separate location. This is crucially important, as contracts can call other contracts, and nested calls can prolong themselves to a point you have dozens if not hundreds of variables with changed values, or several transactions that were made that must be reverted because the call failed at a later point in execution. -This is done within `ContractHost` by means of `ContractStack`. The class maintains a record of the original states of these variables, and registers changes made to them during a contract call. This record-keeping is essential for enabling a complete restoration of the original state in the event of a transaction rollback, ensuring that any adverse changes can be undone, safeguarding the blockchain's consistency, reliability and integrity. +Tthis is done within `ContractHost` by means of the `ContractStack` class, as stated previously in the "Precompiled contracts > Managing precompiled contracts" subsection (although it equally applies to EVM contracts). The class maintains a record of the original states of these variables, and registers changes made to them during a contract call. This record-keeping is essential for enabling a complete restoration of the original state in the event of a transaction rollback, ensuring that any adverse changes can be undone, safeguarding the blockchain's consistency, reliability and integrity. -For example, a call with a variable that starts with the value "10", then five calls down the line it throws for some reason, but the original value was altered mid-way to "50". The role of `ContractStack` in this situation is to revert that variable and *all* the other variables that were changed along the way back to their original values *before* the call was made (so our variable would have to go back to "10"), ensuring the state is not left with any stale or wrong data because of the failed call. Likewise, if the call is successful, `ContractStack` then applies all changes in order at once, in an atomic fashion, ensuring the state now has up-to-date data from after the contract's execution. - -## ContractStack class overview - -Here's an overview of the `ContractStack` class definition and functionalities (comments removed for easier reading, check the `contract/contractstack.h` file for more details): - -```c++ -class ContractStack { - private: - boost::unordered_flat_map code_; - boost::unordered_flat_map balance_; - boost::unordered_flat_map nonce_; - boost::unordered_flat_map storage_; - std::vector events_; - std::vector> contracts_; - std::vector> usedVars_; - - public: - inline void registerCode(const Address& addr, const Bytes& code) { this->code_.try_emplace(addr, code); } - - inline void registerBalance(const Address& addr, const uint256_t& balance) { this->balance_.try_emplace(addr, balance); } - - inline void registerNonce(const Address& addr, const uint64_t& nonce) { this->nonce_.try_emplace(addr, nonce); } - - inline void registerStorageChange(const StorageKey& key, const Hash& value) { this->storage_.try_emplace(key, value); } - - inline void registerEvent(Event event) { this->events_.emplace_back(std::move(event)); } - - inline void registerContract(const Address& addr, BaseContract* contract) { this->contracts_.emplace_back(addr, contract); } - - inline void registerVariableUse(SafeBase& var) { this->usedVars_.emplace_back(var); } - - inline const boost::unordered_flat_map& getCode() const { return this->code_; } - inline const boost::unordered_flat_map& getBalance() const { return this->balance_; } - inline const boost::unordered_flat_map& getNonce() const { return this->nonce_; } - inline const boost::unordered_flat_map& getStorage() const { return this->storage_; } - inline std::vector& getEvents() { return this->events_; } - inline const std::vector>& getContracts() const { return this->contracts_; } - inline const std::vector>& getUsedVars() const { return this->usedVars_; } -}; -``` +For example, a call with a variable that starts with the value "10", then five calls down the line it throws for some reason, but the original value was altered mid-way to "50". The role of `ContractStack` in this situation is to revert that variable and *all* the other variables that were changed along the way back to their original values *before* the start of the call (so our variable would have to go back to "10"), ensuring the state is not left with any stale or wrong data because of the failed call chain. Likewise, if the call is successful, `ContractStack` then applies all changes in order at once, in an atomic fashion, ensuring the state now has up-to-date data from after the contract's ensured execution. The existence of only *one* instance of `ContractStack` per `ContractHost`, as well as its integration within the RAII framework of `ContractHost`, guarantees that state values are meticulously committed or reverted upon the completion or rollback of transactions. This robust design prevents state spill-over between different contract executions, fortifying transaction isolation and integrity across the blockchain network - even in the dynamic and mutable landscape of blockchain transactions, the integrity and consistency of state changes are meticulously maintained, safeguarding against unintended consequences and errors during contract execution. diff --git a/bdk-implementation/database.md b/bdk-implementation/database.md index 5229a30..622dfdf 100644 --- a/bdk-implementation/database.md +++ b/bdk-implementation/database.md @@ -1,52 +1,48 @@ --- -description: How Blockchain Development Kit's (BDK) internal database is structured and how data is stored in it. +description: How the BDK's internal database is structured and how data is stored in it --- # Database -BDK validators use an in-disk database for storing data about themselves and other nodes in the network, such as block/transaction data, contract data, node metadata, etc. Depending on the component (e.g. Storage), they might have their own database folder reserved just for them. +BDK nodes use an in-disk database for storing data about themselves and other nodes in the network, such as blocks, transactions, contracts, node metadata, etc. Depending on the component, they might have their own database folder reserved just for them. -The database itself is an abstraction of a [Speedb](https://github.com/speedb-io/speedb) database - a simple key/value database, but handled in a different way: keys use *prefixes*, which makes it possible to batch read and write, so we can get around the "simple key/value" limitation and divide data into sectors. +The database itself is an abstraction of a [Speedb](https://github.com/speedb-io/speedb) database - a simple key/value database, but handled in a different way: keys use *prefixes*, which makes it possible to batch read and write, so we can get around the "simple key/value" limitation and divide data into logical sectors. -The database requires a filesystem path to open it (if it already exists) or create it on the spot (if it doesn't exist) during construction. It closes itself automatically on destruction. Optionally, it also accepts a bool for enabling compression (disabled by default), if needed. +For contract events specifically, there is a separate [SQLite](https://sqlite.org) abstraction used for better querying capabilities. The rest of this section is related to the Speedb part. -Content in the database is stored as raw bytes. This is due to space optimizations, as one raw byte equals two UTF-8 characters (e.g. an address like `0x1234567890123456789012345678901234567890`, ignoring the "0x" prefix, occupies 20 raw bytes - "12 34 56 ..." - , but 40 bytes if converted to a string, since each byte becomes two separate characters - "1 2 3 4 5 6 ..."). +## General overview -For the main CRUD operations, refer to the `has()`, `get()`, `put()` and `del()` functions. Due to how the database works internally, updating an entry is the same as inserting a different value in a key that already exists, effectively replacing the value that existed before (e.g. `put(oldKey, newValue)`). There's also a few other helper functions such as: +The database requires a filesystem path to open it (if it already exists) or create it on the spot (if it doesn't exist) during construction. It closes itself automatically on destruction. Optionally, it also accepts a bool for enabling compression (disabled by default), if needed. -* `getBatch()` and `putBatch()` for batched operations -* `getKeys()` for fetching only the database's keys -* `keyFromStr()` for encapsulating a key into a Bytes object -* `getLastByPrefix()` for getting the last value stored in a given prefix -* `makeNewPrefix()` for concatenating prefixes when necessary +Content in the database is stored as raw bytes for space optimization purposes - one raw byte equals two UTF-8 characters (e.g. an address like `0x1234567890123456789012345678901234567890`, ignoring the "0x" prefix, occupies 20 raw bytes - "12 34 56 ..." - , but 40 bytes if converted to a string, since each byte becomes two separate characters - "1 2 3 4 5 6 ..."). -## Structs and Prefixes +The main **DB** class supports the typical CRUD operations, as well as a few helper functions for managing batched operations, keys and prefixes (see below). Due to how Speedb works internally, updating an entry is the same as inserting a different value in a key that already exists, effectively replacing the value that existed before (e.g. `put(oldKey, newValue)`). -We have three helper structs to ease database manipulation: +There are three helper structs that further abstract database manipulation in general: -* `DBServer` - struct that contains the host and version of the database that will be connected to -* `DBEntry` - struct that contains an entry to be inserted or read by the database, and has only two members: key and value, both strings -* `DBBatch` - struct that contains multiple `DBEntry`s to be inserted and/or deleted all at once +* **DBServer** - contains the host and version of the database that will be connected to +* **DBEntry** - abstraction of an entry to be inserted or read by the database, and has only two members: key and value, both strings +* **DBBatch** - abstraction for a list of multiple `DBEntry`s to be inserted and/or deleted all at once -We also have a `DBPrefix` namespace to reference the database's prefixes in a simpler way: +There's also a **DBPrefix** namespace for referencing the database's prefixes in a simpler way, using labels instead of raw byte strings. Those prefixes are concatenated to the start of the *key*, so an entry that would have, for example, a key named "abc" and a value of "123", if inserted to the "0003" prefix, would be like this inside the database (in raw bytes, shown as strings here for explanation purposes): `{"0003abc": "123"}`. -| Descriptor | Prefix | -| ------------------ | ------ | -| blocks | 0x0001 | -| heightToBlock | 0x0002 | -| nativeAccounts | 0x0003 | -| txToBlock | 0x0004 | -| rdPoS | 0x0005 | -| contracts | 0x0006 | -| contractManager | 0x0007 | -| events | 0x0008 | -| vmStorage | 0x0009 | -| txToAdditionalData | 0x000A | -| txToCallTrace | 0x000B | +## Prefixes overview -Those prefixes are concatenated to the start of the _key_, so an entry that would have, for example, a key named "abc" and a value of "123", if inserted to the "0003" prefix, would be like this inside the database (in raw bytes format, strings here are just for the sake of the explanation): `{"0003abc": "123"}` +Here's a list of available prefixes from DBPrefix (they're accessed like `DBPrefix::label`, where you change `label` for one of the desired names below): -## Prefixes Overview +| Descriptor | Prefix | +| ------------------- | ------ | +| blocks | 0x0001 | +| heightToBlock | 0x0002 | +| nativeAccounts | 0x0003 | +| txToBlock | 0x0004 | +| rdPoS | 0x0005 | +| contracts | 0x0006 | +| contractManager | 0x0007 | +| events (DEPRECATED) | 0x0008 | +| vmStorage | 0x0009 | +| txToAdditionalData | 0x000A | +| txToCallTrace | 0x000B | ### blocks @@ -123,9 +119,9 @@ Used to store a contract class name based on their address. | ------------------------- | ------------------- | | Prefix + Contract Address | Contract Class Name | -### events +### events (DEPRECATED) -Used to store events emitted from contracts. +Previously used to store events emitted from contracts. Replaced by the SQLite abstraction. DO NOT USE. | Key | Value | | ---------------------------------------------------------------------------------------------------------------- | -------------------------------------- | diff --git a/bdk-implementation/p2p-encoding.md b/bdk-implementation/p2p-encoding.md index 69b36ce..ad6c4b5 100644 --- a/bdk-implementation/p2p-encoding.md +++ b/bdk-implementation/p2p-encoding.md @@ -1,10 +1,10 @@ --- -description: How P2P messages are structured in the Blockchain Development Kit (BDK). +description: How P2P messages are structured in the BDK --- # P2P Encoding -This subchapter explains how P2P data travels through AppLayer, and how said data is encoded and decoded between nodes. All hexes displayed in this document are interpreted as bytes. +This subchapter explains how P2P data travels through a BDK-powered blockchain, and how said data is encoded and decoded between AppLayer nodes. All hex strings displayed in this document are interpreted as raw bytes in code. ## Handshake @@ -15,7 +15,7 @@ After the session successfully creates a connection, the first thing that happen | NodeType | 1 Byte | Host node type | | P2PServerPort | 2 Bytes | P2P Server Port | -## P2P::Message +## Message Structure that holds any type of P2P message, encoded as follows: @@ -26,25 +26,20 @@ Structure that holds any type of P2P message, encoded as follows: | Command ID | 2 Bytes | Command ID | | Payload | X Bytes | Message Command Payload | -### Request Flag +Where each variable represents the following: -Used to tell if the request is a "Request" (0), an "Answer" (1) to a previous request, or a "Broadcast" (2) request (verify and broadcast towards other nodes of the network). +* **Request Flag** - used to tell the message's type (Request, Answer, Broadcast or Notification) + * Requests will always wait and have a respective Answer + * Broadcasts and Notifications are processed directly and independently from each other + * The random ID in a Broadcast message is used to know if the node has previously received that broadcast and if it should rebroadcast or not +* **Random ID** - used to tell different messages apart and make async requests + * Requests calculate their ID with 8 random bytes + * Answers use the same ID from the Request they belong to + * Broadcasts calculate their ID with the first 8 bytes of the hash of the message's payload (using SafeHash) +* **Command ID** - used to tell which command type is encoded in the payload (see below) +* **Payload** - self-explanatory. May be absent as it is not required for certain message types -A Request will always wait and have a respective Answer, while Broadcasts are processed directly. See the difference between functions called by `handleAnswer()`/`handleRequest()` and functions called by `handleBroadcast()`. The random ID of the Broadcast request is used to know if the node has previously received that broadcast and if it should rebroadcast or not. - -### Random ID - -Used to tell different requests apart and make async requests. During a request, the random ID is calculated with 8 random bytes. During an answer, the random ID is equal to the random ID of the respective request. If it's a Broadcast request, the randomID equals `SafeHash(payload)`. - -### Command ID - -Used to tell which command type is encoded in the payload. See below for more info. - -### Payload - -Used as the payload of the request/answer (if needed). - -### Example +### Example of a message Given the example `0x01adf01827349cad810002123456789abcdef` we can extract the following values: @@ -53,8 +48,12 @@ Given the example `0x01adf01827349cad810002123456789abcdef` we can extract the f * Command ID: `0002` * Payload: `123456789abcdef` +Note this is just an example - see below for the real commands and what they return. + ## Commands +Here's a list of all supported P2P message commands available for use in the BDK: + ### Ping Pings another node. diff --git a/bdk-implementation/p2p-overview.md b/bdk-implementation/p2p-overview.md index be3836f..0ec5dec 100644 --- a/bdk-implementation/p2p-overview.md +++ b/bdk-implementation/p2p-overview.md @@ -1,93 +1,72 @@ --- -description: A primer on how P2P messaging works in the Blockchain Development Kit (BDK). +description: A primer on how P2P messaging works in the BDK --- # P2P Overview -This subchapter provides a comprehensive overview of the P2P classes and their organization within the BDK. It further elaborates on the life-cycle of a P2P connection and the dynamic flow of data between nodes. All classes described here are inside the `src/net/p2p` folder. Check their header files for full details on how each of them work. +This subchapter provides a comprehensive overview of the life-cycle of a P2P connection within the BDK and the dynamic flow of data between nodes. -## NodeConns +## Keeping track of peers -The `NodeConns` class (`nodeconns.h`) takes the role of maintaining and periodically updating a list of all peer nodes in the network that are connected to the local node, as well as their respective info. It is through this list that the node keeps itself up-to-date with the most recent node info possible, while also dropping stale connections/timed out peers every now and then to avoid potential network sync problems. +Decentralized networks work with the concept of *peers* - computers in a network running the same kind of software that connect to each other without any kind of third-party in between (e.g. a torrent client that seeds its files to other computers running the torrent protocol). This is one of the essential pillars of the crypto ecosystem - the more active peers a network has, the stronger and more resilient it is against malicious attacks, power failures and other similar issues. -## Session +BDK-powered chains make use of the **NodeConns** class to maintain and periodically update a list of all peer nodes in the network that are connected to the local node, as well as their respective metadata. It is through this list that the local node keeps itself up-to-date with the most recent node info possible, while also dropping stale connections and timed out peers every now and then to avoid potential network sync problems. -The `Session` class (`session.h`) encapsulates a TCP connection with a remote node, responsible for managing handshakes, sending and receiving messages by reading and writing socket data. It servees for both client and server connections, having queues for both inbound and outbound messages, allowing any thread that is responsible for sending a message to carry on with its task without having to wait for the message transmission to complete. +## Communicating with other nodes -Sessions are designed to be used as `shared_ptr`s with `shared_from_this` to manage their lifecycles, as they primarily exist within the handlers of Boost's `io_context`. Once a session has successfully established a connection with a remote node, it is added to a list of sessions controlled by a manager. It is critical to properly manage those shared pointers, as the destructor of the `Session` class is intrinsically linked to the destructor of the socket, which, in turn, calls the `io_context`. If the pointer persists but its referenced `io_context` no longer exists, this will lead to a program crash. +The **Session** class encapsulates a TCP connection with a remote node, being responsible for managing handshakes, sending and receiving messages by reading and writing data through sockets. It serves for both client and server connections, having queues for both inbound and outbound messages, allowing any thread that is responsible for sending a message to carry on with its task without having to wait for the message transmission to complete. -For instance, functions called by the thread pool to parse asynchronous operations accept a `weak_ptr` as a parameter, which is then converted into a `shared_ptr`. This design choice ensures the `Session` object is still viable. In addition, the manager session mutex is locked to prevent the `shared_ptr` from unexpectedly becoming a `unique_ptr`, which would cause a segmentation fault while attempting to delete a session in the absence of its corresponding `io_context`. +Once a session has successfully established a connection with a remote node, it is added to a list of sessions controlled by an internal manager (see below). It is critical to properly manage those active sessions, as destroying the session will also destroy its socket - this design choice is on purpose, since if communication still occurs through a "dead" socket from a session that no longer exists, this will lead to a program crash. ## Life cycle of a session -Upon instantiation, a session's life-cycle is composed of three different routines: *handshake*, *read* and *write*. +Upon instantiation, a session's life-cycle is composed of three different routines: -### Handshake routine +* **Handshake**: first of all, the session gives a handshake to the remote endpoint, effectively making a connection so data can be shared between both nodes +* **Read**: after the handshake, the session listens indefinitely for inbound messages from the remote endpoint. When a message arrives, the session reads it and, if required, proceeds to write a response to the remote endpoint +* **Write**: if the remote endpoint sent a message that requires a response, the session does exactly that - it writes an outbound message for the endpoint and sends it, then goes back to listening indefinitely for the next inbound message -The first routine is giving a handshake to a remote endpoint, effectively making a connection so data can be shared between both ends. It happens like this: +While the handshake routine is done only once, the read and write routines are executed cyclically through the entire duration of the connection, until it is properly closed by one of the endpoints. -* The `run()` method is called first, and uses `boost::asio::dispatch()` to dispatch a call of `write_handshake()` -* `write_handshake()` uses `boost::asio::async_write()` to send a handshake to the remote endpoint, and sets `read_handshake()` as a callback for when the handshake is properly answered -* `read_handshake()` initiates an asynchronous read operation to receive the remote enpoint's handshake, and sets `finish_handshake()` as a callback for when the operation is complete -* `finish_handshake()` validates the recived handshake through the P2P manager and calls `do_read_header()` to start the next routine +## Managing sessions -### Read routine +Sessions are managed by the **ManagerBase** class, which acts as the backbone of the BDK's P2P networking stack. It bears the responsibility of managing sessions, their respective sockets and the global I/O context used by them, overseeing their operations and serving as a base for specialized classes according to th node's type (see below). -The second routine is reading inbound messages coming from the remote endpoint. This routine is executed cyclically through the entire duration of the connection, and goes like this: +Once a session has successfully completed a handshake, it is registered within the manager, which then keeps track of that session's lifecycle. The manager's responsibilities include maintaining a registry of active sessions, handling incoming and outgoing requests and responses, and maintaining the communications between them. -* `do_read_header()` reads an incoming message's header (8 bytes that contain the message's full size), and calls `on_read_header()` when done -* `on_read_header()` is a callback for when the header read process is complete, calling `do_message()` next -* `do_read_message()` reads the full message, using the size in bytes passed to it by `on_header()`, and calls `on_read_message()` when done -* `on_read_message()` is a callback for when the message read process is complete, parsing the message passed by `do_read_message()`, then calling `do_reaD_header()` once again and starting the next round of data exchange +Given its extensive duties, it's imperative that the functions within the manager remain as "active" or "lightweight" as possible - as in, ensure their mutexes are not locked for extended periods, as the manager is concurrently accessed by multiple threads to (de)register sessions, parse messages and/or request information from other nodes. If the manager stays locked for a long time, the node risks being blocked altogether, with potential repercussions extending to the entire network. -### Write routine +It's also important to be aware of the lifespan of the manager's I/O context - sessions do not manage their own isolated context, instead they all use the manager's. This is done for performance purposes, but a greater care must be taken, since if it is deleted at some point and then operations are performed on it (through means of a pointer which would be stale at that point), an exception could trigger when using the now-dangling pointer because the actual I/O context object it referred to would've been already destroyed. -The third routine is writing outbound messages going to the remote endpoint, like this: +## Node types -* An external thread calls `write()` with a shared pointer to a message. Within this function, the queue mutex is locked, and a check is made to see if the outbound message is null - * If it is null, the outbound message is assigned to the message passed as an argument, and `boost::asio::post()` is called with `do_message_header()` as its handler, with the appropriate write strand - * If it is _not_ null (which indicates a write process is already going on), the message is added to the queue, and the write strand will handle it later -* The write strand then begins the following chain of actions: - * `do_write_header()` writes the message's header (8 bytes that contain the message's full size), and calls `on_write_header()` when done - * `on_write_header()` is a callback for when the header write process is done, calling `do_write_message()` next - * `do_write_message()` writes the full message, and calls `on_write_message()` when done - * `on_write_message()` is a callback for when the message write process is done, locking the queue mutex, grabbing the next message from the queue, and calling `do_write_header()` again - this loop goes on until the queue is empty, where the outbound message pointer is set to null and any further callbacks from the write strand come to a halt +As said before, the **ManagerBase** class is not used by itself, rather it is derived into two other specialized classes: **ManagerNormal** and **ManagerDiscovery**. They are aptly named after the two types of nodes that compose a BDK-powered network: *Normal* and *Discovery* nodes respectively. -## ManagerBase and derivatives - -The P2P manager acts as the backbone of the P2P network. It bears the responsibility of managing `Session`s, overseeing their operations and indirectly managing the `io_context`, and serves as a base for the `ManagerNormal` (for Normal nodes) and `ManagerDiscovery` (for Discovery nodes) specialized classes. - -Once a `Session` has successfully completed a handshake, it is registered within the manager, which then oversees the `Session`'s lifecycle. The manager's responsibilities include maintaining a registry of active `Session`s, handling incoming and outgoing requests and responses, and maintaining the communication between them. - -Given its extensive duties, it's imperative that the functions within the manager remain as "lightweight" as possible within their respective mutexes. This is to ensure the manager is not locked for extended periods, as it is concurrently accessed by multiple threads to (de)register `Session`s, parse incoming messages, and/or request information from other nodes. If the manager stays locked for a long time, the node risks being blocked altogether, with potential repercussions extending to the entire network. - -It's also important to be aware of the lifespan of the `io_context` and the objects that use it, before adding or modifying any code that uses the manager. `Session`s do not manage their `io_context`, which means that if you retrieve a `shared_ptr` to a specific `Session`, perform operations on it, and that pointer is deleted elsewhere, you could end up triggering an exception if using the now-dangling pointer because the `io_context` it refers to has already been destroyed. +Normal nodes act as regular nodes in a blockchain, receiving and processing blocks and transactions, while Discovery nodes have the sole purpose of transmitting info about other nodes connected to it to the rest of the network. Due to our usage of CometBFT (which already has its own P2P stack), we have disabled Discovery nodes entirely for now, with plans of reimplementing them in the future. ## Message types -Every incoming message is promptly parsed by the manager via the `handleMessage()` function. These messages can fall into one of the following categories, with each one of them being treated distinctly (the names here are conceptual, the actual variable names differ in implementation): +Every incoming message from a remote endpoint is promptly parsed by the manager and can fall into one of the following categories, with each one of them being treated distinctly (the names here are conceptual, the actual names in code differ a little bit): -* `Request` - a query for specific data from another node - e.g. a list of blocks, a list of transactions, info about the node itself, etc. -* `Answer` - an answer to a given request -* `Broadcast` - a dissemination of specific data to the network *with* possible re-broadcasting ("flooding"), such as a new block or transaction -* `Notification` - a dissemination of specific data to the network *without* re-broadcasting, such as node info +* **Request** - a query for specific data from another node - e.g. a list of blocks/transactions, info about the node itself, etc. +* **Answer** - an answer to a given request +* **Broadcast** - a dissemination of specific data to the network *with* possible re-broadcasting ("flooding"), such as a new block or transaction +* **Notification** - a dissemination of specific data to the network *without* re-broadcasting, such as node metadata -Both `Request` and `Answer` messages work together in a bidirectional flow that goes like this: +*Request* and *Answer* messages work together in a bidirectional flow that goes like this: -* The sender node initiates a `Request` by generating a random 8-byte ID, registering it internally and sending it alongside the message -* The receiver node receives the `Request`, its manager processes it by invoking the corresponding function to address it, formulates an `Answer` with the requested data, assigns it the same ID and sends it back to the sender node -* The sender node receives the `Answer` and checks if the received ID is the same one that was registered earlier. If it is, the manager fulfills the associated `Request` future with the received `Answer` and deregisters the ID. If the ID is _not_ registered, the `Answer` is discarded altogether +* The sender node initiates a Request by generating a random unique 8-byte ID for reference, registering it internally and sending it alongside the message +* The receiver node receives the Request, its manager parses it and formulates an Answer with the requested data, assigns it the same 8-byte ID from the Request and sends it back to the sender node +* The sender node receives the Answer and checks if the received ID is the same one that was registered earlier. If it is, the manager matches the associated Request with the received Answer and deregisters the ID. If the ID is *not* registered, the Answer is discarded altogether -Both `Notification` and `Broadcast` messages, however, work with a simpler unidirectional flow, as the receiver node doesn't have to answer back to the sender. Instead, it verifies the received data and adds it to its own blockchain. The difference between both types is that `Notification` messages are never re-broadcast, while `Broadcast` messages may or may not be re-broadcast to other nodes, depending on whether said nodes had already received or not said broadcast in the past. +*Broadcast* and *Notification* messages work on their own in a simpler unidirectional flow instead, as the receiver node doesn't have to answer back to the sender. Instead, it verifies the received data and adds it to its own blockchain. The difference between both types is that *Notification* messages are never re-broadcast, while *Broadcast* messages may or may not be re-broadcast to other nodes, depending on whether said nodes had already received or not said broadcast in the past. -Due to this specific condition, `Broadcast` messages are specifically handled by the `Broadcaster` class (`net/p2p/broadcaster.h`), while the other types are handled by `ManagerBase` and its derivative classes. +Due to this specific condition, Broadcast messages are specifically handled by the **Broadcaster** class, while the other types are handled normally by ManagerBase and its derivative classes. ## Asynchronous Message Parsing -The first thread under our control that accesses the message is the `io_context` executing that particular `Session`. To optimize the performance of `io_context` and avoid any bottleneck, we offload the task of message parsing to a separate thread pool. This pool handles both parsing of the message and writing back to the `Session`, which involves adding tasks to its write strand or queue. - -One performance-enhancing strategy we employ is the use of `shared_ptr` for handling each message. This prevents unnecessary copying and provides significant benefits when dealing with broadcasts. In such cases, a single message can be utilized by multiple writing sessions, thereby offering a performance boost to the network. +To optimize the performance of the manager's I/O context and avoid any kind of bottleneck, message parsing is offloaded to a separate thread pool. The first thread under our control that accesses the message is the I/O context itself executing that particular session. The thread pool then handles both parsing of the message and writing back to the session, which involves adding tasks to its write strand or queue. The function that handles the answer to a message always returns a promise with the answer. A thread that called a request within the manager towards another node will wait for a few seconds or until the answer is received. -Our current design uses `shared_ptr` for all messages, however, we plan to transition to a system where `unique_ptr` is used for inbound messages and `shared_ptr` for outbound messages. The rationale behind this is to better handle memory ownership. The use of `unique_ptr` provides clearer ownership semantics and improved performance. Since `unique_ptr` cannot be moved into a `std::function` (which would be the task posted to the thread pool), we are simply using `shared_ptr` for now. +One performance-enhancing strategy we employ is the use of pointers for handling each message. This prevents unnecessary copying and provides significant benefits when dealing with broadcasts. In such cases, a single message can be utilized by multiple writing sessions, thereby offering a performance boost to the network. -`handleAnswer()` always fills a `std::promise` with the answer. A thread that called a request within the manager towards another node will wait for a few seconds or until the answer is received. +Our current C++ design uses `shared_ptr` for all messages, however, we plan to transition to a system where `unique_ptr` is used for inbound messages and `shared_ptr` for outbound messages. The rationale behind this is to better handle memory ownership. The use of `unique_ptr` provides clearer ownership semantics and improved performance. Since `unique_ptr` cannot be moved into a function (which would be the task posted to the thread pool), we are simply using `shared_ptr` for now. diff --git a/bdk-implementation/rlp-recursive-length-prefix.md b/bdk-implementation/rlp-recursive-length-prefix.md index a783f9d..4f237d7 100644 --- a/bdk-implementation/rlp-recursive-length-prefix.md +++ b/bdk-implementation/rlp-recursive-length-prefix.md @@ -1,14 +1,14 @@ --- -description: How the Blockchain Development Kit (BDK) uses RLP to encode and decode transactions. +description: How the BDK uses RLP to encode and decode transactions --- # RLP (Recursive-Length Prefix) Transactions coming from the network are (de)serialized through a standard called [Recursive-Length Prefix](http://ethereum.org/en/developers/docs/data-structures-and-encoding/rlp), also known as **RLP** and used extensively by Ethereum. As per their definition: -"*RLP standardizes the transfer of data between nodes in a space-efficient-format. The only purpose of RLP is to encode structure and arbitrarily nested arrays of binary data.*" +*"RLP standardizes the transfer of data between nodes in a space-efficient-format. The only purpose of RLP is to encode structure and arbitrarily nested arrays of binary data."* -### Rules for decoding +## Encoding/Decoding rules First of all, **the first byte of the data string defines what exactly the serialized string is storing**. This can be broken down as follows: @@ -27,7 +27,7 @@ First of all, **the first byte of the data string defines what exactly the seria * e.g. `0xf8a90c8504a817c80082c160944fabb145d64652a948d72533023f6e7a623c7c5380b844a9059cbb0000000000000000000000006b71dcaa3fb9a4901491b748074a314dad9e980b000000000000000000000000000000000000000000000029e7ab336ae0b5000025a0ef2f3450e6860289dce618af68ebc7d518c3cb3ea4d1641cb2fe7c7251ff31d4a0540dcf1500630a1b0d0d0670eee012e2cf2c64cf3288d122e0efb0d3deb0340f` * `0xf8 - 0xf7 = 0x01 -> 1 byte in decimal` -> size is the next 1 byte after `0xf7`, which would be `0xa9 = 169 bytes` -> value is the next 169 bytes after `0xa9` -### Decoding a transaction +## Decoding a transaction Let's take an example of a [serialized and signed transaction](https://etherscan.io/tx/0xfd394cb193386ae904af2ef19247e16c51e6974aa8505dbc9b699cc289fb180d) and decode it: diff --git a/bdk-implementation/source-code-overview.md b/bdk-implementation/source-code-overview.md new file mode 100644 index 0000000..bf3aefb --- /dev/null +++ b/bdk-implementation/source-code-overview.md @@ -0,0 +1,83 @@ +--- +description: An aerial view of the BDK's source code +--- + +# Source code overview + +Looking at a higher level of abstraction, the original C++ implementation of the BDK has its code structured as several modules contained in the following folders: + +* `src/bins`: code for the project's main binaries in their respective subfolders - the blockchain executable itself, contract ABI generator, network simulator, faucet API, test suite, etc. +* `src/bytes`: code related to the `bytes` namespace - a collection of structures, classes and helper functions for manipulating raw byte data +* `src/contract`: everything related to smart contracts - from ABI parsing to custom variable types and template contracts +* `src/core`: the heart of the BDK, contains the main components of the blockchain +* `src/libs`: third-party libraries not inherently tied to the project but used throughout development +* `src/net`: everything related to networking, communication between nodes and support for protocols such as gRPC, HTTP, P2P, JSON-RPC, etc. +* `src/utils`: commonly-used functions, structures, classes, overall logic and miscellanea to support the functioning of the BDK + +There is also a `tests` folder that contains several unit tests for the components described above, following the same subfolder structure for simplicity purposes. You can use the `tree` command on a terminal in the project's root for more details on the folders' structures. + +## Key components + +As the project is constantly iterated upon and components may come and go with time, we rely mostly on self-documenting code (e.g. the Doxygen docs as stated earlier) to carry the burden of detailing them, so we recommend sticking with that if you need a deeper look on what each component does. + +From a simpler point of view, we can pinpoint the most important and/or most frequently used components across the entire project as follows: + +### src/utils + +* **Byte, Bytes, BytesInterface** - aliases for the raw byte data types used throughout the project +* **ContractReflectionInterface** - namespace with utility functions that enable [reflections](https://en.wikipedia.org/wiki/Reflective\_programming) in C++ by extensive use of templates, used mainly for registering contract classes +* **DB, EventsDB** - abstraction of a [Speedb](https://github.com/speedb-io/speedb) (DB) and [SQLite](https://sqlite.org) (EventsDB) database, respectively used internally for manipulating various kinds of storage data, including contract events and helper components (see the Database subchapter) +* **DynamicException** - custom exception class used everywhere in the project +* **FinalizedBlock** - abstraction of a block's structure and data in the chain, inherently unmodifiable after creation (ensures state integrity across nodes) +* **FixedBytes and derivatives** (Address, Functor, Hash, Signature, StorageKey) - abstraction of a C++ STL-compliant fixed-size byte array (e.g. `FixedBytes<10>` has *exactly* 10 bytes - derivatives have predefined sizes) +* **Hex** - abstraction of a strictly hex-formatted string (`0x[1-9][a-f][A-F]`) +* **JsonAbi** - namespace responsible for managing and converting contract ABI data to JSON format (used by the contract ABI generator binary) +* **Log, LogInfo, Logger** - namespace for logging utilities, used for debugging purposes +* **Merkle** - custom implementation of a Merkle Tree, adapted from [those](https://medium.com/coinmonks/implementing-merkle-tree-and-patricia-tree-b8badd6d9591) [sites](https://lab.miguelmota.com/merkletreejs/example/) +* **Options** - singleton with data about the node, frequently accessed by the BDK (generated during build time from a .in file) +* **RandomGen** - implementation of the deterministic randomness generator used for almost everything related to blockchain consensus +* **SafeHash and FNVHash** - custom implementations for unordered map key hashing and [Fowler-Noll-Vo](https://en.wikipedia.org/wiki/Fowler%E2%80%93Noll%E2%80%93Vo\_hash\_function) hashing respectively across the entire project +* **Secp256k1** - namespace that abstracts the functionalities of Bitcoin's [secp256k1](https://en.bitcoin.it/wiki/Secp256k1) elliptic curve cryptography library, used for signing, verification and key derivation (including helper aliases) +* **transactional** - namespace that provides the structs used for committing/reverting transactions and checkpoints during contract execution +* **TxBlock and TxValidator** - abstractions for a block transaction and a Validator transaction respectively (derived from Ethereum's "Account" model, contrary to Bitcoin's "UTXO" model) - includes a **TxAdditionalData** struct with metadata about a deployed contract (e.g. tx hash, gas used, call status, contract address, etc.) +* **UintConv, IntConv, StrConv and EVMCConv** - namespaces related to aliases, conversion and manipulation of specific types of data (e.g. integers, strings, EVMC data, etc.) +* **Utils** - namespace for miscellaneous utility functions, namespaces, enums and typedefs used across the project +* **View** - helper struct for creating views of any kind of data (similar to "std::string_view" for example) + +### src/contract + +See the Contracts section and the respective subsections for more details on the following components: + +* Subfolders for the implemented contract templates, SafeVariables and EVM precompiles (see Precompiled contracts) +* **ABI** - namespace for handling Solidity ABI types and data encoding/decoding (see Solidity ABI) +* **BaseContract and DynamicContract** - base types for Protocol and Dynamic Contracts respectively (includes the **ContractGlobals and ContractLocals** helper classes for managing contract metadata) +* **ContractHost** - class responsible for managing contract execution stacks from beginning to end (including nested calls), the EVMC host implementation for EVM contracts, and general native<->EVM contract interoperability by means of helper members like **MessageDispatcher, ExecutionContext, Cpp/EvmContractExecutor, BlockObservers**, etc. (see EVM contracts) +* **ContractManager** - class responsible for managing all Dynamic Contract creation and deployment logic in the chain (includes the **ContractFactory** helper class) +* **ContractStack** - class responsible for managing temporary data from contract nested call chains (e.g. altered contract balances and SafeVariable commit/revert logic), used by ContractHost in a 1:1 ratio (only one stack instance spawned per host instance during a call) +* **ContractTypes** - list of custom contract names to instance during blockchain deploy (see Creating a Dynamic Contract > Deploying and testing) +* **Event** - abstraction of a Solidity event's structure and data + +### src/core + +* **Blockchain** - the entry point of the system and the main class that unites all the other components (basically the "power button on AppLayer's PC case") +* **Syncer** - class responsible for syncing the chain with other nodes in the network, handling transaction broadcasts and block creation (if the node happens to be a Validator) +* **Consensus** - class responsible applying the network's consensus rules during block and transaction processing +* **Dumpable, DumpManager and DumpWorker** - classes that respectively abstract a dumpable object (that is able to dump itself from memory to disk, e.g. blocks, transactions, contracts, the state itself, etc.), a list of said objects to be dumped when required, and the separate worker thread that does the actual work of dumping the objects +* **State** - abstraction of the chain's current state of accounts, balances, nonces, transactions, token balances, deployed contracts and other kinds of shared data at the current block in the network +* **Storage** - abstraction of the chain's block storage history, including transactions, contracts, accounts, emitted contract events and other immutable blockchain-related data (managed both in memory and disk) + +### src/net + +Under the `http` and `http/jsonrpc` subfolder + +* **HTTPSyncClient and HTTPServer** - classes responsible for respectively acting as an HTTP client and server, for sending and receiving requests to each other +* **HTTPListener** - class responsible for handling incoming HTTP client connections +* **HTTPSession** - class responsible for representing a particular active HTTP connection (includes the **HTTPQueue** helper class for pipelining requests and responses) +* **jsonrpc** - namespace responsible for dealing with JSON-RPC requests, including all the supported commands and their respective data parsing + +Inside the `p2p` subfolder: + +* **Broadcaster** - specifically responsible for handling broadcast messages (e.g. a new block being broadcast to all nodes) +* **ManagerBase and derivatives** (ManagerNormal, ManagerDiscovery) - responsible for managing a list of several session connections, their inter-communications and lifecycles +* **NodeConns** - abstraction of a list of currently connected and regularly refreshed peer nodes and their metadata +* **Session** - class that represents a TCP connection with another node, effectively communicating and sharing data with it diff --git a/bdk-implementation/the-contract-folder.md b/bdk-implementation/the-contract-folder.md deleted file mode 100644 index 4064d3f..0000000 --- a/bdk-implementation/the-contract-folder.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: The base for smart contract support in Blockchain Development Kit (BDK). ---- - -# The contract folder - -This subchapter contains a brief overview of each one of the components inside the `src/contract` folder. - -
- -## ABI - -The `abi.h` file contains the **ABI** namespace - helper functions for handling Solidity ABI types and data natively, as well as the encoding and decoding of said data, also used for C++ <-> EVM inter-operability. - -## CallTracer - -The `calltracer.h` file contains the **Call** struct and the **CallTracer** class respectively, both under the **trace** namespace - those are used specifically for debugging purposes, tracing the flux of a contract call and collecting specific details about it, such as the call type, the sender and receiver addresses, the value sent, the gas used, inputs and outputs, errors and any related subcalls. - -## BaseContract - -The `contract.h` file contains the **BaseContract** class - the base from which all smart contracts are derived - as well as the **ContractGlobals** and **ContractLocals** helper classes that provide access to global and local variables, respectively, for those contracts to work. - -## ContractFactory - -The `contractfactory.h` file contains the **ContractFactory** namespace - helper functions for aiding contract creation, typically used by ContractManager. - -## ContractHost - -The `contracthost.h` file contains the **ContractHost** class - responsible for managing a single chain of contract execution from beginning to end (including nested calls), including the EVMC host implementation for EVM contracts, holding the execution stack and commit/revert during or at the end of the call (via ContractStack), and allowing both C++ and EVM contract calls and inter-operability. - -## ContractManager - -The `contractmanager.h` contains the **ContractManager** class - responsible solely for creating and deploying contracts, passing their ownership to State when done. - -## ContractStack - -The `contractstack.h` file contains the **ContractStack** class - responsible for managing temporary data from contract nested call chains, such as used SafeVariables and altered balances, acting as the one who effectively decides whether those changes are committed or reverted during a contract call. Only one instance of ContractStack is spawned per ContractHost during a call - this ensures the contract call lifecycle is properly contained and state data doesn't spill over other calls. - -## CustomContracts - -The `customcontracts.h` file contains a tuple that holds all the registered contracts within the blockchain. It is also used as a reference for generating the ABI of said contracts. - -## DynamicContract - -The `dynamiccontract.h` file contains the **DynamicContract** class - the base from which all Dynamic Contracts are derived (while _BaseContract_ is mainly used for Protocol Contracts). - -## Event - -The `event.h` file contains the **Event** class - an abstraction of a Solidity event's structure and data, used extensively by contracts. Events are managed within the blockchain by the Storage class and are supported for both C++ and EVM contracts. - -## The variables subfolder - -The `variables` subfolder contains implementations for SafeVariables - special classes that abstract safe versions of variables used within contracts. - -## The templates subfolder - -The `templates` subfolder contains several contracts used as templates, examples and for internal testing purposes. It's also meant to be the folder where user-coded contracts are stored. diff --git a/bdk-implementation/the-core-folder.md b/bdk-implementation/the-core-folder.md deleted file mode 100644 index b1ca2be..0000000 --- a/bdk-implementation/the-core-folder.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -description: The heart of the Blockchain Development Kit (BDK). ---- - -# The core folder - -This subchapter contains a brief overview of each one of the components inside the `src/core` folder. - -
- -## Blockchain and Syncer - -The `blockchain.h` file contains the **Blockchain** and **Syncer** classes. - -The **Blockchain** class acts as the mother class that unites all the other components described throughout the docs, including the Syncer. Think of it as "the power button on AppLayer's PC case" - its objective is to be the entry point of the system and act as a mediator for the other components, passing around data to each other, such as (but not limited to): - -* The global options singleton -* The P2P connection manager -* The database -* The blockchain history/storage (for blocks and contract events) -* The blockchain state (which contains the rdPoS protocol in itself) -* The HTTP server -* The consensus protocol - -The **Syncer** class is responsible for syncing the blockchain with other nodes in the network, as well as handling proper transaction broadcasts and block creations (if the node happens to be a Validator). - -## Consensus - -The `consensus.h` file contains the **Consensus** class - responsible for processing blocks and transactions in the blockchain and applying the network's consensus rules into them. - -## Database dumping - -The `dump.h` file contains the **Dumpable**, **DumpManager** and **DumpWorker** classes - together they are responsible for dumping the blockchain's components from memory to disk when required (e.g. blocks, transactions, contracts, the state itself, etc.) - -*Dumpable* is an abstraction of a dumpable object - that is, any component that inherits it is able to dump itself to disk. All classes that inherit it must implement their own dumping routine accordingly. - -*DumpManager* is the class that manages a list of Dumpable components in the blockchain, iterating through each of them and dumping them one by one when told to. - -*DumpWorker* acts as a worker thread for DumpManager, doing the actual work of dumping each Dumpable component sent by DumpManager in a separate thread. This allows for parallel dumps, speeding up the process significantly, which is important when there are many objects or particularly heavy ones (e.g. State) that need to be dumped ASAP without hanging the entire blockchain. - -## rdPoS - -The `rdpos.h` file contains the **rdPoS** class - the implementation of the *Random Deterministic Proof of Stake* algorithm used by the AppLayer network - as well as the **Validator** class. rdPoS is also considered a smart contract (it derives from **BaseContract**), but as it is an essential part of the AppLayer core protocol, it remains in the `core` folder. - -## State - -The `state.h` file contains the **State** class - an abstraction of the blockchain's current state of accounts, balances, nonces, transactions, token balances and deployed contracts at the current block in the network, responsible for owning and maintaining all of those and a few other shared inner variables. - -A node's state and its data can only be altered through the process of block creation, either by creating a block itself, or receiving one from the network. In AppLayer's case, the class is often used for querying account data (current balance and nonce) and also processing and validating blocks and their transactions, as well as contract calls, following requirements such as (not limited to, but those are some of the most common): - -* Ensuring replay protection (e.g. checking if the transaction has already been validated) -* Checking if the sender address exists and has enough balance to make the transaction -* Checking if the sender address nonce is valid (if it matches what was sent in the transaction) -* Checking if the transaction is not already in the mempool, thus avoiding double spends - -Not all functions from the class update the state. Check the [Doxygen](https://doxygen.nl) docs for more info on that. - -## Storage - -The `storage.h` file contains the **Storage** class - an abstraction of the blockchain's history, storing and maintaining a collection of blocks approved and validated by the network, other nodes, or itself through time. Those blocks store transactions, contracts, accounts, emitted contract events (if the node is initialized with the `RPC_TRACE` indexing mode set) and other blockchain-related data. Said data can't be altered once it's in the blockchain, it can only be searched for or read from. - -On node initialization, if there are no blocks (e.g. the blockchain was just deployed and initialized for the first time), a "genesis" block is automatically created and loaded in memory. The "latest" block is always kept in memory, with subsequent older blocks being dumped to and queried constantly from the database. This makes the blockchain lightweight memory-wise and extremely responsive. - -Searching for and reading from blocks in history is done in several places in the system, so when the Storage and DB classes are working together, they act as the end point of the blockchain's operations history. diff --git a/bdk-implementation/the-utils-folder.md b/bdk-implementation/the-utils-folder.md deleted file mode 100644 index b347b78..0000000 --- a/bdk-implementation/the-utils-folder.md +++ /dev/null @@ -1,133 +0,0 @@ ---- -description: The building blocks of Blockchain Development Kit's (BDK) structure. ---- - -# The utils folder - -This subchapter contains a brief overview of each one of the components inside the `src/utils` folder. - -
- -## Clargs - -The `clargs.h` file contains a few helper functions, structs and enums to parse command-line arguments passed to the project's executables. For an executable to be aware of the argument parser, it must be registered inside the **BDKTool** enum and the executable itself must call the `parseCommandLineArgs()` function, passing the arguments in a C-style manner (argc and argv) and the respective enum value. Check the executables' source files for more info. - -## ContractReflectionInterface - -The `contractreflectioninterface.h` file contains the **ContractReflectionInterface** namespace - utility functions for enabling [reflections](https://en.wikipedia.org/wiki/Reflective\_programming) in C++ through the extensive use of templates, which helps with writing native contracts in a quicker and easier way (e.g. registering functions and variables using less lines of code, abstracting away and taking care of most details). - -## Database - -The `db.h` file contains the **DB** class - an abstraction of a [Speedb](https://github.com/speedb-io/speedb) database used internally for various kinds of storage - as well as helper classes, structures and namespaces to manipulate the database. - -## DynamicException - -The `dynamicexception.h` file contains the **DynamicException** class - a custom exception class inherited from `std::exception` used across the whole project. It is meant to be used when there's no applicable exception from the STD library for a given error that should be caught - usually the STD library is too generic, as the project grows some exceptions may become specific to the point we need to handle them in a customized manner. - -## ECDSA (Secp256k1) - -The `ecdsa.h` file contains the **Secp256k1** namespace - helper functions that abstract the functionalities of Bitcoin's [secp256k1](https://en.bitcoin.it/wiki/Secp256k1) elliptic curve cryptography library, used for handling, deriving and recovering private/public keys and addresses, as well as signing and verifying signed messages. - -The file also contains a few aliases for easier key handling, which are based on our own string abstractions (see FixedBytes below): - -* **PrivKey** (same as **Hash**, or **FixedBytes<32>**) - alias for a given private key -* **PubKey** (same as **FixedBytes<33>**) - alias for a _compressed_ public key -* **UPubKey** (same as **FixedBytes<65>**) - alias for an _uncompressed_ public key - -## FinalizedBlock - -The `finalizedblock.h` file contains the **FinalizedBlock** class - an abstraction of the structure of a block sent through the network and stored in the blockchain. A finalized block is inherently *final* - it cannot be modified anymore after construction. - -The class only contains the bare structure and data of a block - it doesn't do any kind of operation, validation or verification on it. Having only finalized blocks across the entire project ensures block state integrity across all nodes. - -## Hex - -The `hex.h` file contains the **Hex** class - an abstraction of a strictly hex-formatted string (meaning it only accepts the characters within the range of `0x[1-9][a-f][A-F]`), which can also be set to strict or not (whether the string REQUIRES the `0x` prefix or not to be considered valid). Also contains aliases for working with raw-byte strings, such as **Byte, Bytes and BytesArr**. - -## JsonAbi - -The `jsonabi.h` file contains the **JsonAbi** namespace - utility functions for managing and converting contract ABI data to JSON format, used by the contract ABI generator tool in `src/main-contract-abi.cpp`. - -## Logger - -The `logger.h` file contains the **Logger** class - a singleton responsible for logging any kind of info - and helper components such as the **Log** namespace (a namespace with predefined string names for referencing other modules), the **LogInfo** class (encapsulated log data), and the **LogType** enum (for flagging the severity of log messages). The `Logger::logToFile()` and `Logger::logToDebug()` functions print the given details to the respective `log.txt` and `debug.txt` files inside the node's directory. - -The file also contains a plethora of macros to leverage the logging functions and their flags in a more "hands-on" approach, usable anywhere in the project (with a few exceptions depending on the context of where the macro is used in code). Check the Doxygen comments in the file for more info on how to use them. - -## Merkle - -The `merkle.h` class contains the **Merkle** class - a custom implementation of a Merkle Tree, adapted from the following sites: - -* [https://medium.com/coinmonks/implementing-merkle-tree-and-patricia-tree-b8badd6d9591](https://medium.com/coinmonks/implementing-merkle-tree-and-patricia-tree-b8badd6d9591) -* [https://lab.miguelmota.com/merkletreejs/example/](https://lab.miguelmota.com/merkletreejs/example/) - -A "Merkle Tree" is a data structure in binary tree format (e.g. "heap sort"), where data is stored in the "leaves", and the "branches" are paths to reach their data. This structure is commonly used in the crypto space as a tool for _verification_: it hashes the previous layers in pairs to make new layers, bottom-up, until it reaches a single result which would be the "root" of the tree - this makes the root a unique fingerprint for the entire tree, so you only need to check the root hash to verify both the tree and its leaves were not tampered with. - -## Options - -The `options.h` file contains the **Options** class - a singleton with data about the node, frequently accessed by the BDK. This file is pre-generated from its respective `options.h.in` file during the CMake build process, as some of the info have to be gathered on-the-spot from the CMake config files. - -## RandomGen - -The `randomgen.h` contains the **RandomGen** class - the implementation of the RNG (Random Number Generator) used in rdPoS for almost everything related to consensus, responsible for ensuring a satisfactory level of deterministic randomness for the algorithm, and shuffling Validator lists. - -This deterministic randomness guarantees that every node has a chance to answer for a given request (block, randomness, bridging, etc.), while making sure that selected nodes from the network are truly random and not malicious nodes from a bad actor. - -For `RandomGen` to be useful, it needs to be seeded with a truly random number. Therefore, we have to pay attention to the current state of `RandomGen`, making sure that all nodes are always in the same internal state so they can properly sync with each other. - -## SafeHash - -The `safehash.h` contains the **SafeHash** struct - a custom hashing implementation for use with `unordered_map` and/or derivatives, replacing the one used by default, like this for example: `std::unordered_map`. - -Previously, we used `std::unordered_map` in conjunction with [a custom fix from this CodeForces article](https://codeforces.com/blog/entry/62393) so we could continue using the C++ STD's implementation of `unordered_map` due to its blazing fast query times (basically the STD implementation uses `uint64_t` hashes, which is vulnerable to a potentially dangerous edge case where collisions could happen by having an enormous number of accounts and distributing them in a way that they have the same hash across all nodes - this fix is not perfect, since it still uses `uint64_t`, but it's better than nothing since nodes keep different hashes). - -Currently, we replaced the usage of `std::unordered_map` in the whole project with Boost's implementation (`boost::unordered_flat_map`) in conjunction with the [Wyhash](https://github.com/wangyi-fudan/wyhash) library, the highest and fastest quality hash function available for size_t (64-bit) hashes, to achieve a simpler, faster and more solid functionality. - -The file also contains the **FNVHash** struct - a custom implementation of the [Fowler-Noll-Vo](https://en.wikipedia.org/wiki/Fowler%E2%80%93Noll%E2%80%93Vo\_hash\_function) hash struct, used within broadcast messages. This skips compiler optimizations and forces the whole string to be hashed, diminishing the chance of collisions (because, again, we're using 64-bit hashes). - -## FixedBytes and its child classes - -The `strings.h` file contains the **FixedBytes** template class - an abstraction of a normal `std::array` with a fixed size. For example, a `FixedBytes<10>` would have *exactly* 10 characters in it - no more, no less. If initialized as an empty string, it will remain as a 10-character string nonetheless, with all characters set to "empty" (or `\x00` to be more exact). - -Even though FixedBytes can be used on its own (*it's meant to store only bytes*, after all), it also serves as a base for specific classes, also declared within the same file and created with the intent of dealing with the many different ways that data strings are managed and transferred through the project in a better, less confusing and less convoluted way: - -* **Hash** inherits **FixedBytes<32>** and abstracts a given 32-byte hash -* **Functor** abstracts the first 4 bytes of a Solidity function's keccak hash, but does not inherit **FixedBytes** directly - instead it opts for a more practical approach and just treats those bytes as a `uint32_t` -* **Signature** inherits **FixedBytes<65>** and abstracts a given full ECDSA signature (r, s and v) -* **Address** inherits **FixedBytes<20>** and abstracts a given 20-byte address -* **StorageKey** inherits **FixedBytes<52>** and abstracts an EVM storage key (20 bytes address + 32 bytes slot key) - -All of these custom types are standard compliant (trivially copyable and trivially destructible), which means they can be handled like a STD container such as `std::vector` for example. - -## TxBlock and TxValidator - -The `tx.h` file contains the **TxBlock** and **TxValidator** classes - abstractions for a block transaction and a Validator transaction, respectively. The implementation logic and details for those transactions are derived from the "Account" model, used by Ethereum and implemented by the [Aleth](https://github.com/ethereum/aleth) library, which is different from the "UTXO" model used by Bitcoin. - -It also contains a helper struct called `TxAdditionalData`, which contains metadata about a contract that was deployed in the chain, such as the transaction hash, how much gas was used in the transaction, if the call succeeded or not, and the contract's address. - -## UintConv, IntConv, StrConv and EVMCConv - -The respective files `uintconv.h`, `intconv.h`, `strconv.h` and `evmcconv.h` contain several namespaces related to aliases, conversion and manipulation of specific types of data (previously in the **Utils** namespace, now divided into their own namespaces): - -* **UintConv**: contains several `uintX_t` primitive type aliases used across the project, as well as their respective conversion functions (e.g. `uintXToBytes()`/`bytesToUintX()`) -* **IntConv**: contains several `intX_t` primitive type aliases used across the project, as well as their respective conversion functions (e.g. `intXToBytes()`/`bytesToIntX()`) -* **StrConv**: contains a few functions for converting and manipulating raw byte and UTF-8 strings (e.g. `padLeft()`/`padRight()` and their respective raw byte counterparts, `toLower()`/`toUpper()`, `bytesToString()`, `stringToBytes()`, etc.) -* **EVMCConv**: contains a few functions for converting and manipulating EVMC-specific data types (e.g. functors and their specific implementation of uint256) - -## Utils - -The `utils.h` file contains the **Utils** namespace - a place for generalized miscellaneous utility functions, namespaces, enums and typedefs used across the BDK. - -This list is only an example and does not reflect the entire contents of the file. We suggest you read the [Doxygen](https://doxygen.nl/) docs for more info about the class: - -* Helper functions that deal with printing (`safePrint()`, `safePrintTest()`, `printXYZ()`, etc.) -* Aliases for working with raw-byte strings (`Byte`,`Bytes`, `BytesArr`) and helper functions for converting and/or manipulating them (e.g. `appendBytes()`) - * For `appendBytes()` specifically, it is recommended to use it if you need a buffer, otherwise you can use `bytes::join()` as a slightly faster replacement (e.g. if you have all the data required at once, use `bytes::join()`, if you have the data scattered across different places, use a `Bytes` object as a buffer and use it with `appendBytes()`) -* The `ProtocolContractAddress` map for storing addresses for deployed Protocol Contracts (e.g. `rdPoS` and `ContractManager`) -* Enums for network types (`Networks`), contract function types (`FunctionTypes`) and contract types (`ContractType`) -* The `Account` struct, used to maintain account balance and nonce statuses, as well as contract-specific data (if the account represents a contract) -* A wrapper for a pointer that ensures the pointer is never null (`NonNullUniquePtr`), as well as a wrapper for a pointer that forcefully nullifies it on destruction (`PointerNullifier`) -* The `EventParam` struct, used for abstracting a contract event parameter -* Several templated helper functions that deal with tuples, as well as helper functions that deal with Functors -* The `sha3()` function, used extensively as the primary hash function for the entire project -* The `randBytes()` function, used extensively as a random bytes string generator diff --git a/bdk-implementation/transactions-and-blocks.md b/bdk-implementation/transactions-and-blocks.md index 2c21558..746899d 100644 --- a/bdk-implementation/transactions-and-blocks.md +++ b/bdk-implementation/transactions-and-blocks.md @@ -1,22 +1,17 @@ --- -description: How transactions and blocks work in Blockchain Development Kit (BDK). +description: How transactions and blocks work in the BDK --- -# Transactions and Blocks +# Transactions and blocks -This subchapter explains the structure of blocks, transactions, and how both are properly parsed within BDK and AppLayer. +This subchapter explains in depth the structure of blocks and transactions within the BDK, and how both are properly parsed within the AppLayer network. ## How transactions are parsed No matter the type, there are two ways a transaction can be parsed from a bytes string: -* *Directly from RLP* - * Requires deriving the sender (`from`) address and a validity check using secp256k1 - * *Not* included in a block, which means it's a new transaction coming from the network - * Equivalent to Ethereum's `rawTransaction` -* *Directly from the database* - * Considered trustworthy since it already went through the process above - * *Is* included in a block, therefore it's part of the blockchain +* **Directly from RLP** - not included in the chain since it's a new transaction coming from the network, which requires deriving the sender (`from`) address and a validity check using secp256k1 before adding it to the block (this is equivalent to using Ethereum's `eth_sendRawTransaction` command, see the RLP subsection for more details) +* **Directly from the database** - is already included in a block within the chain, therefore it's considered trustworthy since it already went through the validation process stated above ## Transaction structure @@ -27,7 +22,7 @@ Depending on the type, a transaction will contain the following data fields once * **(Both) data** - arbitrary data field (used in contracts) * **(Both) chainId** - unique blockchain ID where the transaction was made * **(TxValidator) nHeight** - height of the block where the transaction was included -* **(TxBlock) nonce** - number of the transaction made from the sender address - always starts at 0, so a nonce of `4` means this is the _fifth_ transaction from the address +* **(TxBlock) nonce** - number of the transaction made from the sender address - always starts at 0, so a nonce of 4 means this is the address' 5th transaction * **(TxBlock) value** - transaction value in its lowest unit * e.g. "satoshi" for BTC, "Wei" for ETH, etc. - "100000000" satoshis would be 1 BTC, "5000000000" Wei would be 0.000000005 ETH (or 5 gwei), so on and so forth * Since we're using an Ethereum-based format, we commonly refer to its terminology, so "value" is in "Wei" @@ -52,8 +47,8 @@ A block will contain the following conceptual structure: * The block's header, which is made of: * Previous block hash (a hash of the previous block's header, signed by the Validator) * "Randomness" (a random seed generated by RandomGen during the previous block's creation) - * Validator Tx Merkle Tree (to verify the integrity of Validator transactions) - * Block Tx Merkle Tree (to verify the integrity of Block transactions) + * Validator Merkle Tree (to verify the integrity of Validator transactions) + * Block Merkle Tree (to verify the integrity of Block transactions) * UNIX timestamp of the block, in microseconds * Block height (commonly known as `nHeight`) * List of Validator transactions diff --git a/understanding-contracts/README.md b/contracts/README.md similarity index 61% rename from understanding-contracts/README.md rename to contracts/README.md index 4bd4d49..5b60fbd 100644 --- a/understanding-contracts/README.md +++ b/contracts/README.md @@ -1,11 +1,11 @@ --- -description: How smart contracts work in the AppLayer protocol. +description: How smart contracts work in AppLayer --- -# Understanding contracts +# Contracts -Contracts in the AppLayer network are custom, developer-made classes that directly interact with the current State of the Blockchain. This chapter will comprehensively cover creating native contracts for AppLayer using the BDK, as well as operating the AppLayer EVM to leverage existing Solidity contracts. +Smart contracts, in short, are custom developer-made classes that directly interact with the current state of the blockchain. This chapter will comprehensively cover the details of smart contract usage and development in the AppLayer network - going from creating precompiled, natively-coded contracts using the BDK, to operating the AppLayer EVM to leverage existing Solidity contracts. AppLayer ensures that contracts deployed in its network, no matter the type, remain compatible with existing frontend Web3 tools (e.g. [MetaMask](https://metamask.io/), [ethers.js](https://github.com/ethers-io/ethers.js/), [web3.js](https://docs.web3js.org/), etc.). Those are originally designed to interact with Solidity contracts and thus require a similar interface. -To call your contract's functions from a frontend, you'll also need to generate its ABI - you can either do it directly with our generator tool if coding a pre-compiled contract (explained further), or replicate their definitions in Solidity and use an external tool like Ethereum's [Remix](https://remix.ethereum.org/) or any other of your preference. This ABI can then be used by your preferred Web3 frontend. +To call your contract's functions from a frontend, you'll also need to generate its ABI - you can either do it directly with our generator tool if coding a pre-compiled contract (explained further), or replicate their definitions in Solidity and use an external tool like Ethereum's [Remix](https://remix.ethereum.org/) or any other of your preference. This ABI can then be used by your preferred Web3 frontend. \ No newline at end of file diff --git a/contracts/evm-contracts/README.md b/contracts/evm-contracts/README.md new file mode 100644 index 0000000..98bf5fc --- /dev/null +++ b/contracts/evm-contracts/README.md @@ -0,0 +1,75 @@ +--- +description: A primer on how EVM smart contracts work in the AppLayer ecosystem +--- + +# EVM contracts + +Aside from native contracts, AppLayer can also execute Solidity contracts as-is by the use of the AppLayer EVM, which is compatible with bytecode deployment. This means any language that compiles to EVM bytecode (e.g. [Solidity](https://soliditylang.org/), [Vyper](https://docs.vyperlang.org/en/stable/), etc.) can be used to deploy contracts in the AppLayer EVM in a seamless, straight-forward way. + +This kind of compatibility is possible thanks to the integration of the [EVMOne](https://github.com/ethereum/evmone) virtual machine (originally made by the Ethereum devs) and [EVMC](https://github.com/ethereum/evmc) libraries. Read the "BDK implementation" section for more details. + +## State management and VM instance creation + +The VM itself is owned and instantiated by the `State` class, which reflects a crucial design decision: centralizing the management of virtual machine resources ensures that each contract execution context is cleanly managed and isolated. Whenever a new transaction or contract call needs to be executed, regardless of its nature (be it a contract execution or a simple native transfer), the `State` class is responsible for instantiating a new `ContractHost` object with the relevant parameters required for execution: + +```cpp +ContractHost( + evmc_vm* vm, + DumpManager& manager, + Storage& storage, + const Hash& randomnessSeed, + ExecutionContext& context, + BlockObservers *blockObservers = nullptr +); +``` + +### Determining contract types and executing calls + +Once an instance of `ContractHost` is created, it delegates contract execution calls to a few members like `ExecutionContext` (which keeps track of data like transaction hash, index, gas limit, etc., as well as the logic required for reverting alterations made during the call when required, e.g. when it fails), `MessageDispatcher` (which re-routes the call to its respective executor - `CppContractExecutor` for C++ calls and `EvmContractExecutor` for EVM calls) and `CallTracer` (which answers RPC calls related to debugging purposes, if the node is set to RPC_TRACE). + +`EvmContractExecutor` also extends the functionalities of `evmc::Host` by overriding several key functions that interface directly with the Ethereum Virtual Machine, which are obligatory for the VM to be able to interact with the blockchain's state: + +```cpp +bool account_exists(const evmc::address& addr) const noexcept final; +evmc::bytes32 get_storage(const evmc::address& addr, const evmc::bytes32& key) const noexcept final; +evmc_storage_status set_storage(const evmc::address& addr, const evmc::bytes32& key, const evmc::bytes32& value) noexcept final; +evmc::uint256be get_balance(const evmc::address& addr) const noexcept final; +size_t get_code_size(const evmc::address& addr) const noexcept final; +evmc::bytes32 get_code_hash(const evmc::address& addr) const noexcept final; +size_t copy_code(const evmc::address& addr, size_t code_offset, uint8_t* buffer_data, size_t buffer_size) const noexcept final; +bool selfdestruct(const evmc::address& addr, const evmc::address& beneficiary) noexcept final; +evmc::Result call(const evmc_message& msg) noexcept final; +evmc_tx_context get_tx_context() const noexcept final; +evmc::bytes32 get_block_hash(int64_t number) const noexcept final; +void emit_log(const evmc::address& addr, const uint8_t* data, size_t data_size, const evmc::bytes32 topics[], size_t topics_count) noexcept final; +evmc_access_status access_account(const evmc::address& addr) noexcept final; +evmc_access_status access_storage(const evmc::address& addr, const evmc::bytes32& key) noexcept final; +evmc::bytes32 get_transient_storage(const evmc::address &addr, const evmc::bytes32 &key) const noexcept final; +void set_transient_storage(const evmc::address &addr, const evmc::bytes32 &key, const evmc::bytes32 &value) noexcept final; +``` + +These methods manage everything from account validation to logging, providing access to the blockchain's state and storage, and handling calls between contracts. The `EvmContractExecutor` class encapsulates these functions, ensuring that each contract execution is properly secured and isolated from each other. + +## Seamless native/EVM integration + +Achieving seamless integration between native (e.g. C++) and EVM contracts revolves around the uniformity in the encoding and decoding of their arguments. By standardizing the process, we ensure that calls between different contract types are handled efficiently without the need for separate mechanisms, therefore allowing the BDK to handle it all at once. + +### The evmc\_message struct + +We do this by using the `evmc_message` struct as a base, aligning the call structures between native and EVM environments. This uniformity simplifies the interaction framework and reduces the potential for errors and data mismanagement: + +```cpp +struct evmc_message { + enum evmc_call_kind kind; + uint32_t flags; + int32_t depth; + int64_t gas; + evmc_address recipient; + evmc_address sender; + const uint8_t* input_data; + size_t input_size; + evmc_uint256be value; + evmc_bytes32 create2_salt; + evmc_address code_address; +}; +``` diff --git a/contracts/evm-contracts/calling-contracts.md b/contracts/evm-contracts/calling-contracts.md new file mode 100644 index 0000000..696fb56 --- /dev/null +++ b/contracts/evm-contracts/calling-contracts.md @@ -0,0 +1,82 @@ +--- +description: How different types of contracts call each other in AppLayer +--- + +# Calling contracts + +The previous subchapters explained how both types of contracts (native and EVM) are handled and executed internally in the BDK. Here's how they actually interact with each other in AppLayer's ecosystem (as in, at a higher "user/developer" level of abstraction). + +Most details from native contract implementations were taken out for simplicity purposes. See the "Precompiled contracts" subsection for more details on how to properly code native contracts. + +## Calling native contracts from EVM + +To invoke a native (e.g. C++) contract by calling it from an EVM contract, we use the standard Solidity interface to abstract the native implementation. This approach ensures that EVM-to-native calls are as straightforward as EVM-to-EVM calls. + +For example, let's suppose we have a C++ contract coded like this: + +```c++ +class MyContract : public DynamicContract { + private: + // ...some code... + public: + // ...some more code... + uint256_t myFunction(const uint256_t& arg1, const uint256_t& arg2) const; + // ...yet some more code... +} +``` + +First, we define a Solidity interface that matches the signature of the native functions you wish to call. This interface acts as a facade, providing a Solidity view of the naitive contract's functionalities: + +```solidity +interface MyContract { + // ...some code... + function myFunction(uint256 arg1, uint256 arg2) external view returns (uint256); + // ...some more code... +} +``` + +Then, we use the defined interface to make calls to the native contract. This is handled similarly to any inter-contract communication in Solidity, ensuring a seamless integration layer: + +```solidity +contract AnotherContract { + function callMyFunction(address cppAddr, uint256 arg1, uint256 arg2) public view returns (uint256) { + return MyContract(cppAddr).myFunction(arg1, arg2); + } +} +``` + +## Calling EVM contracts from native + +To invoke an EVM contract by calling it from a native contract, we leverage a templated approach that mimics the EVM contract's functions in a native class (basically the same idea as above but going the other way around). This approach provides a type-safe way to interact with contracts written in Solidity or other EVM-compatible languages. + +For example, let's suppose we have the same contract from the previous section, coded in Solidity: + +```solidity +interface MyContract { + // ...some code... + function myFunction(uint256 arg1, uint256 arg2) external view returns (uint256); + // ...some more code... +} +``` + +First, we define a proxy native (in this case, C++) class that represents the EVM contract. This class will include stubs of the contract's functions, which do not contain actual logic but serve to match the contract's interface in the blockchain: + +```cpp +class MyContract { + private: + // ...some code... + public: + // ...some more code... + uint256_t myFunction(const uint256_t& arg1, const uint256_t& arg2) const; + // ...yet some more code... +}; +``` + +Then, we ensure that the proxy class is registered within the blockchain before any calls are made. Typically, this registration is done once, often in the constructor of the calling native contract (using `registerContract()`), to set up the reflection system used for method invocation: + +```cpp +uint256_t AnotherContract::callMyFunction(const Address& targetAddr, const uint256_t& arg1, const uint256_t& arg2) const { + MyContract::registerContract(); // This is usually done in the contract itself + return this->callContractViewFunction(this, targetAddr, &MyContract::myFunction, arg1, arg2); +} +``` diff --git a/evm-contracts/executing-contract-calls-via-evmc.md b/contracts/evm-contracts/executing-contract-calls-via-evmc.md similarity index 59% rename from evm-contracts/executing-contract-calls-via-evmc.md rename to contracts/evm-contracts/executing-contract-calls-via-evmc.md index 7d9d46d..2ed1460 100644 --- a/evm-contracts/executing-contract-calls-via-evmc.md +++ b/contracts/evm-contracts/executing-contract-calls-via-evmc.md @@ -1,10 +1,10 @@ --- -description: How to use AppLayer's EVM directly to call contracts. +description: How AppLayer's EVM contracts interact with the native blockchain state --- # Executing contract calls via EVMC -To execute a contract call within AppLayer's EVM environment, we use the `evmc_execute()` function from the EVMC library. This function orchestrates the execution of contract bytecode, interfacing directly with the Ethereum Virtual Machine: +To execute a contract call within AppLayer's EVM environment, we use the `evmc_execute()` function from the EVMC library. This function orchestrates the execution of contract bytecode and interfaces directly with the virtual machine: ```cpp static inline struct evmc_result evmc_execute( @@ -14,9 +14,9 @@ static inline struct evmc_result evmc_execute( ); ``` -It's crucial to understand that the VM itself is _stateless_—it does not maintain any information about the contracts' states or their data. The VM's role is strictly to interpret and execute bytecode according to the Ethereum protocol specifications. +It's crucial to understand that the VM itself is *stateless* - it does not maintain any information about the contracts' states or their data. Its role is strictly to interpret and execute bytecode according to the Ethereum protocol specifications. -To enable the stateless VM to interact with the state (such as storage keys, account balances, or initiating further contract calls), we must provide it with access to the state through the `evmc_host_interface` and `evmc_host_context` structs. The `evmc_host_interface` struct contains a set of callback functions that the VM can use to query or modify the state: +To enable the stateless VM to interact with the blockchain's state (such as querying storage keys, account balances, or initiating further contract calls), we must provide it with access to the state through the `evmc_host_interface` and `evmc_host_context` structs. The former contains a set of callback functions that the VM can use to query or modify the blockchain's state: ```cpp struct evmc_host_interface { @@ -39,11 +39,11 @@ struct evmc_host_interface { }; ``` -Within AppLayer's BDK, we use `evmc_execute()` like this: +Within AppLayer's BDK (more specifically inside `EvmExecutionContext::executeEvmcMessage()`), we use `evmc_execute()` like this for example: ```cpp evmc::Result result (evmc_execute(this->vm_, &this->get_interface(), this->to_context(), evmc_revision::EVMC_LATEST_STABLE_REVISION, &msg, recipientAcc.code.data(), recipientAcc.code.size())); ``` -`ContractHost` is casted into `evmc_host_interface` to provide the VM with the necessary state access functions. This allows the VM to interact with the state and execute contract calls within AppLayer's environment. +`ContractHost` is casted into `evmc_host_interface` to provide the VM with the necessary state access functions. This allows the VM to interact directly with the state and execute contract calls within AppLayer's native environment. diff --git a/contracts/evm-contracts/handling-contract-calls.md b/contracts/evm-contracts/handling-contract-calls.md new file mode 100644 index 0000000..c1df95e --- /dev/null +++ b/contracts/evm-contracts/handling-contract-calls.md @@ -0,0 +1,79 @@ +--- +description: How contract calls happen from both sides in AppLayer +--- + +# Handling contract calls + +Here's a quick overview on how AppLayer differentiates between both types of contract calls (native and EVM). As always, we recommend reading the "BDK implementation" for more details on how it all works. + +We employ templated functions to support flexible and efficient interaction. These templates allow passing any combination of arguments and return types (including `void`) to and from other types of contracts. This helps with leveraging a fast ABI encoding/decoding process, ensuring optimal performance and flexibility during contract execution and allowing for dynamic contract interactions by accommodating various contract behaviors and states, without having to pre-define all possible function signatures. + +## Native calls + +The `ContractHost` class employs several functions dedicated to native contract execution contexts. For calls from a native (e.g. C++) contract to another contract, we have two main templated functions - `callContractViewFunction()` (for view functions) and `callContractFunction()` (for non-view/callable/non-payable functions): + +```cpp +template +R callContractViewFunction( + const BaseContract* caller, const Address& targetAddr, R(C::*func)(const Args&...) const, const Args&... args +); + +template +R callContractFunction( + BaseContract* caller, const Address& targetAddr, const uint256_t& value, R(C::*func)(const Args&...), const Args&... args +); +``` + +## EVM calls + +The `EvmContractExecutor` class employs several functions dedicated to EVM contract execution contexts. For calls from the EVM to another contract, the `call()` function plays a crucial role. It is tasked with creating and handling calls to other contracts, encapsulating the complexity of contract interaction within a simple interface. + +This function is designed to handle all kinds of known EVM contract call types, as shown below: + +```cpp +evmc::Result EvmContractExecutor::call(const evmc_message& msg) noexcept { + Gas gas(msg.gas); + const uint256_t value = EVMCConv::evmcUint256ToUint256(msg.value); + + const auto process = [&] (auto& msg) { + try { + const auto output = messageHandler_.onMessage(msg); + + if constexpr (concepts::CreateMessage) { + return evmc::Result(EVMC_SUCCESS, int64_t(gas), 0, bytes::cast(output)); + } else { + return evmc::Result(EVMC_SUCCESS, int64_t(gas), 0, output.data(), output.size()); + } + } catch (const OutOfGas&) { + return evmc::Result(EVMC_OUT_OF_GAS); + } catch (const std::exception& err) { + Bytes output; + + if (err.what() != nullptr) { + output = ABI::Encoder::encodeError(err.what()); + } + + return evmc::Result(EVMC_REVERT, int64_t(gas), 0, output.data(), output.size()); + } + }; + + if (msg.kind == EVMC_DELEGATECALL) { + EncodedDelegateCallMessage encodedMessage(msg.sender, msg.recipient, gas, value, View(msg.input_data, msg.input_size), msg.code_address); + return process(encodedMessage); + } else if (msg.kind == EVMC_CALL && msg.flags == EVMC_STATIC) { + EncodedStaticCallMessage encodedMessage(msg.sender, msg.recipient, gas, View(msg.input_data, msg.input_size)); + return process(encodedMessage); + } else if (msg.kind == EVMC_CALL) { + EncodedCallMessage encodedMessage(msg.sender, msg.recipient, gas, value, View(msg.input_data, msg.input_size)); + return process(encodedMessage); + } else if (msg.kind == EVMC_CREATE) { + EncodedCreateMessage encodedMessage(msg.sender, gas, value, View(msg.input_data, msg.input_size)); + return process(encodedMessage); + } else if (msg.kind == EVMC_CREATE2) { + EncodedSaltCreateMessage encodedMessage(msg.sender, gas, value, View(msg.input_data, msg.input_size), msg.create2_salt); + return process(encodedMessage); + } + + std::unreachable(); +} +``` diff --git a/contracts/internal-and-external-contract-calls.md b/contracts/internal-and-external-contract-calls.md new file mode 100644 index 0000000..52d6db0 --- /dev/null +++ b/contracts/internal-and-external-contract-calls.md @@ -0,0 +1,13 @@ +--- +description: Not all contract calls are equal +--- + +# Internal and external contract calls + +In the context of interacting with smart contracts, calls can be classified into two types: *external* and *internal*. + +**External calls** occur when a contract is invoked from an external source, which can be either a user transaction (when a block is being processed) or a remote procedure call (RPC for short, e.g. an `eth_call` request). External calls serve as entry points to the contract from the outside world, initiating a transaction or a query. Within AppLayer's BDK, to process an external call, a `ContractHost` object is created, which establishes the execution context based on the specified parameters. + +**Internal calls** occur when a contract, already running as part of an ongoing transaction or chain of execution, calls another contract or a function within another contract. This is considered an internal process, as it happens within the chain of execution started by an external call. In AppLayer's BDK, internal calls use the same `ContractHost` object created for the initial external call. This maintains consistency and control over the execution environment, allowing for commits or rollbacks within the same transactional context. + +Basically, an external call creates a new "chain of execution", while internal calls only exist within that same chain of execution, thus they cannot create new chains by themselves. A chain of execution is a sequence of calls that are executed in a given external call, always composed of an initial external call and a set of internal calls (if applicable). diff --git a/contracts/precompiled-contracts/README.md b/contracts/precompiled-contracts/README.md new file mode 100644 index 0000000..f262eb3 --- /dev/null +++ b/contracts/precompiled-contracts/README.md @@ -0,0 +1,96 @@ +--- +description: A primer on natively-coded smart contracts in the AppLayer ecosystem +--- + +# Precompiled contracts + +Precompiled contracts (also known as "native contracts", "natively-coded contracts" or "stateful pre-compiles") are contracts coded with the blockchain's native language. Things like transaction parsing methods and arguments, as well as management and storage of the contract's variables in a database, are manually coded in the blockchain's native language (e.g. C++ in bdk-cpp) to be tightly integrated with the blockchain itself. + +The term "stateful pre-compile" comes from the notion that it can maintain a state (thus "stateful") and is basically machine code, not interpreted by a virtual machine (thus "pre-compile"). + +Similar to Solidity contracts, they can be used to employ any type of logic within the network. Unlike Solidity, however, they aren’t subject to EVM constraints. This means we can take advantage of that fact and have full control of the contract's logic, unleashing blazing fast performance, flexibility and power. + +The precompiled contract templates provided by AppLayer's BDK (in the `src/contract/templates` folder) are based on OpenZeppelin contracts, maintaining the same operational standards known in the Solidity ecosystem, but coded in C++ (in the case of bdk-cpp). + +## Example of a precompiled contract + +Given the example Solidity contract: + +```cpp +// SPDX-License-Identifier: MIT +pragma solidity ^0.8.10; + +contract ExampleContract { + mapping(address => uint256) values; + function setValue(address addr, uint256 value) external { + values[addr] = value; + return; + } +} +``` + +As a precompiled C++ contract, the code should look something like this (don't worry about the details, they will be explained in further subchapters): + +### Contract header + +```cpp +#include <...> +class ExampleContract : public DynamicContract { + private: + std::unordered_map values; // or boost::unordered_flat_map for example + // Const-reference as they are not changed by the function. + void setValue(const Address& addr, const uint256_t& value); + public: + // Constructor for creating the contract the first time. + // "address" is where the contract will be deployed. + // "creator" is the address that created the contract. + // "chainId" is the chain ID where the contract will operate. + ExampleContract( + const Address& address, const Address& creator, const uint64_t& chainId + ); + // Constructor for loading the already-created contract from the database. + ExampleContract( + const Address& address, const DB& db + ); + void callContractWithTransaction(const Tx& transaction); +} +``` + +### Contract source + +```cpp +#include "ExampleContract.h" + +ExampleContract( + const Address& address, const Address& creator, const uint64_t& chainId +) : DynamicContract("ExampleContract", address, creator, chainId) { + // Initialize the contract's variables for the first time as the contract is being created + ... +} + +ExampleContract::ExampleContract( + const Address& address, + const DB& db +) : DynamicContract(address, db) { + // Load the contract's variables from the database as it already exists there + ... +} + +void ExampleContract::setValue(const Address &addr, const uint256 &value) { + this->values[addr] = value; + return; +} + +void ExampleContract::callContractWithTransaction(const Tx& transaction) { + // Used to route and decode transactions. + // The data inside the if block is only an example. + // A real contract would match both functor and arguments to the called contract. + std::string_view txData = transaction.getData(); + auto functor = txData.substr(0,8); + // "0x48461b56" is equivalent to Keccak256("setValue(address,uint256)") + if (functor == Utils::hexToBytes("0x48461b56")) { + this->setValue(ABI::Decoder::decodeAddress(txData, 8), ABI::Decoder::decodeUint256(txData, 8 + 32)); + } + return; +} +``` diff --git a/precompiled-contracts/creating-a-dynamic-contract-advanced.md b/contracts/precompiled-contracts/creating-a-dynamic-contract-advanced.md similarity index 77% rename from precompiled-contracts/creating-a-dynamic-contract-advanced.md rename to contracts/precompiled-contracts/creating-a-dynamic-contract-advanced.md index de0cdc5..860ae2a 100644 --- a/precompiled-contracts/creating-a-dynamic-contract-advanced.md +++ b/contracts/precompiled-contracts/creating-a-dynamic-contract-advanced.md @@ -1,5 +1,5 @@ --- -description: A walkthrough on a more advanced Dynamic Contract usage. +description: A walkthrough on a more advanced DynamicContract usage --- # Creating a Dynamic Contract (Advanced) @@ -86,7 +86,7 @@ Inside `erc20wrapper.h`, let's implement the header (comments were taken out so #include "../contractmanager.h" #include "../dynamiccontract.h" #include "../variables/safeunorderedmap.h" -#include "erc20.h" +#include "standards/erc20.h" class ERC20Wrapper : public DynamicContract { private: @@ -101,16 +101,17 @@ class ERC20Wrapper : public DynamicContract { ~ERC20Wrapper() override; static void registerContract() { - ContractReflectionInterface::registerContractMethods< - ERC20Wrapper, const Address&, const Address&, const uint64_t&, DB& - >( - std::vector{}, - std::make_tuple("getContractBalance", &ERC20Wrapper::getContractBalance, FunctionTypes::View, std::vector{"token"}), - std::make_tuple("getUserBalance", &ERC20Wrapper::getUserBalance, FunctionTypes::View, std::vector{"token", "user"}), - std::make_tuple("withdraw", &ERC20Wrapper::withdraw, FunctionTypes::NonPayable, std::vector{"token", "value"}), - std::make_tuple("transferTo", &ERC20Wrapper::transferTo, FunctionTypes::NonPayable, std::vector{"token", "to", "value"}), - std::make_tuple("deposit", &ERC20Wrapper::deposit, FunctionTypes::NonPayable, std::vector{"token", "value"}) - ); + static std::once_flag once; + std::call_once(once, []() { + DynamicContract::registerContractMethods( + std::vector{}, + std::make_tuple("getContractBalance", &ERC20Wrapper::getContractBalance, FunctionTypes::View, std::vector{"token"}), + std::make_tuple("getUserBalance", &ERC20Wrapper::getUserBalance, FunctionTypes::View, std::vector{"token", "user"}), + std::make_tuple("withdraw", &ERC20Wrapper::withdraw, FunctionTypes::NonPayable, std::vector{"token", "value"}), + std::make_tuple("transferTo", &ERC20Wrapper::transferTo, FunctionTypes::NonPayable, std::vector{"token", "to", "value"}), + std::make_tuple("deposit", &ERC20Wrapper::deposit, FunctionTypes::NonPayable, std::vector{"token", "value"}) + ); + }); } uint256_t getContractBalance(const Address& token) const; @@ -129,13 +130,13 @@ Here, we recreated the contract's functions but also added a few extra functions * Two constructors - one for creating the contract from scratch, and another for loading it from the database * The `ConstructorArguments` tuple, `registerContract()` and `registerContractFunctions()` functions for proper contract registering (notice that the tuple is required, even though it's empty) -* The `dump()` function for saving the contract's variables -* Private SafeVariables (in this case, `SafeUnorderedMap`) to handle the contract's variables +* The `dump()` function for saving the contract's variables in the database +* Private SafeVariables (in this case, `SafeUnorderedMap`) to handle the contract's data * The contract's functions according to the Solidity signatures -Like in SimpleContract's case, you must include your contract's header in `customcontracts.h` to register it, and check it's set to generate its ABI through `main-contract-abi.cpp`. In this specific case for `ERC20Wrapper`, it's assumed that both steps are already done, but it's good to check again just in case. +Like with `SimpleContract`, you must include your contract's header in `src/contract/customcontracts.h` to register it, and check it's set to generate its ABI in `src/bins/contractabigenerator/main.cpp`. In this specific case for `ERC20Wrapper`, it's assumed that both steps are already done, but it's good to check again just in case. -## Implementing the contract constructors and dumping function +## Implementing contract constructors and dump function Inside `erc20wrapper.cpp`, let's implement both constructors and the dumping function: @@ -146,7 +147,7 @@ ERC20Wrapper::ERC20Wrapper(const Address& contractAddress, const DB& db ) : DynamicContract(contractAddress, db), tokensAndBalances_(this) { for (const auto& dbEntry : db.getBatch(this->getNewPrefix("tokensAndBalances_"))) { - bytes::View valueView(dbEntry.value); + View valueView(dbEntry.value); this->tokensAndBalances_[Address(dbEntry.key)][Address(valueView.subspan(0, 20))] = Utils::fromBigEndian(valueView.subspan(20)); } @@ -183,11 +184,11 @@ DBBatch ERC20Wrapper::dump() const { } ``` -One constructor will create a new contract from scratch, as there is no previous existing contract to load, while the other will load the contract from the database when it already exists there. On both cases you are required to initialize, commit and enable registering for _all_ the variables of your contract by hand within the `DynamicContract` constructor, as well as calling `registerContractFunctions()`, all in the same order as explained in the previous subchapter. The dumping function on the other hand is responsible for saving the current information within the contract back to the database. +One constructor will create a new contract from scratch, as there is no previous existing contract to load, while the other will load the contract from the database when it already exists there. On both cases you are required to initialize, commit and enable registering for *all* the existing SafeVariables in your contract's constructors, as well as calling `registerContractFunctions()`, all in the same order as explained in the previous subchapter. The dumping function on the other hand is responsible for saving the current information within the contract back to the database. -Notice that your contract's name ("ERC20Wrapper") is the same as your contract's class name (`ERC20Wrapper`) - again, just like with SimpleContract, this match is **mandatory**, otherwise a segfault will happen. `getNewPrefix()` does the same as `getDBPrefix()`, but with a user-defined string appended to it, so this would be equivalent to `DBPrefix::contracts` + the contract's address + `tokensAndBalances_`. +Notice that your contract's name ("ERC20Wrapper") is the same as your contract's class name (`ERC20Wrapper`) - again, just like with `SimpleContract`, this match is **mandatory**, otherwise a segfault will happen. `getNewPrefix()` does the same as `getDBPrefix()`, but with a user-defined string appended to it, so this would be equivalent to "`DBPrefix::contracts` + the contract's address + `tokensAndBalances_`" (as a raw bytes string). -## Implementing the contract functions +## Implementing contract functions This step is pretty straightforward, we just follow the rules explained previously: @@ -232,14 +233,14 @@ void ERC20Wrapper::deposit(const Address& token, const uint256_t& value) { ### Calling functions from another contract -Notice that, in the example above, some functions are calling functions from another contract. This is done by calling `callContractViewFunction()` (**for view functions**) and `callContractFunction()`(**for non-view/callable functions**), both of which require the following arguments: +Note that, in the example above, some functions are calling functions from another contract. This is done by calling `callContractViewFunction()` (**for view functions**) and `callContractFunction()`(**for non-view/callable functions**), both of which require the following arguments: * The other contract's address (in this case, `token`) * A reference to the function that will be called - in this case: * `getContractBalance()` calls `ERC20::balanceOf()` * `withdraw()` and `transferTo()` call `ERC20::transfer()` * `deposit()` calls`ERC20::transferFrom()` -* The function's arguments, if there's any - in this case: +* The function's arguments, if there are any - in this case: * `ERC20::balanceOf()` will receive our contract's own address as `getContractAddress()` * `ERC20::transfer()` will receive the receiver address as `to` or `getCaller()`, and the value to be transferred as `value` * `ERC20::transferFrom()` will receive the sender's address as `getCaller()`, the receiver's address as `getContractAddress()`, and the value to be transferred as `value` @@ -273,18 +274,20 @@ This creates a new `ERC20` contract with the respective parameters: * The caller/transaction value, again, `0` * The new contract's constructor parameters, in this case an `ERC20` contract needs the token name (`"TestToken"`), its ticker (`"TST"`), number of decimals (`18`), and the amount of tokens that will be minted at creation (`1000000000000000000`, which equals exactly 1 TST with 18 decimals) -## Registering the contract's functions +## Registering contract functions -Once we're done with implementing the contract, we must register it. We've already coded the `ConstructorArguments` tuple and the `registerContract()` function in the header, so all that's left is to override `registerContractFunctions()` so we can register the contract's functions. +Once we're done with implementing the contract, we must register it. We've already coded the `ConstructorArguments` tuple and the `registerContract()` function in the header, so all that's left is to override `registerContractFunctions()` so we can register the contract's functions: ```cpp void ERC20Wrapper::registerContractFunctions() { registerContract(); - this->registerMemberFunction("getContractBalance", &ERC20Wrapper::getContractBalance, FunctionTypes::View, this); - this->registerMemberFunction("getUserBalance", &ERC20Wrapper::getUserBalance, FunctionTypes::View, this); - this->registerMemberFunction("withdraw", &ERC20Wrapper::withdraw, FunctionTypes::NonPayable, this); - this->registerMemberFunction("transferTo", &ERC20Wrapper::transferTo, FunctionTypes::NonPayable, this); - this->registerMemberFunction("deposit", &ERC20Wrapper::deposit, FunctionTypes::NonPayable, this); + this->registerMemberFunctions( + std::make_tuple("getContractBalance", &ERC20Wrapper::getContractBalance, FunctionTypes::View, this), + std::make_tuple("getUserBalance", &ERC20Wrapper::getUserBalance, FunctionTypes::View, this), + std::make_tuple("withdraw", &ERC20Wrapper::withdraw, FunctionTypes::NonPayable, this), + std::make_tuple("transferTo", &ERC20Wrapper::transferTo, FunctionTypes::NonPayable, this), + std::make_tuple("deposit", &ERC20Wrapper::deposit, FunctionTypes::NonPayable, this) + ); } ``` diff --git a/precompiled-contracts/creating-a-dynamic-contract-simple/README.md b/contracts/precompiled-contracts/creating-a-dynamic-contract-simple/README.md similarity index 67% rename from precompiled-contracts/creating-a-dynamic-contract-simple/README.md rename to contracts/precompiled-contracts/creating-a-dynamic-contract-simple/README.md index 281a7de..4bc515b 100644 --- a/precompiled-contracts/creating-a-dynamic-contract-simple/README.md +++ b/contracts/precompiled-contracts/creating-a-dynamic-contract-simple/README.md @@ -1,18 +1,20 @@ --- -description: A walkthrough on the basics of a Dynamic Contract's structure. +description: A walkthrough on the basics of a Dynamic Contract's structure --- # Creating a Dynamic Contract (Simple) -Let's create a test contract that allows three variables - `name`, `number` and `tuple` - to be changed by the owner of the contract, as well as perform basic operations with structs/tuples and event emissions. We will call this contract `SimpleContract` (which just so happens to be already implemented by the project due to internal testing purposes, but you can do it yourself by hand - check the `src/contract/templates/simplecontract.{h,cpp,sol}` files for reference). +Let's create a test contract that allows three variables - `name`, `number` and `tuple` - to be changed by the owner of the contract, as well as perform basic operations with structs/tuples and event emissions. We will call this contract `SimpleContract`, which just so happens to be already implemented by the project due to internal testing purposes, but we'll be implementing a simplified version of it here. Check the `src/contract/templates/simplecontract.{h,cpp,sol}` files for the full reference. -### Creating the Files +This guide covers specifically the C++ implementation of BDK (bdk-cpp). It assumes you already have a local testnet setup in your environment. See the "BDK implementation" section to further understand how to do that. + +## Creating the files First we need to create the contract's header (`.h`) and source (`.cpp`) files, as is customary in C++ development - the header will contain the definition of our contract class, and the source will contain the implementation details. Go to your local testnet's root folder, into the `src/contract/templates` subfolder, and create two new files - `simplecontract.h` and `simplecontract.cpp`. Those files will contain the declaration and definition of your contract's logic, respectively. -Then, add both files to the `CMakeLists.txt` file in the same folder, so they can be compiled with the project: +Then, add both files to the `CMakeLists.txt` file in that same folder, so they can be compiled with the project: ```cmake set(CONTRACT_HEADERS diff --git a/precompiled-contracts/creating-a-dynamic-contract-simple/deploying-and-testing.md b/contracts/precompiled-contracts/creating-a-dynamic-contract-simple/deploying-and-testing.md similarity index 83% rename from precompiled-contracts/creating-a-dynamic-contract-simple/deploying-and-testing.md rename to contracts/precompiled-contracts/creating-a-dynamic-contract-simple/deploying-and-testing.md index ed88a4d..68b41f2 100644 --- a/precompiled-contracts/creating-a-dynamic-contract-simple/deploying-and-testing.md +++ b/contracts/precompiled-contracts/creating-a-dynamic-contract-simple/deploying-and-testing.md @@ -1,10 +1,10 @@ --- -description: How to deploy a contract and write tests for it. +description: How to deploy a contract with BDK and write tests for it --- # Deploying and testing -Now that the contract is finished and implemented, all that's left to do is deploy it. Though we would also suggest writing some tests to make sure it's working as intended. This subchapter will explain how to do both. +Now that the contract is finished and implemented, all that's left to do is deploy it. Though we would also suggest generating the ABI of your contracts and writing some tests to make sure they're working as intended. This subchapter will explain how to do both. ## Generating the ABI @@ -16,9 +16,9 @@ We have a tool that is compiled with the project that generates the ABI of your int main() { return JsonAbi::writeContractsToJson(); } ``` -Compile the project and run `./contractabigenerator`. All of your Dynamic Contracts registered in `src/contract/customcontracts.h` (in the `ContractTypes` tuple shown earlier in the previous step) will have their ABIs generated inside the `ABI` folder, as `.json`. +Compile the project and run `./contractabigenerator`. All of your Dynamic Contracts registered in `src/contract/customcontracts.h` (inside the `ContractTypes` tuple shown earlier) will have their ABIs generated inside the `ABI` folder, as `.json`. -If you want a finer control of which ABIs you want to generate, replace `ContractTypes` with the exact names of the contract classes that should be generated: +If you want a finer control of which ABIs you want to generate, replace `ContractTypes` with the exact names of the contract classes that should be generated (note the `ContractManager` contract will *always* have its ABI generated regardless, as it is a vital part of the project): ```cpp #include "utils/jsonabi.h" @@ -28,15 +28,9 @@ int main() { } ``` -The `ContractManager` contract will always have its ABI generated in both cases, as it is a vital part of the project. +## Testing the contract -## Deploying the Blockchain - -Go back to the project's root, compile and deploy the blockchain by running `./scripts/AIO-setup.sh`. See "Setting up the development environment" for more information. Your contract should be initialized alongside the blockchain, ready to be interacted with. - -## Testing the Contract - -We use the [catch2](https://github.com/catchorg/Catch2) framework to test our project as a whole, so it is expected to have at least a little bit of familiarity with how it works (it's pretty easy!). +We use the [catch2](https://github.com/catchorg/Catch2) framework to test our project as a whole, so it is expected to be a bit familiar with how it works (it's pretty easy!). For testing contracts specifically, we have developed our own test suite called `SDKTestSuite`, in `tests/sdktestsuite.hpp` (the `.cpp` file is for testing the test suite itself). Using this test suite is highly recommended, as it simulates the environment of the blockchain (e.g. creating blocks, advancing the chain, taking care of the state and its variables, etc.) to make sure contracts are behaving the way they would in the actual deployed blockchain. Check the [Doxygen](https://doxygen.nl) docs, as well as the contract test files in the `tests/contract` subfolder, for more information on how to use the test suite. @@ -123,6 +117,10 @@ namespace TSimpleContract { Keep in mind that we're not accessing the contract directly, we're interacting with it through `SDKTestSuite`, and that requires us to parse its inputs and outputs accordingly. -In order to run your tests, compile the project as you normally would, and then run `./src/bins/bdkd-tests/bdkd-tests -d yes [simplecontract]` from within your build directory. The `[simplecontract]` tag forces only these specific tests for SimpleContract to run (this is set in the `TEST_CASE()` lines in the example above). The `-d yes` flag makes it more verbose, showing exactly which test case is being run at the moment. +In order to run your tests, compile the project as you normally would, and then run `./src/bins/bdkd-tests/bdkd-tests -d yes [simplecontract]` from within your build directory. The `[simplecontract]` tag forces only these specific tests for `SimpleContract` to run (this is set in the `TEST_CASE()` lines in the example above). The `-d yes` flag makes it more verbose, showing exactly which test case is being run at the moment. As a bonus, even though this whole chapter is focused on precompiled contracts, the `SDKTestSuite` also has a function that can deploy a contract's bytecode directly, called `SDKTestSuite::deployBytecode()`, which takes a `Bytes` object as a parameter that represents the contract's bytecode. + +## Deploying the blockchain + +For real-world testing, go back to the project's root and deploy the blockchain. See "Developing with BDK" for more information. Your contract should be initialized alongside the blockchain, ready to be interacted with. diff --git a/precompiled-contracts/creating-a-dynamic-contract-simple/simple-contract-header.md b/contracts/precompiled-contracts/creating-a-dynamic-contract-simple/simple-contract-header.md similarity index 64% rename from precompiled-contracts/creating-a-dynamic-contract-simple/simple-contract-header.md rename to contracts/precompiled-contracts/creating-a-dynamic-contract-simple/simple-contract-header.md index ce73b48..5ce92a9 100644 --- a/precompiled-contracts/creating-a-dynamic-contract-simple/simple-contract-header.md +++ b/contracts/precompiled-contracts/creating-a-dynamic-contract-simple/simple-contract-header.md @@ -1,12 +1,12 @@ --- -description: Coding the Simple Contract's header file +description: Coding the SimpleContract's header file --- # Simple Contract Header Having created the `SimpleContract` files and registered them in CMake, let's implement the contract's header first, as most of the registration is done there. -## Declaring the Contract Class +## Declaring the contract class Open the header file (`simplecontract.h`) and add the following lines: @@ -37,9 +37,9 @@ This is a simple skeleton so we can start building the proper contract. From top Now we can declare the members of our contract. We have to pay attention to some rules described earlier, which we'll go through slowly, one part at a time. -## Declaring the Contract Variables +## Declaring contract variables -Variables MUST be `private` and MUST inherit one of the SafeVariable classes. So our class declaration would start with something like this: +Variables MUST be `private` and MUST inherit one of the SafeVariable classes (so the blockchain state can be aware of them). Our class declaration would start with something like this: ```cpp class SimpleContract : public DynamicContract { @@ -51,13 +51,13 @@ class SimpleContract : public DynamicContract { } ``` -Our three variables `name_`, `number_` and `tuple_` are respectively declared as a `SafeString`, `SafeUint256_t` and `SafeTuple`. Notice we can use primitive types inside the tuple just fine, as the contract's inner variables already ensure commit/revert safety due to their types being inherited from their SafeVariable counterparts. +Our three variables `name_`, `number_` and `tuple_` are respectively declared as a `SafeString`, `SafeUint256_t` and `SafeTuple`. Notice we can use primitive types inside the tuple just fine, as the SafeVariable type itself ensures the necessary commit/revert safety of its children types. -## Declaring the Contract Functions +## Declaring contract functions -Functions in general can return either `void` or an ABI-supported C++ type (see the correlation on Solidity ABI). The main difference between both is that **view** functions MUST be `const` (e.g. `getName() const;`), while **non-view** functions MUST NOT be `const` (e.g. `setName(std::string name);`). +Functions in general can return either `void` or an ABI-supported C++ type (see the correlations on the "Solidity ABI" section). The main difference is that **view** functions MUST be `const` (e.g. `getName() const;`), while **non-view** functions MUST NOT be `const` (e.g. `setName(std::string name);`). -For the two registering functions, `registerContract()` MUST be `public static void`, and `registerContractFunctions()` MUST be `private void override`. For the `dump()` function, it MUST be `const override` and return a `DBBatch` object. +For the two registering functions, `registerContract()` MUST be `public static void`, and `registerContractFunctions()` MUST be `private void override`. For the `dump()` function, it MUST be `const override` and return a `DBBatch` object: ```cpp class SimpleContract : public DynamicContract { @@ -85,7 +85,7 @@ class SimpleContract : public DynamicContract { Just like with the tuple variable, we can use primitive types and returns on functions just fine, for the same reasons stated above. This also extends to constructors, whch we'll see below. -## Declaring the Contract Constructor +## Declaring contract constructors Like any C++ derived class, we must call its base class constructor and pass the proper arguments to it (besides the arguments for the derived class itself) so it can be constructed properly. Any contract derived from `DynamicContract` MUST have *two* constructors - one for creating a new contract from scratch, and another for loading the contract from the database: @@ -130,7 +130,7 @@ DynamicContract( ) : BaseContract(address, db) {}; ``` -`address`, `creator`, `chainId` and `db` are internal variables used by the base class, and should *always* be declared *last*. They are equivalent to: +`address`, `creator`, `chainId` and `db` are internal variables used by the base class, and should *always* be declared *last* (as in, *after* the contract's own variables if they exist). They are equivalent to: | DynamicContract constructor argument | Taken from | | ------------------------------------ | ------------------------------------------- | @@ -139,11 +139,11 @@ DynamicContract( | chainId | this->options->getChainID() | | DB | this->db | -Keep in mind that, when calling the base class constructor later on, the contract's name argument (in this case, "SimpleContract") MUST be EXACTLY the same as your contract's class name. This is because the name is used to load the contract type from the database, so incorrectly naming it will result in a segfault at load time. +Keep in mind that, when defining the base class constructor later on in the `.cpp` file, the contract's name argument (in this case, "SimpleContract") MUST be EXACTLY the same as your contract's class name. This is because the name is used to load the contract type from the database, so incorrectly naming it will result in a segfault at load time. -## Declaring the Contract Events +## Declaring contract events -Events MUST be `public`, `void`, non-`const`, AND call an internal function named `emitEvent()`, which is available to all Dynamic Contracts and does the proper emission of the event. +If your contract has events, they MUST be `public`, `void`, non-`const`, AND call an internal function named `emitEvent()`, which is available to all Dynamic Contracts and does the proper emission of the event: ```cpp class SimpleContract : public DynamicContract { @@ -174,14 +174,15 @@ class SimpleContract : public DynamicContract { `emitEvent()` requires at most three arguments: -* The event name (you can just pass `__func__` which is the same as `this->emitEvent("nameChanged", ...)`) +* The event name (you can just pass `__func__` which is the same as e.g. `this->emitEvent("nameChanged", ...)`) * **(Optional)** A tuple of `EventParam` objects representing the event's arguments, where: * The first element is the argument type (e.g. `name` is a `std::string`, `number` is a `uint256_t`, `tuple` is a `std::tuple`) - * The second element is a bool that indicates whether the argument should be indexed or not. + * The second element is a bool that indicates whether the argument should be indexed or not * If your event has no arguments at all you can omit it or pass an empty tuple instead (e.g. `this->emitEvent(__func__, std::make_tuple())`) -* **(Optional)** A flag that indicates whether the event is anonymous or not. Events are non-anonymous by default (the flag defaults to `false`), so if you wish you can omit this (which is the case for our example, it is equivalent to `this->emitEvent(__func__, std::make_tuple(name), false)` - if it was an anonymous event, the last flag would be `true` instead) +* **(Optional)** A flag that indicates whether the event is anonymous or not + * Events are non-anonymous by default (the flag defaults to `false`), so if you wish you can omit this (which is the case for our example, it is equivalent to `this->emitEvent(__func__, std::make_tuple(name), false)` - if it was an anonymous event, the last flag would be `true` instead) -## Registering the Contract Class +## Registering the contract class One last thing we have to do within our header is properly register the contract class. All Dynamic Contracts use templating to automate most of the hard work for both our contract and the BDK. @@ -198,79 +199,82 @@ class SimpleContract : public DynamicContract { } ``` -Then, implement the `registerContract()` function declared previously by calling another function from `ContractReflectionInterface` called `registerContractMethods()`, passing a few arguments to it, like this: +Then, implement the `registerContract()` function declared previously by calling a function from the `DynamicContract` class called `registerContractMethods()`, passing a few arguments to it, like this: ```cpp class SimpleContract : public DynamicContract { public: // ... static void registerContract() { - ContractReflectionInterface::registerContractMethods< - SimpleContract, const std::string&, const uint256_t&, const std::tuple&, - ContractManagerInterface&, - const Address&, const Address&, const uint64_t&, const DB& - >( - std::vector{"name_", "number_", "tuple_"}, - std::make_tuple("getName", &SimpleContract::getName, FunctionTypes::View + static std::once_flag once; + std::call_once(once, []() { + DynamicContract::registerContractMethods( + std::vector{"name_", "number_", "tuple_"}, + std::make_tuple("getName", &SimpleContract::getName, FunctionTypes::View , std::vector{}), - std::make_tuple("getNumber", &SimpleContract::getNumber, FunctionTypes::View + std::make_tuple("getNumber", &SimpleContract::getNumber, FunctionTypes::View , std::vector{}), - std::make_tuple("getTuple", &SimpleContract::getTuple, FunctionTypes::View, std::vector{}), - std::make_tuple("setName", &SimpleContract::setName, FunctionTypes::NonPayable + std::make_tuple("getTuple", &SimpleContract::getTuple, FunctionTypes::View, std::vector{}), + std::make_tuple("setName", &SimpleContract::setName, FunctionTypes::NonPayable , std::vector{"argName"}), - std::make_tuple("setNumber", &SimpleContract::setNumber, FunctionTypes::NonPayable + std::make_tuple("setNumber", &SimpleContract::setNumber, FunctionTypes::NonPayable , std::vector{"argNumber"}), - std::make_tuple("setTuple", &SimpleContract::setTuple, FunctionTypes::NonPayable, std::vector{"argTuple"}) - ); + std::make_tuple("setTuple", &SimpleContract::setTuple, FunctionTypes::NonPayable, std::vector{"argTuple"}) + ); + // ... + }); } } ``` -Inside the chevrons (`registerContractMethods<...>()`): - -* The first argument is the contract's class type (in this case, `SimpleContract`) -* The following arguments are all the types of arguments inside its first constructor (from scratch - the same ones that were put inside `ConstructorArguments`) - you can copy-paste the constructor's arguments as-is and take out the names - * It's important to remember that contract arguments should be declared *before* the internal arguments used by the base class constructor, as stated in the previous step +The `std::call_once` function is used to guarantee contract registering will be done exactly once, even if called from several threads. Inside the chevrons (`registerContractMethods<...>()`) there's only one argument needed, which is the contract's class type itself (in this case, `SimpleContract` - *no quotes ""*). Inside the parentheses (`registerContractMethods<>(...)`): -* The first argument is a string vector that is a list of all the exact names of the arguments in the constructor, each one separated by a comma - in this case, `"name_"`, `"number_"` and `"tuple_"` - * Note this does not include arguments used by the base class' constructor (e.g. `interface`, `address`, `creator`, `chainId`, `db`), only the ones inherent to the contract itself -* The following arguments are tuples, one for each function from the contract, that contain respectively: +* The first argument is a string vector that is a list of all the exact names of the arguments in the constructor, each one separated by a comma - in this case, `"name_"`, `"number_"` and `"tuple_"`. If the contract has no variables in it, you can pass an empty list instead + * This does not include arguments used by the base class' constructor (e.g. `interface`, `address`, `creator`, `chainId`, `db`), only the ones inherent to the contract itself +* The following arguments are tuples, one for each implemented contract function, that contain respectively: * The exact name of the function (`"getName"`) * A reference to the function itself (`&SimpleContract::getName`) - if you happen to have one or more overloads of the same function, you may need to specify which function is which using `static_cast` (e.g. `getNumber()` would be registered as `static_cast(&SimpleContract::getNumber)`, while `getNumber(const uint256_t&)` would be registered as `static_cast(&SimpleContract::getNumber)`) - * The [state mutability](https://docs.soliditylang.org/en/latest/contracts.html#state-mutability) of said function, accessed by a `FunctionTypes` enum (available values are `"View"`, `"NonPayable"` and `"Payable"`) - * A string vector that is the list of arguments that the function takes, if any (or a blank list if none) + * The [state mutability](https://docs.soliditylang.org/en/latest/contracts.html#state-mutability) of said function, accessed by a `FunctionTypes` enum - available values are `"View"`, `"NonPayable"` and `"Payable"` + * A string vector that is the list of arguments that the function takes, if any (or an empty list if none) + +Every contract MUST have both `ConstructorArguments` AND `registerContract()` implemented in order to be registered, even if `ConstructorArguments` is empty (has no arguments at all). -Every contract MUST have both `ConstructorArguments` and `registerContract()` implemented in order to be registered, even if `ConstructorArguments` is empty (has no arguments at all). +## Registering contract events -If your contract has events, you should also register them so you can generate their ABI later on. Events are registered the same way as the contract class and its functions, but using a separate function from `ContractReflectionInterface` called `registerContractEvents()`. +If your contract has events, you should also register them so you can generate their ABI later on. Events are registered the same way as the contract class and its functions, but using a separate function from `ContractReflectionInterface` called `registerContractEvents()`: ```cpp class SimpleContract : public DynamicContract { public: // ... static void registerContract() { - // registerContract() called here - ContractReflectionInterface::registerContractEvents( - std::make_tuple("nameChanged", false, &SimpleContract::nameChanged, std::vector{"name"}), - std::make_tuple("numberChanged", false, &SimpleContract::numberChanged, std::vector{"number"}), - std::make_tuple("tupleChanged", false, &SimpleContract::tupleChanged, std::vector{"tuple"}) - ); + static std::once_flag once; + std::call_once(once, []() { + // DynamicContract::registerContractMethods() called here + ContractReflectionInterface::registerContractEvents( + std::make_tuple("nameChanged", false, &SimpleContract::nameChanged, std::vector{"name"}), + std::make_tuple("numberChanged", false, &SimpleContract::numberChanged, std::vector{"number"}), + std::make_tuple("tupleChanged", false, &SimpleContract::tupleChanged, std::vector{"tuple"}) + ); + }); } } ``` -Inside the chevrons (`registerContractEvents<...>()`), the only argument is the contract's class name (in this case, `SimpleContract`). +Inside the chevrons (`registerContractEvents<...>()`), like before, the only argument needed is the contract's class name (in this case, `SimpleContract` - again, *no quotes ""*). + +Inside the parentheses (`registerContractEvents<>(...)`), much like the other register function, there is one tuple per implemented event, that contain respectively: -Inside the parentheses (`registerContractEvents<>(...)`): +* The event's name (`"nameChanged"`) +* A flag indicating whether the event is anonymous or not +* A reference to the event itself (`&SimpleContract::nameChanged`) +* A string vector that is the list of parameters that the event takes, if any (or an empty list if none) -* The first argument is the event's name (`"nameChanged"`) -* The second argument is a flag indicating whether the event is anonymous or not -* The third argument is a reference to the event itself (`&SimpleContract::nameChanged`) -* The last argument is a string vector that is the list of parameters that the event takes, if any (or a blank list if none) +## Registering the contract in the blockchain -Finally, we go to the `src/contract/customcontracts.h` file, include our contract's header and add it to the `ContractTypes` tuple. +Finally, we go to `src/contract/customcontracts.h`, include our contract's header and add it to the `ContractTypes` tuple so it can be compiled alongside the blockchain: ```cpp // ...some includes ... diff --git a/precompiled-contracts/creating-a-dynamic-contract-simple/simple-contract-source.md b/contracts/precompiled-contracts/creating-a-dynamic-contract-simple/simple-contract-source.md similarity index 51% rename from precompiled-contracts/creating-a-dynamic-contract-simple/simple-contract-source.md rename to contracts/precompiled-contracts/creating-a-dynamic-contract-simple/simple-contract-source.md index 4a479db..17d5809 100644 --- a/precompiled-contracts/creating-a-dynamic-contract-simple/simple-contract-source.md +++ b/contracts/precompiled-contracts/creating-a-dynamic-contract-simple/simple-contract-source.md @@ -1,23 +1,23 @@ --- -description: Coding the Simple Contract's source file +description: Coding the SimpleContract's source file --- # Simple Contract Source -With the `SimpleContract` header, declarations and (most of the) registering done, now we can proceed to the implementation itself. +With the `SimpleContract` header, declarations and (most of the) registering done, now we can proceed to the contract's implementation. -## Defining the Contract Constructors and Dump Function +## Defining the contract constructors and dump function Open the source file (`simplecontract.cpp`) and `#include "simplecontract.h"` right at the beginning. The first thing we'll implement is the constructors and dumping function of our contract class. The implementation must follow a certain order of events: * The base `DynamicContract` constructor must be called and its respective arguments must be passed in order -* Contract SafeVariables must be accessed with `this` (e.g. `this->name`) and initialized accordingly with their values if necessary (e.g. directly from the constructor, or by fetching values from the database) -* Contract SafeVariables must call `commit()` manually to properly set their values to the values assigned before -* `registerContractFunctions()` must be called to properly register the contract's functions and events -* Contract SafeVariables must call `enableRegister()` manually so they can be set to be properly marked as "used" during contract calls (for the commit/revert logic to work) -* If anything happens during construction that would require throwing an exception, said throw should be done ***before*** calling `enableRegister()` on any contract SafeVariable - enabling registers should be the *last* thing done by the constructor to avoid heap-use-after-free errors caused by accessing variables that are accessed after a throw happens +* The contract's SafeVariables must be accessed with `this` (e.g. `this->name`) and initialized accordingly with their values if necessary (e.g. directly from the constructor, or by fetching values from the database) +* The contract's SafeVariables must call `commit()` to properly set their values to the values they were assigned during construction +* `registerContractFunctions()` must be called to properly register the contract's functions and events (if there are any) +* The contract's SafeVariables must call `enableRegister()` so they can be set to be properly marked as "used" during contract calls (this is required for the commit/revert logic to work) +* If anything happens during construction that would require throwing an exception, said throw should be done ***before*** calling `enableRegister()` on any SafeVariable - enabling registers should be the *last* thing done by the constructor to avoid heap-use-after-free errors caused by variables being accessed after a throw happens Our source file will look something like this: @@ -25,6 +25,7 @@ Our source file will look something like this: #include "simplecontract.h" #include "../../utils/uintconv.h" +#include "../../utils/strconv.h" #include "../../utils/utils.h" SimpleContract::SimpleContract( @@ -45,7 +46,7 @@ SimpleContract::SimpleContract( this->number_.commit(); this->tuple_.commit(); - registerContractFunctions(); + registerContractFunctions(); // DO NOT THROW AFTER THIS LINE! this->name_.enableRegister(); this->number_.enableRegister(); @@ -67,7 +68,7 @@ SimpleContract::SimpleContract( this->number_.commit(); this->tuple_.commit(); - registerContractFunctions(); + registerContractFunctions(); // DO NOT THROW AFTER THIS LINE! this->name_.enableRegister(); this->number_.enableRegister(); @@ -75,7 +76,7 @@ SimpleContract::SimpleContract( } DBBatch SimpleContract::dump() const { - DBBatch dbBatch; + DBBatch dbBatch = BaseContract::dump(); dbBatch.push_back(StrConv::stringToBytes("name_"), StrConv::stringToBytes(this->name_.get()), this->getDBPrefix()); dbBatch.push_back(StrConv::stringToBytes("number_"), UintConv::uint256ToBytes(this->number_.get()), this->getDBPrefix()); dbBatch.push_back(StrConv::stringToBytes("tuple_name"), StrConv::stringToBytes(get<0>(this->tuple_)), this->getDBPrefix()); @@ -84,19 +85,21 @@ DBBatch SimpleContract::dump() const { } ``` -Notice that, in the first constructor, we use `SimpleContract` as the `contractName` argument in the base `DynamicContract` constructor. As stated in the previous subchapter, this match is a **requirement**, otherwise it will result in a segfault. Both constructors initialize the inner variables of the contract - the first one using the arguments directly, and the second one loading them directly from the database. +Notice that, in the first constructor, we use `SimpleContract` as the `contractName` argument in the base `DynamicContract` constructor. As stated previously, this match is a **requirement**, otherwise it will result in a segfault. Both constructors initialize the inner variables of the contract - the first one using the arguments directly, and the second one loading them directly from the database. -The dumping function is called periodically and is responsible for collecting the contract variables' values and send them back to `DumpManager`, which will save those values in the database so that they can be loaded later by the second constructor, when `ContractManager` is being constructed. `getDBPrefix()` is a getter for the contract's own prefix in the database, which would be equivalent to `DBPrefix::contracts` + the contract's address. +The dumping function is called periodically and is responsible for collecting the contract variables' values and sending them back to `DumpManager` (the internal class that does the actual database dump, see "BDK implementation" for more details), which will save those values in the database so that they can be loaded later by the second constructor, when `ContractManager` is being constructed. `getDBPrefix()` is a getter for the contract's own prefix in the database, which would be equivalent to `DBPrefix::contracts` + the contract's address. Keep in mind that **the database stores data as raw bytes** - this is why we use the respective conversion functions from Utils when saving (`XyzToBytes()`) and loading (`bytesToXyz()`) variables. -## Defining the Contract Functions +Also keep in mind that **you should always dump every parent contract class' data** as well - this is why we do `BaseContract::dump()` in the code above, since Dynamic Contracts inherit directly from `BaseContract` and their metadata (creator, timestamp, etc.) is stored there. Another example would be the `NativeWrapper` contract - it inherits directly from the `ERC20` class and depends on its variables to construct itself, so you must dump *both* (see `src/contract/templates/nativewrapper.cpp` for more details). Forgetting to do this may result in undefined behaviour when loading the contract's data from the database. -Now, let's implement the proper functions of our contract - first, the **view** functions (that only read and never change the contract's variables when called), then, the **non-view** functions (that do change the contract's variables when called). +## Defining contract functions + +Now let's implement the proper functions of our contract - first, the **view** functions (that only read and never change the contract's variables when called), then, the **non-view** functions (that do change the contract's variables when called). **View** functions MUST be `const`, while **non-view** functions MUST NOT be `const`, and both functions can return either `void` or one of the ABI-supported types. -In our case, we have three view functions which would be `getName()`, `getNumber()` and `getTuple()`, which are the getters for the variables of our contract - `name`, `number` and `tuple`, respectively. We can return the inner data from any SafeVariable by calling the `get()` function, like this: +In our case, we have three view functions which would be `getName()`, `getNumber()` and `getTuple()`, which are the getters for the variables of our contract - `name_`, `number_` and `tuple_`, respectively. We can return the inner data from any SafeVariable by calling the `get()` function, like this: ```cpp std::string SimpleContract::getName() const { return this->name_.get(); } @@ -106,9 +109,11 @@ std::tuple SimpleContract::getTuple() const { } ``` +Note the templated `get<>()` functions in the tuple are *not* C++'s `std::get<>()` implementation, but rather the SafeTuple's own implementation. This is due to how SafeVariables work internally - as it is custom functionality, the C++ Standard Library is unaware of it, thus using `std::` here is not going to work (or even compile for that matter). + For the three non-view functions we have, which would be the setters (`setName()`, `setNumber()` and `setTuple()` respectively), we must also check that whoever is calling those functions is the actual creator of the contract, as we want to prevent unwanted calls from other addresses (this is how it's coded in the original Solidity code reference). We can do that by calling `getCaller()` and `getContractCreator()`, respectively, to access the address of the caller and the address of the contract creator, and then we check if both addresses are the same. -If your contract has events, you can emit them by simply calling them like they were any other function (which they actually are if you think about it!). The only thing you have to pay attention to is that **events can ONLY be emitted from non-view functions**, due to how const correctness works in C++ (view functions are `const`, so trying to emit an event from one of them will result in a compilation error). +If your contract has events, you can emit them by simply calling them like they were any other function (they actually are if you think about it!). The only thing you have to pay attention to is that **events can ONLY be emitted from NON-view functions**, due to how const correctness works in C++ (view functions are `const`, so trying to emit an event from one of them will result in a compilation error). ```cpp void SimpleContract::setName(const std::string& argName) { @@ -136,28 +141,24 @@ void SimpleContract::setTuple(const std::tuple& argTuple } ``` -Notice when we use `get<>()` on the tuple, we're NOT calling `std::get<>()`, but rather the `SafeTuple`'s own `get<>()` implementation. This is due to how SafeVariables work internally - as it is custom functionality, the STD library is unaware of it, thus `std::get<>()` is not going to work (or even compile for that matter). - -## Registering the Contract Functions +## Registering contract functions -After all functions are implemented, we must implement one more - `registerContractFunctions()`, which is responsible for registering the other functions so they can be called later by a transaction or an RPC `eth_call`. +After all functions are implemented, we must implement one more - `registerContractFunctions()`, which is responsible for registering the other functions so they can be called later by a transaction or an RPC `eth_call`. Their respective functors/signatures will be stored in an internal map, allowing a given transaction to call any function within that contract. Registration is done within try/catch blocks internally, which allows the protection of SafeVariables against any exceptions thrown by the function. -The first thing it should do is call `registerContract()` right away, so it's guaranteed that the contract itself will be registered before its functions. - -When registering the contract's functions, their respective functors/signatures will be stored in an internal map, allowing a given transaction to call any function within that contract. Registration is done within try/catch blocks internally, which allows the protection of SafeVariables against any exceptions thrown by the function. - -As for the functions themselves, they are registered by calling `this->registerMemberFunction()` for each function your contract has (NOT including events), always passing four arguments to it: the function's name, a reference to the function, its state mutability, and `this` (a pointer to the contract itself). +The first thing it should do is call `registerContract()` right away, so it's guaranteed that the contract itself will be registered before its functions. As for the functions themselves, they are registered by calling `this->registerMemberFunctions()` and passing to it several tuples - one for each function your contract has (NOT including events). Each tuple needs four arguments - the function's name, a reference to the function, its state mutability, and `this` (a pointer to the contract itself), as follows: ```cpp void SimpleContract::registerContractFunctions() { registerContract(); - this->registerMemberFunction("getName", &SimpleContract::getName, FunctionTypes::View, this); - this->registerMemberFunction("getNumber", &SimpleContract::getNumber, FunctionTypes::View, this); - this->registerMemberFunction("getTuple", &SimpleContract::getTuple, FunctionTypes::View, this); - this->registerMemberFunction("setName", &SimpleContract::setName, FunctionTypes::NonPayable, this); - this->registerMemberFunction("setNumber", &SimpleContract::setNumber, FunctionTypes::NonPayable, this); - this->registerMemberFunction("setTuple", &SimpleContract::setTuple, FunctionTypes::NonPayable, this); + this->registerMemberFunctions( + std::make_tuple("getName", &SimpleContract::getName, FunctionTypes::View, this), + std::make_tuple("getNumber", &SimpleContract::getNumber, FunctionTypes::View, this), + std::make_tuple("getTuple", &SimpleContract::getTuple, FunctionTypes::View, this), + std::make_tuple("setName", &SimpleContract::setName, FunctionTypes::NonPayable, this), + std::make_tuple("setNumber", &SimpleContract::setNumber, FunctionTypes::NonPayable, this), + std::make_tuple("setTuple", &SimpleContract::setTuple, FunctionTypes::NonPayable, this) + ); } ``` -Note that the complete implementation of this contract has overloads on `getNumber()`, so you may see `static_cast(&SimpleContract::getNumber)` in place of simply `&SimpleContract::getNumber` - it's done this way so we know exactly which function we are referring to. Since this example is only partial and we only have one `getNumber()` function in it, we're referencing it here normally as simply `&SimpleContract::getNumber`. +Note that the complete implementation of this contract has overloads on `getNumber()`, so you may see `static_cast(&SimpleContract::getNumber)` in place of simply `&SimpleContract::getNumber` - it's done this way so we know exactly which function we are referring to. Since this implementation is a simplified version and we only have one `getNumber()` function in it, we reference it here simply as `&SimpleContract::getNumber`. If your contract has one or more overloads for the same function though, you should keep this in mind and cast them accordingly. diff --git a/precompiled-contracts/creating-a-protocol-contract-advanced.md b/contracts/precompiled-contracts/creating-a-protocol-contract.md similarity index 91% rename from precompiled-contracts/creating-a-protocol-contract-advanced.md rename to contracts/precompiled-contracts/creating-a-protocol-contract.md index 3cc7d9e..384b557 100644 --- a/precompiled-contracts/creating-a-protocol-contract-advanced.md +++ b/contracts/precompiled-contracts/creating-a-protocol-contract.md @@ -1,14 +1,14 @@ --- -description: A primer on what to expect when creating Protocol Contracts. +description: A primer on what to expect when creating Protocol Contracts --- -# Creating a Protocol Contract (Advanced) +# Creating a Protocol Contract Protocol Contracts do not offer the same level of ease of use and security as Dynamic Contracts. However, by sacrificing some of these features, they enable more complex functionalities that would be impossible to achieve with Dynamic Contracts alone. For instance, Protocol Contracts can call functions without an active transaction call when processing a block, access system files and make requests to other nodes. Introducing this new layer for processing information inherently carries risks, particularly in a decentralized network. To maintain a stable network, you must ensure that every node can execute the same operation, given a previous context. In a typical VM blockchain, this is achieved by allowing operations to be callable only by transactions and packaging these transactions into a block. When processing a block, all transactions that call a contract attempt to execute, ensuring that all operations are performed consistently within the given context. -As a developer of a Protocol Contract, it is crucial to manage the processing of information in a way that ensures different nodes do not arrive at different results when given the same context. The challenge lies in defining "context". In a VM blockchain, the context can be defined based on a block and all its past blocks. However, in AppLayer, Protocol Contracts that process beyond a transaction call are actively changing their own context. Consequently, it is essential to guarantee that the context remains consistent across all nodes. +As a developer of a Protocol Contract, it is crucial to manage the processing of information in a way that ensures different nodes do not arrive at different results when given the same context. The challenge lies in defining "context". In a VM blockchain, the context can be defined based on a block and all its past blocks. However, in AppLayer, Protocol Contracts can process beyond a transaction call and are actively changing their own context. Consequently, it is essential to guarantee that the context remains consistent across all nodes. By adhering to the following best practices, you can create Protocol Contracts that maintain consistent behavior across nodes, and ensure the stability and security of your blockchain network. Remember that the key to developing robust Protocol Contracts lies in managing the context and ensuring that all nodes in the network can process information the same way. @@ -44,7 +44,7 @@ Under `src/contract/contractmanager.h`, you will find a global map which contain ## Pay attention to the contract's state -It is required to pay attention to the state of the contract and how nodes will interact between each other. For example, if you have a contract that is doing parallel processing of a task, you need to make sure that this task can be replicated with the same result in any node. Not taking care of the state of a contract will eventually lead to undefined behavior, and that's a hornet's nest nobody wants to touch, right? +It is required to pay attention to the state of the contract and how nodes will interact between each other. For example, if you have a contract that is doing parallel processing of a task, you need to make sure that this task can be replicated with the same result in any node. Not taking care of the state of a contract will eventually lead to undefined behavior, and that's a hornet's nest nobody wants to touch. ## Override `ethCall()` and take care of your revert conditions @@ -52,7 +52,7 @@ It is necessary to override the `ethCall()` function in order to make functions ## Pay attention to the contract state's commit status -When the contract state's "commit" flag is set to `false`, it means that the current call is trying to simulate if it's going to throw or not, but if `true`, it means that the current call is trying to commit to the state, and you should do it respectively if it doesn't throw. +When the contract state's `commit` flag is set to `false`, it means that the current call is trying to simulate if it's going to throw or not, but if `true`, it means that the current call is trying to commit to the state, and you should do it respectively if it doesn't throw. ## Add a reference to `ContractManager` diff --git a/contracts/precompiled-contracts/dynamic-and-protocol-contracts.md b/contracts/precompiled-contracts/dynamic-and-protocol-contracts.md new file mode 100644 index 0000000..8094438 --- /dev/null +++ b/contracts/precompiled-contracts/dynamic-and-protocol-contracts.md @@ -0,0 +1,23 @@ +--- +description: Which kinds of precompiled contracts can be coded and managed with the BDK +--- + +# Dynamic and Protocol Contracts + +AppLayer's BDK offers two main classes of contracts: **Dynamic Contracts** and **Protocol Contracts**. The differences between both classes come from how they are created and managed within the BDK. + +## Dynamic Contracts (recommended) + +Dynamic Contracts are the recommended type for usage with AppLayer. They are *managed by the state* and can only be created by `ContractManager`, which enables the chain owner to create an unlimited number of contracts. + +They can also *use special types called **SafeVariables*** (explained further) - an additional layer of protection that allows better control over whether variable changes are committed to the state or automatically reverted when necessary (e.g. when a transaction fails). + +Dynamic Contracts can *only be called when a block is being processed* and are *directly loaded into memory*, working very similarly to Solidity contracts (including *event emission*). + +## Protocol Contracts + +Protocol Contracts are *directly integrated into the blockchain*, therefore not linked to the `ContractManager` class and not contained by the state, which removes some restrictions but adds others (explained further). + +This makes them able to be designed to *process information beyond transaction calls*, like communicating directly with other nodes, accessing files within the current system, and even automatically calling themselves when certain conditions are met - anything is possible, as long as you don't break your own blockchain (more on that later). + +As a downside to that, unlike Dynamic Contracts, they *cannot use SafeVariables nor emit events*. This means it's up to the developer (for the most part) to handle the contract's variables and their commit/revert logic within the blockchain's source code. diff --git a/contracts/precompiled-contracts/dynamic-contract-templates.md b/contracts/precompiled-contracts/dynamic-contract-templates.md new file mode 100644 index 0000000..cd4267a --- /dev/null +++ b/contracts/precompiled-contracts/dynamic-contract-templates.md @@ -0,0 +1,43 @@ +--- +description: Which kinds of Dynamic Contract templates and precompiles are offered by AppLayer +--- + +# Dynamic Contract Templates and Precompiles + +AppLayer's BDK provides ready-to-use templates for the following Dynamic Contracts: + +* Under `src/contract/templates/standards`: + * `ERC20` (template for an ERC20 token) + * `ERC721` (template for an ERC721 token) + * `ERC721URIStorage` (template for managing ERC721 token storage, converted directly from OpenZeppelin) + * `IERC721Receiver` (template for an interface that enables safeTransfer support for ERC721 tokens, converted directly from OpenZeppelin) +* Under `src/contract/templates/dexv2`: + * `DEXV2Factory` (template for a DEX factory) + * `DEXV2Library` (namespace for commonly used DEX functions) + * `DEXV2Pair` (template for a DEX contract pair) + * `DEXV2Router02` (template for a DEX contract router) + * `UQ112x112` (namespace for dealing with fixed point fractions in DEX contracts) +* Under `src/contract/templates`: + * `ERC20Wrapper` (template for an ERC20 wrapper) + * `MintableERC20` (template for a mintable ERC20 token) + * `NativeWrapper` (template for a native asset wrapper) + * `Ownable` (template for managing authorized access to certain contract calls, converted directly from OpenZeppelin) + +We also have contract precompiles under the `src/contract/templates/precompiles` folder, inside the `precompiles` namespace. They are stateful in the C++ side (so you only have to include their headers) and stateless in the EVM side (so you have to call them through a specific address). Here's a list of the available precompiles and their respective addresses, as per the [official EVM Codes reference list](https://www.evm.codes/precompiled): + +* `ecrecover` (0x01) +* `sha256` (0x02) +* `ripemd160` (0x03) +* `modexp` (0x05) +* `blake2f` (0x09) + +There are also specific contracts that only exist for internal testing purposes and are not meant to be used as templates: + +* `BuildTheVoid` and `BTV*` (contracts specific to the on-chain game Build The Void) +* `ERC721Test` (derivative contract meant to test the capabilities of the ERC721 template) +* `Pebble` (contract specific to the on-chain NFT minting game Pebble) +* `RandomnessTest` (contract for testing random number generation) +* `SimpleContract` (what it says on the tin - a simple contract, used for both testing and teaching purposes) +* `SnailTracer` / `SnailTracerOptimized` (C++ conversions of the [SnailTracer](https://github.com/karalabe/snailtracer) contract, used for benchmarking purposes) +* `TestThrowVars` (contract meant to test SafeVariable commit/revert functionality using exception throwing) +* `ThrowTest*` (contracts meant to test nested call revert functionality) diff --git a/precompiled-contracts/how-to-code-a-precompiled-contract.md b/contracts/precompiled-contracts/how-to-code-a-precompiled-contract.md similarity index 70% rename from precompiled-contracts/how-to-code-a-precompiled-contract.md rename to contracts/precompiled-contracts/how-to-code-a-precompiled-contract.md index 13eb4fb..0b6803a 100644 --- a/precompiled-contracts/how-to-code-a-precompiled-contract.md +++ b/contracts/precompiled-contracts/how-to-code-a-precompiled-contract.md @@ -1,10 +1,10 @@ --- -description: A primer on how precompiled contracts should be coded in AppLayer's BDK. +description: A primer on how precompiled contracts should be coded in AppLayer's BDK --- # How to code a precompiled contract -When creating precompiled contracts for AppLayer, there are a few rules that must be followed to ensure they work as intended. While each contract type has its own rules, some other rules apply to both. This will be explained and demonstrated further. +When creating precompiled contracts for AppLayer, there are a few rules that must be followed to ensure they work as intended. While each contract type has its own unique rules, some other rules are common and apply to both. This will be explained and demonstrated further. ## General contract rules @@ -13,7 +13,7 @@ As a general stance, contracts must: * Inherit from their respective base class, depending on their type (see below) and make sure you're passing the right arguments for their constructors * Implement, initialize and manage variables within the state and database, as well as the respective view and non-view functions that manage them when required - those variables should be loaded during contract construction and saved by means of an overriden dump function * Register callbacks for contract functions with their proper functors/signatures (if functions are called by an RPC `eth_call` or a transaction) -* Ensure that their assigned name and their own class name match - both contract constructors take a `contractName` string as an argument, i.e. if your contract is called "TestContract", your constructor's definition would be `TestContract(...) : DynamicContract(interface, "TestContract", ...)` - _both names HAVE to match_, otherwise a segfault may happen +* Ensure that their assigned name and their own class name match - both contract constructors take a `contractName` string as an argument, i.e. if your contract is called "TestContract", your constructor's definition would be `TestContract(...) : DynamicContract(interface, "TestContract", ...)` - *both names HAVE to match*, otherwise a segfault will happen * Declare view functions, non-view functions and events (if they exist) correctly, accoding to their specific types (explained further) ### Rules for Protocol Contracts @@ -31,21 +31,19 @@ Dynamic Contracts specifically must: * Provide a `ConstructorArguments` tuple with the contract's constructor argument types for registering the contract, and two registering functions: `registerContract()` and `registerContractFunctions()`, for contract metadata, variables, functions, and events (both of which should be called inside the contract's constructor) * Provide two constructors: one for creating the contract from scratch within `ContractManager`, and one for loading the contract from the database * Only allow contract creation through a transaction call to the `ContractManager` contract -* Develop functions for handling your contract's creation and logic -* Override `ethCall()` functions to register and properly call those functions +* Develop and properly register functions for handling your contract's creation and logic * Override the `dump()` function to properly save the contract's variables in the database * Set **all** of the contract's internal variables as `private`, inherit them from one of the many SafeVariable classes, and always reference them with `this` to ensure correct semantics - e.g. `string name` and `uint256 value` in Solidity should be `SafeString name` and `SafeUint256_t value` in C++, respectively - referencing them in your definition would be `this->name`, `this->value`, so on and so forth * Allow loops using containers such as `SafeUnorderedMap`, but keep in mind how safe containers work - * e.g. when you access a key from a `SafeUnorderedMap`, it'll check if it exists and copy _only_ the key, not the entire map or its value - thus when iterating a loop, you can't assume the "temporary" value is the original one - * We recommended you only loop inside _view_ functions to ensure value safety, but you can do it on non-view functions as well, just be careful when doing so + * e.g. when you access a key from a `SafeUnorderedMap`, it'll check if it exists and copy *only* the key, not the entire map or its value - thus when iterating a loop, you can't assume the "temporary" value is the original one + * We recommended you only loop inside *view* functions to ensure value safety, but you can do it on non-view functions as well, just be careful when doing so * Trigger state changes only via transaction calls to contract functions -* Call `updateState(true)` at the end of the contract's constructor ## Inherited functions and variables -Every contract within AppLayer's BDK inherits from the following classes, which means they can use their functions anywhere in their logic. Check the [Doxygen](https://doxygen.nl) docs for more details on each function's implementation, parameters, returns and overloads. +Every contract in the BDK inherits from the following classes, which means they can use their functions anywhere in their logic. Check the [Doxygen](https://doxygen.nl) docs for more details on each function's implementation, parameters, returns and overloads. -* **ContractGlobals** - global variables accessible through transaction/RPC calls, set by the State when calling the contract +* **ContractGlobals** - global variables accessible through transaction/RPC calls, set by the state when calling the contract | Function | Description | | ----------------- | --------------------------------------------------- | @@ -58,22 +56,25 @@ Every contract within AppLayer's BDK inherits from the following classes, which | Function | Description | | --------- | ---------------------------- | -| getOrigin | Get the transaction's origin | | getCaller | Get the transaction's caller | | getValue | Get the transaction's value | * **BaseContract** - base class for all contracts, provides and stores the respective contract's information -| Function | Description | -| ------------------ | ------------------------------------------------------------------- | -| getContractAddress | Get the contract's address | -| getContractCreator | Get the contract's owner | -| getContractChainId | Get the contract's chainId | -| getContractName | Get the contract's name | -| getDBPrefix | Get the contract's database prefix | -| getNewPrefix | Same as getDBPrefix() but with a user-defined prefix appended to it | - -**For Dynamic Contracts specifically**, you can also use the following: +| Function | Description | +| -------------------------- | ------------------------------------------------------------------- | +| getContractAddress | Get the contract's address | +| getContractCreator | Get the contract's owner | +| getContractChainId | Get the contract's chainId | +| getContractName | Get the contract's name | +| getDBPrefix | Get the contract's database prefix | +| getNewPrefix | Same as getDBPrefix() but with a user-defined prefix appended to it | +| getOrigin | Get the transaction's origin | +| getNonce | Get the address' nonce | +| getBlockNumberObservers | Get the contract's block number observers | +| getBlockTimestampObservers | Get the contract's block timestamp observers | + +* **DynamicContract** - for Dynamic Contracts specifically, you can also use the following: | Function | Description | | ------------------------ | -------------------------------------- | diff --git a/contracts/precompiled-contracts/managing-precompiled-contracts.md b/contracts/precompiled-contracts/managing-precompiled-contracts.md new file mode 100644 index 0000000..9119632 --- /dev/null +++ b/contracts/precompiled-contracts/managing-precompiled-contracts.md @@ -0,0 +1,25 @@ +--- +description: How are precompiled contracts managed within the BDK +--- + +# Managing precompiled contracts + +Contracts in the BDK are managed by a few classes working together (check the `src/core` and `src/contract` folders for more info on each class): + +* `State` (`src/core/state.h`) is responsible for owning all the Dynamic Contracts registered in the blockchain (which you can get a list of by calling the class' `getCppContracts()` and/or `getEVMContracts()` functions, depending on which ones you need), as well as their global variables (name, address, owner, balances, etc.) +* `Storage` (`src/core/storage.h`) is responsible for properly storing contract data, such as emitted events and the transactions that triggered them +* `ContractManager` (`src/contract/contractmanager.h`) is responsible for solely creating, registering and passing contracts to the state (with the `ContractFactory` namespace providing helper functions to do so) +* `ContractHost` (`src/contract/contracthost.h`) is responsible for allowing contracts to interact with each other and enabling them to modify balances - this kind of inter-communication is done by intercepting calls of functions from registered contracts (if the functor/signature matches), which is done through either an `eth_call` request or a transaction processed from a block +* `ContractStack` (`src/contract/contractstack.h`) is responsible for managing alterations of data, like contract variables and account balances during nested contract call chains, by keeping a stack of changes made during a contract call, and automatically committing or reverting them in the account state when required (for non-view functions) + +The `ContractManager` class is a *Protocol Contract* by itself, but it does not own or create any Protocol Contracts - they are created during blockchain initialization, and a reference to each of them is stored within the class, allowing it to access them directly. Dynamic Contracts, however, are fully owned and stored by the state in an internal map. This ensures that each contract has a unique address, which is derived using a similar scheme as an EVM. + +Because of this, we don't necessarily need to know the type of the contract stored within the pointer, only during either creating it or loading it from the database. This is why all contracts inherit the `BaseContract` class, as it contains a `name_` variable specifying the name of the contract. This name must be the same as the name of the class, as it is used to identify the contract during loading and creation. + +Using `ContractHost` for contract inter-communication instead of accessing the state directly allows us to do it in an isolated and secure way. For example, if we want to send tokens from a contract to another address, we don't access the balance directly from the state. Every time a function is called by a **user** (not another contract), the balances map available for contracts is empty, only populating what is currently being accessed and not previously available. When modifying the balance, only the mapping within `ContractStack` is modified, therefore allowing for multiple nested contract functions to revert in an atomic fashion while not affecting either the state or other contracts altogether. + +## The BaseContract class + +The `BaseContract` class, declared in `src/contract/contract.h`, is the base class from which all contracts derive from. This class holds all the [Solidity global variables](https://docs.soliditylang.org/en/v0.8.17/units-and-global-variables.html), besides variables common among these contracts (such as contract address). Have a look at the header file for further reference on its structure. + +Regarding the `callContractWithTransaction` and the `ethCallContract` functions in the previous example, the former is used by the state when calling from `processNewBlock()`, while the latter is used by RPC to answer for `eth_call`. Strings returned by `ethCallContract` are hex strings encoded with the desired function result. diff --git a/precompiled-contracts/safevariables-and-commit-revert-logic.md b/contracts/precompiled-contracts/safevariables-and-commit-revert-logic.md similarity index 85% rename from precompiled-contracts/safevariables-and-commit-revert-logic.md rename to contracts/precompiled-contracts/safevariables-and-commit-revert-logic.md index 892a632..25a35a6 100644 --- a/precompiled-contracts/safevariables-and-commit-revert-logic.md +++ b/contracts/precompiled-contracts/safevariables-and-commit-revert-logic.md @@ -1,10 +1,10 @@ --- -description: How we simulate Solidity commit/revert logic in precompiled contracts. +description: How we simulate Solidity commit/revert logic in precompiled contracts --- # SafeVariables and commit/revert logic -In C++, when you call a function that changes a variable and then throw an exception later, the changed variable is *not* reverted automatically. Consider the following example: +In C++, when you call a function that modifies a variable and then throws an exception, the changed variable is *not* reverted automatically. Consider the following example: ```cpp MyClass::updateValueAndThrow(const uint64_t key, const uint64_t value) { @@ -29,11 +29,11 @@ MyClass::updateValueAndThrow(const uint64_t key, const uint64_t value) { } ``` -For Protocol Contracts, that's most of the required context. Dynamic Contracts, however, automatically provide their functionality with the use of special types called **SafeVariables**, declared inside the `src/contract/variables` folder. Each Dynamic Contract includes a vector of references to SafeVariables, which is used to register used variables within a specific function call. +For Protocol Contracts, that's most of the required context. Dynamic Contracts, however, automatically provide this kind of functionality with the use of special types called **SafeVariables**, declared inside the `src/contract/variables` folder. Each Dynamic Contract includes a vector of references to SafeVariables, which is used to register used variables within a specific function call. All SafeVariables inherit from the `SafeBase` class, which adhere to the following rules: -* Have two internal variables: one for the current/original value and another for the previous/temporary value +* Have two internal variables: one for the current value and another for the previous value * Optionally, if required, an undo stack for dealing with more complex variables such as containers * Must override the `commit()` and `revert()` functions * `commit()` should keep the current value as-is and either discard the previous one or equal it to the current, depending on the implementation details diff --git a/understanding-contracts/solidity-abi.md b/contracts/solidity-abi.md similarity index 64% rename from understanding-contracts/solidity-abi.md rename to contracts/solidity-abi.md index b4d8e10..9a05bf3 100644 --- a/understanding-contracts/solidity-abi.md +++ b/contracts/solidity-abi.md @@ -6,17 +6,17 @@ description: >- # Solidity ABI -AppLayer is primarily a _native, pre-compiled_ blockchain, which means its main focus is to run without the need for an EVM. However, the vast majority of the smart contract ecosystem operates and depends on [Solidity](https://docs.soliditylang.org/en/latest) - not only the contracts themselves but also the data they share across each other. +AppLayer is primarily a *native, pre-compiled* blockchain, which means its main focus is to run without the need for an EVM. However, the vast majority of the smart contract ecosystem operates and depends on [Solidity](https://docs.soliditylang.org/en/latest) - not only the contracts themselves but also the data they share across each other. -When developing pre-compiled contracts, AppLayer makes use of an abstraction of Solidity's ABI encoding and decoding processes to properly translate between native and non-native data types. The **ABI** namespace (`src/contract/abi.h`) contains several functions for Solidity ABI-related operations, for managing and manipulating data in Solidity format. +When developing pre-compiled contracts, AppLayer makes use of an abstraction of Solidity's ABI encoding and decoding processes to properly translate between native and non-native data types. The **ABI** namespace (`src/contract/abi.h`) contains several functions for Solidity ABI-related operations, such as managing and manipulating data in Solidity format. -This is only an overview, check the [Doxygen](https://doxygen.nl) docs for more details on how those functions work. +This is only an overview, check the "BDK implementation" section and the [Doxygen](https://doxygen.nl) docs for more details on how it all works. -### Solidity types +## Data types -The **Types** enum contains the supported Solidity data types in the ABI. Each value has an intrinsic equivalency with both the Solidity data type and the native C++ data type that it represents. +We support the most common Solidity types in AppLayer by means of a **Types** enum - each value has an intrinsic equivalency with both the Solidity data type and the native data type that it represents. -Replace the **X** in "uintX" and "intX" with the desired size number. The ABI supports every size from 8 to 256 (inclusive), in multiples of 8 (e.g. 8, 16, 24, 32, 40, 48, ..., until 256) - in other words, `x <= 256 && x % 8 == 0`. **Enums are encoded as uint8**. +For example, here's a simple table correlating each Types enum value with its respective Solidity and C++ data type: | Enum | Solidity | C++ | | :--------: | :--------: | :-----------------------: | @@ -34,17 +34,19 @@ Replace the **X** in "uintX" and "intX" with the desired size number. The ABI su | stringArr | string\[] | std::vector\ | | enum | enum | uint8\_t | -### MethodDescription and EventDescription +Replace the **X** in "uintX" and "intX" with the desired size number. The ABI supports every size from 8 to 256 (inclusive), in multiples of 8 (e.g. 8, 16, 24, 32, 40, 48, ...) - in other words, `x <= 256 && x % 8 == 0`. **Note that the enum type itself is encoded as an uint8**. -The **MethodDescription** and **EventDescription** structs abstract, respectively, the structures for a given Solidity method and Solidity event, such as their name, type, inputs and outputs, state mutability, anonymity and indexations. Those are used extensively by `ContractReflectionInterface` and `JsonAbi`, to make it easier to pass data around when performing actions like registering the contract and generating ABI for events. +## Methods and events -### Encoding and decoding +The **MethodDescription** and **EventDescription** structs abstract, respectively, the structures for a given Solidity method and Solidity event, such as their name, type, inputs and outputs, state mutability, anonymity and indexations. Those are used extensively by the `ContractReflectionInterface` and `JsonAbi` classes to make it easier to pass data around when performing actions like registering the contract and generating its ABI. -Encoding and decoding ABI data is done by calling the `ABI::Encoder::encodeData()` and `ABI::Decoder::decodeData()` functions, respectively. The encode function asks for one or more native C++ types, returning a `Bytes` object that is the encoded ABI string. The decode function does the inverse, asking for a `Bytes` object with the ABI encoded data (and optionally an index for said data) and returning a `std::tuple` with the decoded native C++ types. +## Encoding and decoding -If encoding a function call, its signature (also called "functor" - the first 4 bytes of `keccak(functionSignature)`) can be encoded with `ABI::FunctorEncoder::encode()`, passing the function's name and its arguments along the template. +Encoding and decoding Solidity ABI data is done by calling the `ABI::Encoder::encodeData()` and `ABI::Decoder::decodeData()` functions, respectively. The encode function asks for one or more native types, returning a `Bytes` object that is the encoded ABI string. The decode function does the inverse, asking for a `Bytes` object with the ABI encoded data (and optionally an index for said data) and returning a `std::tuple` with the decoded native types. -Here's an example: +If encoding a function call, its signature (also called "functor" - the first 4 bytes of `keccak(functionSignature)`) can be encoded with `ABI::FunctorEncoder::encode()`, passing along the function's name and its arguments to the template. + +Here's an example in C++: ```cpp // Encoding diff --git a/evm-contracts/README.md b/evm-contracts/README.md deleted file mode 100644 index c85b812..0000000 --- a/evm-contracts/README.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -description: A primer on EVM smart contracts in the AppLayer ecosystem. ---- - -# EVM contracts - -Aside from native contracts, AppLayer can also execute Solidity contracts as-is by the use of the AppLayer EVM, which is compatible with bytecode deployment. This means any language that compiles to EVM bytecode (e.g. [Solidity](https://soliditylang.org/), [Vyper](https://docs.vyperlang.org/en/stable/), etc.) can be used to deploy contracts in the AppLayer EVM in a seamless, straight-forward way. - -This kind of compatibility is possible thanks to the integration of the [EVMOne](https://github.com/ethereum/evmone) virtual machine (originally made by the Ethereum devs) and [EVMC](https://github.com/ethereum/evmc) libraries. diff --git a/evm-contracts/calling-cpp-contracts-from-evm.md b/evm-contracts/calling-cpp-contracts-from-evm.md deleted file mode 100644 index 5bab2b0..0000000 --- a/evm-contracts/calling-cpp-contracts-from-evm.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -description: How a C++ contract calls an EVM contract in AppLayer. ---- - -## Calling C++ contracts from EVM - -Calling a C++ contract from an EVM contract uses a standard Solidity interface to abstract the C++ implementation. This approach ensures that calls from EVM to C++ are as straightforward as EVM-to-EVM calls. - -First, we define a Solidity interface that matches the signature of the C++ functions you wish to call. This interface acts as a facade, providing a Solidity view of the C++ contract functionalities: - -```solidity -interface MyContract { - function myFunction(uint256 arg1, uint256 arg2) external view returns (uint256); -} -``` - -Then, we use the defined interface to make calls to the C++ contract. This is handled similarly to any inter-contract communication in Solidity, ensuring a seamless integration layer: - -```solidity -contract AnotherContract { - function callMyFunction(address cppAddr, uint256 arg1, uint256 arg2) public view returns (uint256) { - return MyContract(cppAddr).myFunction(arg1, arg2); - } -} -``` diff --git a/evm-contracts/calling-evm-contracts-from-cpp.md b/evm-contracts/calling-evm-contracts-from-cpp.md deleted file mode 100644 index 43f1322..0000000 --- a/evm-contracts/calling-evm-contracts-from-cpp.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: How an EVM contract calls a C++ contract in AppLayer. ---- - -# Calling EVM contracts from C++ - -To invoke EVM contract functions from C++, we leverage a templated approach that mimics the contract's functions in a C++ class. This method provides a type-safe way to interact with contracts written in Solidity or other EVM-compatible languages. - -First, we define a proxy C++ class that represents the EVM contract. This class will include stubs of the contract's functions, which do not contain actual logic but serve to match the contract's interface in the blockchain: - -```cpp -class SolMyContract { - public: - uint256_t myFunction(const uint256_t& arg1, const uint256_t& arg2) const {}; - static void registerContract() { - ContractReflectionInterface::registerContractMethods( - std::vector{}, // List of dependencies or related artifacts if any - std::make_tuple("myFunction", &SolMyContract::myFunction, FunctionTypes::View, std::vector{"arg1", "arg2"}) - ); - } -}; -``` - -Then, we ensure that the proxy class is registered within the blockchain before any calls are made. Typically, this registration is done once, often in the constructor of the calling C++ contract, to set up the reflection system used for method invocation: - -```cpp -uint256_t AnotherContract::callMyFunction(const Address& targetAddr, const uint256_t& arg1, const uint256_t& arg2) const { - SolMyContract::registerContract(); // Ensure the EVM contract's methods are registered (Can be done only a single time in the constructor) - return this->callContractViewFunction(this, targetAddr, &SolMyContract::myFunction, arg1, arg2); -} -``` diff --git a/evm-contracts/cpp-to-other-contract-calls.md b/evm-contracts/cpp-to-other-contract-calls.md deleted file mode 100644 index 3fd9dfd..0000000 --- a/evm-contracts/cpp-to-other-contract-calls.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -description: How contract calls happen from the C++ side in AppLayer. ---- - -# C++ to other contract calls - -The `ContractHost` class employs templated functions to support flexible and efficient interaction with contracts. These templates enable passing any combination of arguments and return types (including `void`) to and from other types of contracts. This use of templates helps to leverage the fast ABI encoding/decoding processes, ensuring optimal performance and flexibility during contract execution: - -```cpp -template -R callContractViewFunction( - const BaseContract* caller, const Address& targetAddr, - R(C::*func)(const Args&...) const, const Args&... args -) const; - -template -R callContractFunction( - BaseContract* caller, const Address& targetAddr, - const uint256_t& value, - R(C::*func)(const Args&...), const Args&... args -); -``` - -This approach allows for dynamic interaction with contracts without pre-defining all possible function signatures, accommodating various contract behaviors and states dynamically. diff --git a/evm-contracts/evm-to-other-contract-calls.md b/evm-contracts/evm-to-other-contract-calls.md deleted file mode 100644 index a5d0253..0000000 --- a/evm-contracts/evm-to-other-contract-calls.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -description: How contract calls happen from the EVM side in AppLayer. ---- - -# EVM to other contract calls - -For calls from the EVM to another contract, the `ContractHost::call()` function plays a crucial role. It is tasked with creating and handling calls to other contracts, encapsulating the complexity of contract interaction within a simple interface: - -```cpp -evmc::Result call(const evmc_message& msg) noexcept final; -``` - -This function is designed to handle both C++ and EVM contract calls, as shown below: - -```cpp -evmc::Result ContractHost::call(const evmc_message& msg) noexcept { - evmc::Result result; - const bool isContractCall = isCall(msg); - - if (isContractCall) { - this->traceCallStarted(msg); - } - - switch (this->decodeContractCallType(msg)) - { - case ContractType::CREATE: { - result = this->callEVMCreate(msg); - break; - } - case ContractType::CREATE2: { - result = this->callEVMCreate2(msg); - break; - } - case ContractType::PRECOMPILED: { - result = this->processBDKPrecompile(msg); - break; - } - case ContractType::CPP: { - result = this->callCPPContract(msg); - break; - } - default: - result = this->callEVMContract(msg); - break; - } - - if (isContractCall) { - this->traceCallFinished(result.raw()); - } - - return result; -} -``` diff --git a/evm-contracts/seamless-cpp-evm-integration.md b/evm-contracts/seamless-cpp-evm-integration.md deleted file mode 100644 index 1fa6b9d..0000000 --- a/evm-contracts/seamless-cpp-evm-integration.md +++ /dev/null @@ -1,108 +0,0 @@ ---- -description: How AppLayer achieves a seamless integration between C++ and EVM contracts. ---- - -# Seamless C++/EVM integration - -Achieving seamless integration between C++ and EVM contracts revolves around the uniformity in the encoding and decoding of arguments. By standardizing these processes, we ensure that calls between different contract types are handled efficiently without the need for separate mechanisms for each. - -## The evmc\_message struct - -We do this by using the `evmc_message` struct, aligning the call structures between C++ and EVM environments. This uniformity simplifies the interaction framework and reduces the potential for errors and data mismanagement: - -```cpp -struct evmc_message { - enum evmc_call_kind kind; // The kind of the call. - uint32_t flags; - int32_t depth; - int64_t gas; - evmc_address recipient; - evmc_address sender; - const uint8_t* input_data; - size_t input_size; - evmc_uint256be value; - evmc_bytes32 create2_salt; - evmc_address code_address; -}; -``` - -## Determining contract types and executing calls - -`ContractHost` plays a critical role in distinguishing whether a contract is implemented in C++ or EVM and executing it accordingly. Below is an example illustrating how C++ contracts can invoke functions in other contracts, whether they are C++ or EVM: - -```c++ -template -R callContractFunctionImpl( - BaseContract* caller, const Address& targetAddr, - const uint256_t& value, - R(C::*func)(const Args&...), const Args&... args -) { - // 1000 Gas Limit for every C++ contract call! - auto& recipientAcc = *this->accounts_[targetAddr]; - if (!recipientAcc.isContract()) { - throw DynamicException(std::string(__func__) + ": Contract does not exist - Type: " - + Utils::getRealTypeName() + " at address: " + targetAddr.hex().get() - ); - } - if (value) { - this->sendTokens(caller, targetAddr, value); - } - NestedCallSafeGuard guard(caller, caller->caller_, caller->value_); - switch (recipientAcc.contractType) { - case ContractType::EVM: { - this->deduceGas(10000); - evmc_message msg; - msg.kind = EVMC_CALL; - msg.flags = 0; - msg.depth = 1; - msg.gas = this->leftoverGas_; - msg.recipient = targetAddr.toEvmcAddress(); - msg.sender = caller->getContractAddress().toEvmcAddress(); - auto functionName = ContractReflectionInterface::getFunctionName(func); - if (functionName.empty()) { - throw DynamicException("ContractHost::callContractFunction: EVM contract function name is empty (contract not registered?)"); - } - auto functor = ABI::FunctorEncoder::encode(functionName); - Bytes fullData; - Utils::appendBytes(fullData, UintConv::uint32ToBytes(functor.value)); - if constexpr (sizeof...(Args) > 0) { - Utils::appendBytes(fullData, ABI::Encoder::encodeData(args...)); - } - msg.input_data = fullData.data(); - msg.input_size = fullData.size(); - msg.value = EVMCConv::uint256ToEvmcUint256(value); - msg.create2_salt = {}; - msg.code_address = targetAddr.toEvmcAddress(); - evmc::Result result (evmc_execute(this->vm_, &this->get_interface(), this->to_context(), - evmc_revision::EVMC_LATEST_STABLE_REVISION, &msg, recipientAcc.code.data(), recipientAcc.code.size())); - this->leftoverGas_ = result.gas_left; - if (result.status_code) { - auto hexResult = Hex::fromBytes(bytes::View(result.output_data, result.output_data + result.output_size)); - throw DynamicException("ContractHost::callContractFunction: EVMC call failed - Type: " - + Utils::getRealTypeName() + " at address: " + targetAddr.hex().get() + " - Result: " + hexResult.get() - ); - } - if constexpr (std::same_as) { - return; - } else { - return std::get<0>(ABI::Decoder::decodeData(bytes::View(result.output_data, result.output_data + result.output_size))); - } - } break; - case ContractType::CPP: { - this->deduceGas(1000); - C* contract = this->getContract(targetAddr); - this->setContractVars(contract, caller->getContractAddress(), value); - try { - return contract->callContractFunction(this, func, args...); - } catch (const std::exception& e) { - throw DynamicException(e.what() + std::string(" - Type: ") - + Utils::getRealTypeName() + " at address: " + targetAddr.hex().get() - ); - } - } - default : { - throw DynamicException("PANIC! ContractHost::callContractFunction: Unknown contract type"); - } - } -} -``` diff --git a/evm-contracts/state-management-and-vm-instance-creation.md b/evm-contracts/state-management-and-vm-instance-creation.md deleted file mode 100644 index da89022..0000000 --- a/evm-contracts/state-management-and-vm-instance-creation.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -description: How the AppLayer ecosystem manages the blockchain state alongside an EVM. ---- - -# State management and VM instance creation - -The VM itself is owned and instantiated by the `State` class, which reflects a crucial design decision: centralizing virtual machine resource management like this ensures that each contract execution context is cleanly managed and isolated. Whenever a new transaction or contract call needs to be executed, regardless of its nature (be it an EVM/C++ contract execution or a simple native transfer), the `State` class is responsible for instantiating a new `ContractHost` object with the relevant parameters required for execution: - -```cpp -ContractHost( - evmc_vm* vm, - DumpManager& manager, - const Storage& storage, - const Hash& randomnessSeed, - const evmc_tx_context& currentTxContext, - boost::unordered_flat_map, SafeHash>& contracts, - boost::unordered_flat_map, SafeHash>& accounts, - boost::unordered_flat_map& vmStorage, - const Hash& txHash, - const uint64_t txIndex, - const Hash& blockHash, - int64_t& txGasLimit -); -``` - -Once an instance of `ContractHost` is created, it offers methods like `execute()` to run the contract, `simulate()` for simulating the transaction (useful for gas estimation), and `ethCallView()` for making calls to other contracts within a non-state-changing context. - -`ContractHost` also extends the functionalities of `evmc::Host` by overriding several key functions that interface directly with the Ethereum Virtual Machine, which are obligatory for the VM to interact with the State: - -```cpp -bool account_exists(const evmc::address& addr) const noexcept final; -evmc::bytes32 get_storage(const evmc::address& addr, const evmc::bytes32& key) const noexcept final; -evmc_storage_status set_storage(const evmc::address& addr, const evmc::bytes32& key, const evmc::bytes32& value) noexcept final; -evmc::uint256be get_balance(const evmc::address& addr) const noexcept final; -size_t get_code_size(const evmc::address& addr) const noexcept final; -evmc::bytes32 get_code_hash(const evmc::address& addr) const noexcept final; -size_t copy_code(const evmc::address& addr, size_t code_offset, uint8_t* buffer_data, size_t buffer_size) const noexcept final; -bool selfdestruct(const evmc::address& addr, const evmc::address& beneficiary) noexcept final; -evmc::Result call(const evmc_message& msg) noexcept final; -evmc_tx_context get_tx_context() const noexcept final; -evmc::bytes32 get_block_hash(int64_t number) const noexcept final; -void emit_log(const evmc::address& addr, const uint8_t* data, size_t data_size, const evmc::bytes32 topics[], size_t topics_count) noexcept final; -evmc_access_status access_account(const evmc::address& addr) noexcept final; -evmc_access_status access_storage(const evmc::address& addr, const evmc::bytes32& key) noexcept final; -evmc::bytes32 get_transient_storage(const evmc::address &addr, const evmc::bytes32 &key) const noexcept final; -void set_transient_storage(const evmc::address &addr, const evmc::bytes32 &key, const evmc::bytes32 &value) noexcept final; -``` - -These methods manage everything from account validation to logging, providing access to the state and storage, and handling calls between contracts. The `ContractHost` class encapsulates these functions, ensuring that each contract execution is isolated and secure. diff --git a/future-plans/README.md b/future-plans/README.md new file mode 100644 index 0000000..d5ac207 --- /dev/null +++ b/future-plans/README.md @@ -0,0 +1,9 @@ +--- +description: Our potential works for the future +--- + +# Future Plans + +As it is with all kinds of software development, there's always room for improvement. This chapter lays out a few ideas and reasonings that could be implemented in AppLayer in the future. + +**Please note** that this is not a roadmap! The sections written here were brought over from older versions of the documentation (pre-mainnet release), they may change or be dropped entirely depending on the case. They're here mostly for reference purposes. \ No newline at end of file diff --git a/future-plans/appchains.md b/future-plans/appchains.md new file mode 100644 index 0000000..c1fe577 --- /dev/null +++ b/future-plans/appchains.md @@ -0,0 +1,19 @@ +--- +description: How Application Chains would work on AppLayer +--- + +# Application Chains + +An **application chain**, also known as an AppLayer™, is a blockchain built using the BDK and deployed on the Chain Abstraction Network. An AppLayer™ would primarily enable developers to create a chain dedicated to a particular application, with the chain's rules specifically tailored to the security, performance, speed, decentralization and other service delivery needs of the application itself. These AppLayers would be responsible for securing their own ledger of accounts and balances along with their own execution logic. + +AppLayer's BDK currently supports C++ and Solidity for blockchain development, with future plans for other languages such as Rust, C#, Golang, and more. These application chains would ideally compile to a binary to enable efficient execution in parallel to Solidity bytecode. Whoever sets up an application chain would be in charge of validating its own transactions and attracting others to run its Validators. + +## Types of precompiled contracts + +With this idea implemented, AppLayer would conceptually have three different types of contracts: *1st-party* , *3rd-party*, and *AppChains/AppLayers*. + +**1st-party** contracts are contracts provided by AppLayer itself as ready-to-use templates in the `src/contract/templates` subfolder (e.g. ERC20, ERC721, etc. - what we have right now), officially supported and leveraging all the features of the blockchain. Those contracts are also available in the AppLayer EVM chain. + +**3rd-party** contracts would act like a "slim Layer 2", in the sense that they would be only processed, but have no concepts of consensus, blocks, transactions, etc. unlike a normal contract. They would essentially act like a "daemon" of sorts, running atop the main chain (just like a Layer 2), reading transactions made on it and reacting accordingly if said transaction happens to call it. The transaction would be executed and the contract would publish the results back on the main chain (e.g. a user sent tokens to a 3rd-party exchange contract, once the transaction is confirmed on the main chain the contract reads the data field and executes its own logic, then sends the exchanged tokens back on the main chain as its own transaction). + +**AppChains/AppLayers** themselves would act as a "full Layer 2" instead, having their own blocks, transactions, consensus, etc., but still depending on the main chain to execute their separate logic. \ No newline at end of file diff --git a/future-plans/bridging/README.md b/future-plans/bridging/README.md new file mode 100644 index 0000000..c76afc9 --- /dev/null +++ b/future-plans/bridging/README.md @@ -0,0 +1,19 @@ +--- +description: How bridging would work on AppLayer +--- + +# Bridging + +There are inherent flexibility issues with **native** and **application-specific** chains when compared to traditional EVM chains. An application-specific chain is inherently limited as it can not support a complete service that involves interaction between more than one application. Those problems could heavily damage the reputation of a project built on them. + +Our conceptual solution to this issue is to allow AppLayer-enabled blockchains to natively communicate with each other by using the Chain Abstraction Network (**CAN** for short) as a middleman, where AppLayer serves as an intermediary between two dApp chains trying to communicate with each other. We call that **bridging**. + +At the moment, bridging in AppLayer is implemented in a centralized manner, much like other conventional networks. Eventually it would evolve to a fully decentralized solution, which would allow bridging for *arbitrary data* and *tokens*, both *between AppLayer nodes* and *between AppLayer and external networks*. + +## How would safety be ensured? + +The Validator (and eventual Sentinel) nodes that read from a given chain are determined using *RandomGen* - a trustless, decentralized randomness generator developed by AppLayer Labs. We would ensure and keep a fair selection of nodes, however, there could be a possibility of a 51% attack. + +For example, in a network with 100 nodes, if a malicious user controls 51 of them, and all of them get selected for driving a cross-chain request and a block, they could collude and forward any message they want. We would avoid this by introducing Sentinels to the network, which would ensure this collusion does not happen by working together with Validators to fortify the network’s security. + + diff --git a/how-applayer-works/bridging/applayer-to-applayer-data-bridging.md b/future-plans/bridging/applayer-to-applayer-data-bridging.md similarity index 61% rename from how-applayer-works/bridging/applayer-to-applayer-data-bridging.md rename to future-plans/bridging/applayer-to-applayer-data-bridging.md index 25f3216..e7bf804 100644 --- a/how-applayer-works/bridging/applayer-to-applayer-data-bridging.md +++ b/future-plans/bridging/applayer-to-applayer-data-bridging.md @@ -1,10 +1,10 @@ --- -description: How native chains exchange data on AppLayer. +description: How native chains would exchange data on AppLayer --- # AppLayer-to-AppLayer Data Bridging -Bridging arbitrary data from an AppLayer chain (A) to another (B) is simple: +Here's how bridging arbitrary data from an AppLayer chain (A) to another (B) would work: * Chain A sends a request in the upcoming block and communicates this, along with the block reference, to the Chain Abstraction Network. This request is written to the next block and relayed to AppLayer's Chain Abstraction Network. @@ -18,15 +18,15 @@ Bridging arbitrary data from an AppLayer chain (A) to another (B) is simple:

B gathers data from itself and sends it back to the Chain Abstraction Network

-* Validators and Sentinels then check and corroborate the data submitted by Chain B with other nodes from Chain B to validate its presence in the designated block. +* Validators and Sentinels check and corroborate the data submitted by Chain B with other nodes from Chain B to validate its presence in the designated block.

Chain Abstraction Network checks other B nodes to verify data

-* Once data is verified, Validators and Sentinels sign the data and publish it inside the Chain Abstraction Network, while also relaying it back to A along with their signatures. +* Once the data is verified, Validators and Sentinels sign the data and publish it inside the Chain Abstraction Network, while also relaying it back to Chain A along with their signatures.

Chain Abstraction Network relays signed data back to A

-* A verifies the signatures and checks if the randomly selected nodes were using the network's RandomGen seed. If everything matches, the exchange is complete, but if not, there's a malicious node in the network which should be reported. +* Chain A verifies the signatures and checks if the randomly selected nodes were using the network's RandomGen seed. If everything matches, the exchange is complete, but if not, there's a malicious node in the network which should be reported. -It is possible for A to only send information to the target chain without having to wait for an answer. This call can possibly trigger logic within the target chain, depending on how their developers decided to handle your message. +It is possible for Chain A to only send information to the target chain without having to wait for an answer. This call can possibly trigger logic within the target chain, depending on how its developers decided to handle the message. diff --git a/future-plans/bridging/applayer-to-applayer-token-bridging.md b/future-plans/bridging/applayer-to-applayer-token-bridging.md new file mode 100644 index 0000000..623534e --- /dev/null +++ b/future-plans/bridging/applayer-to-applayer-token-bridging.md @@ -0,0 +1,22 @@ +--- +description: How native chains would exchange tokens on AppLayer +--- + +# AppLayer-to-AppLayer Token Bridging + +Token bridging between two AppLayer nodes would work the same way as arbitrary data bridging, but with extra checkups to make sure that a given chain could not mint another chain's tokens. + +Due to how the network was designed, when doing a cross-chain transaction we can only ensure that the data *exists*, not that it is *valid in context*. That breach allows a given chain to mint the native token of another chain, because the Chain Abstraction Network does not verify if the token is valid inside that network. + +We could avoid this problem by keeping a "token table", which is just a "spreadsheet" of chains and their external token balances. This doesn't include the given chain's own native token, since it can freely mint its own token itself and does its own internal validations to avoid invalid minting conditions. + +For example, we have chains A, B, and C, each one with tokens of each other. The Chain Abstraction Network would keep track of: + +* How many B and C tokens exist on A +* How many A and C tokens exist on B +* How many A and B tokens exist on C + +When bridging another chain's tokens, the Chain Abstraction Network would check if that chain has enough balance to do so. When bridging your own tokens, the Chain Abstraction Network would only have to increase the balance at the target chain, since the `exit` transaction from your chain has to be included in one of your blocks, which means it has been verified and validated inside your own network, so there's no need to do it again from the outside. + +This method of bridging would follow mint/burn mechanisms. + diff --git a/future-plans/bridging/applayer-to-external-bridging.md b/future-plans/bridging/applayer-to-external-bridging.md new file mode 100644 index 0000000..ae090ea --- /dev/null +++ b/future-plans/bridging/applayer-to-external-bridging.md @@ -0,0 +1,12 @@ +--- +description: How AppLayer would bridge with other blockchains +--- + +# AppLayer-to-External Bridging (Ethereum, Solana, etc.) + +Bridging between AppLayer and other conventional blockchains brings multiple edge cases. For example, it's not possible to natively push data into these chains without paying transaction fees, and those external networks are limited on both processing power and how much signature verification can be done. + +Knowing this, at least for now, the bridging implementation for them would be handled by Sentinels and owned by AppLayer Labs and its most trusted partners to ensure operational safety. The contract would check signatures of Validators and Sentinels but only the Sentinels could write into the contracts on these chains. + +This method of bridging would follow lock/release mechanisms, unless a token was fully integrated with mint/burn bridging. + diff --git a/future-plans/rdpos.md b/future-plans/rdpos.md new file mode 100644 index 0000000..25d4172 --- /dev/null +++ b/future-plans/rdpos.md @@ -0,0 +1,91 @@ +--- +description: The solution to consensus on performance-driven blockchains +--- + +# rdPoS + +To keep consensus on such a blazing fast network without tripping up and/or having to deal with rollbacks, it would be necessary to use a *random deterministic block creation* that allows only one given node to create a block for a given time, eliminating the risk of a block race condition in the network. + +Going beyond the current consensus engine (CometBFT), AppLayer would ideally implement its own consensus algorithm, hereby denominated **rdPoS** (*Random Deterministic Proof of Stake*). It would empower Validators and Sentinels to deal with block congestion and random number generation. This section aims to explain in-depth how such algorithm would work and be used by the AppLayer protocol. + +## A primer on blockchain rollbacks + +One of the biggest problems of blockchain development is handling block rollbacks. For example, on the Bitcoin chain, assuming there is a latest block that has another block after it. If a node receives a block that replaces the latest block, the next block and all the transactions in it are replaced too, which results in a rollback of the blockchain’s state by one block. + +The Bitcoin blockchain and other derivatives follow the "longest lived chain" rule (the chain with the most accumulated proof of work is the main chain). However, rollbacks unearth problems in that rule. For instance, when a developer is building dApps where they have to deal with such special conditions, it could take a greater effort depending on the size and/or complexity of the application. + +

In this example, block C is replaced by block D followed by block E, rolling back the transactions made in block C

+ +The ideal solution is to avoid the rollback condition altogether. This can be done by deterministically defining which network node can create a block, thereby eliminating the block race condition and keeping everyone in the network synchronized to the same latest block. This is where rdPoS comes in. It would pair a block congestion system and a random number generator system, allowing only one Validator to create a block at any given time, thus avoiding rollbacks and achieving consensus on ultra-fast networks. + +## How rdPoS would work + +The heart of rdPoS is *RandomGen*, a deterministic uint256\_t generator used for almost everything related to consensus. This deterministic randomness ensures that every node has a chance to respond to a given request (block, randomness, bridging, etc.), while making sure that the nodes selected from the network are truly random and not problematic nodes operated by a malicious actor. + +For RandomGen to be viable, it needs to be seeded with a truly random number. This is how it works: + +* Every time a new block is about to be created, 16 random nodes are selected using RandomGen with the previous block’s randomness seed +* These nodes make a 32-byte random string (`RandomnessSeed`) and hash it (`RandomnessHash`), then sign the hash and publish it to the network +* After all the nodes have signed and published their hashes to the network, they can publish the real data, verifying that no one is trying to manipulate the end result +* After the data is published and included in the block, the randomness seeds are concatenated and hashed, and the resulting hash is used to seed the next block creation + +We must pay attention to the current state of RandomGen to ensure all nodes are always in the same internal state, so they can properly synchronize with each other. + +## Creating a block under rdPoS + +A block in an rdPoS network would be created by the following rules: + +* A list of network Validators is randomly generated and sorted using the "randomness" seed from the previous block + +

New random Validator list being created

+ +* The first Validator from the list will be the block creator, while at least 4 others will create a random 32-byte string and make two transactions with it: one containing the hash of said string, and another containing the string itself, both signed + +

Validators performing a hash transaction broadcast

+ +

Validators performing a random transaction broadcast

+ +* The hashes are verified to make sure they match their respective random strings + +

Transactions are checked against each other

+ +* A new block is created by the first Validator, concatenating and hashing the other Validators' random strings to create a new "randomness" seed that will be used at the next block + +

New randomness seed is generated

+ +

New block is created with randomness seed, Validator signature and the transactions

+ +* The block is signed and published to the network by the first Validator, while the other Validators verify that all transaction signatures (random and hashed) correspond with the list created at the start +* The genesis block (the very first block in the chain) enforces a given fixed randomness to be valid, since there is no previous block before it to derive the randomness from. At least five hardcoded Validators are needed to bootstrap the network, as each block requires at least four Validators to confirm the string and hash transaction signatures, and one for signing the block itself + +As quoted by [Supra](https://github.com/Jean-Lessa): *"It's like playing poker but everyone hashes their hands first before showing the real cards"*. + +## Validator implementations + +Under rdPoS, developers would get to choose how Validators are added to the network, based on three pre-established implementation options: *permissionless*, *permissioned*, and *semi-permissioned*. + +### Permissionless + +In a permissionless implementation, all Validators would have to participate in block creation to ensure that there’s no collusion. A totally permissionless network using rdPoS could face problems when the number of Validators in the network grows massively (e.g. around 10,000), because the latency between those nodes could increase significantly. + +To solve this problem, the block time in a permissionless network should be bigger (e.g. 15-30 seconds) so all the nodes have enough time to respond. Validators could be added to a permissionless network by locking a certain amount of tokens in the contract that contains the rdPoS logic. + +### Permissioned + +In a permissioned implementation, every network would have a "master address" that could add as many Validators as desired. Whoever sets up this implementation would be responsible for keeping the chain up and running. The recommended number of nodes for this implementation would be at least 32, but more or less nodes could be used depending on the application’s needs. + +### Semi-permissioned + +In a semi-permissioned implementation, both Validators and Sentinels would be used in tandem. Validators would mirror the "permissionless" side (being added to the network by locking tokens), while Sentinels would mirror the "permissioned" side (being added to the network with a master address). + +This implementation is different in the sense that neither Validators nor Sentinels can create a block on their own. The deterministic randomness would require at least one of the transactions from a Sentinel and whoever publishes the block would have to follow the Validator list order set during rdPoS processing. + +This means the network could have a smaller number of Validators (e.g. 16 instead of 32), requiring less computing power for verification but remaining highly secure. As Sentinels take part in the process, every extra byte in the concatenated randomness seed will change the resulting hash, thus ensuring that any attempt of tampering is easily detected and dealt with. + +## Slashing + +What happens when a Validator answers with a "randomness" hash that does not match its own hash? Or when a Validator creates an invalid block with invalid transactions? Or when a Validator can't create a block before reaching the network's time limit? + +Misbehaving nodes must suffer dire consequences. Since Validator signatures are required at protocol level, if a Validator tries to break the rules, it's possible to know who it is thanks to the signature, and "slash" it from the network. + +The biggest problem with this is a group of Validators being "slashed" and halting network activity. This could be solved by adding extra conditions to the network - for example, if the network wants to change the current block creator (in case it's been "slashed"), at least 90% of the Validators in the network have to sign a transaction consenting with the change, always maintaining the majority's consensus. diff --git a/future-plans/sentinels.md b/future-plans/sentinels.md new file mode 100644 index 0000000..4e39a7c --- /dev/null +++ b/future-plans/sentinels.md @@ -0,0 +1,9 @@ +--- +description: How Sentinels would work on AppLayer +--- + +# Sentinels + +**Sentinels** in concept are similar to Validators, except they cannot create blocks nor act on their own. Both randomly selected Validators and Sentinels should send the same data to the requester, otherwise they would be reported to the network as a malicious node. + +Those Sentinels would be hosted by AppLayer Labs and its partners to ensure that such kind of malicious activity could not happen, but could also be open for trusted KYC'd 3rd parties or entities to host with a high level of scrutiny. Eventually as the network and underlying technology matures, Sentinels would be phased out in favor of a fully permissionless system. diff --git a/get-in-touch.md b/get-in-touch.md index b25e0e7..072f0e0 100644 --- a/get-in-touch.md +++ b/get-in-touch.md @@ -1,5 +1,5 @@ --- -description: Start building on AppLayer today +description: Start building on AppLayer today! --- # Get in touch diff --git a/getting-started/README.md b/getting-started/README.md new file mode 100644 index 0000000..e2bbc0d --- /dev/null +++ b/getting-started/README.md @@ -0,0 +1,9 @@ +--- +description: A step-by-step guide on how to onboard AppLayer +--- + +# Getting started + +This chapter aims to guide new users through interacting with the AppLayer network, with step-by-step instructions for engaging with our mainnet and testnet respectively, and for developers to set up their own environment for BDK so they can build their own AppLayer-powered blockchains. + +The instructions laid out here are simple and to the point, as they're meant to be an entrypoint for the general AppLayer ecosystem. Developers who want a deeper look on how BDK works should read the "BDK implementation" section for a more thorough tour on the specifics of BDK's internal workings. diff --git a/getting-started/applayer-testnet/README.md b/getting-started/applayer-testnet/README.md new file mode 100644 index 0000000..98ffd71 --- /dev/null +++ b/getting-started/applayer-testnet/README.md @@ -0,0 +1,37 @@ +--- +description: The playground for AppLayer's future endeavours +--- + +# AppLayer Testnet + +We have a testnet for developers and users to, well, test the network! It is highly recommended to start here before going to the mainnet, as is the case with any other blockchain. + +We'll be using the [Remix IDE](https://remix.ethereum.org) as a tool to aid us in deploying contracts in the network. It is recommended to have some basic knowledge on how to operate it. + +For compatibility purposes, **set the EVM in Remix to "Shanghai"** before using it, as that is the version our network currently supports. + +## Switching to the testnet + +First, make sure you have set your preferred Web3 frontend (e.g. MetaMask) to connect to the AppLayer Testnet. Configuration is as follows: + +* **Network Name**: AppLayer Testnet +* **RPC URL**: https://testnet-api.applayer.com/ +* **Chain ID**: 75338 +* **Currency Symbol**: APPL +* **Block Explorer URL**: https://testnet-explorer.applayer.com/ + +Here's a video showing the configuration on MetaMask as an example: + +{% embed url="https://drive.google.com/file/d/1j1i2Poox0JdjeXKOX1c6SB5_ggNWVd4W/view?usp=drive_link" %} + +## Claiming your tokens + +Tokens on a testnet are usually distributed by *faucets* - apps or sites that distribute small amounts of said token to users for free (since tokens on a testnet have no inherent value, contrary to tokens on the mainnet which represent real money). + +To claim your APPL tokens on the testnet, open our [testnet faucet website](https://testnet-faucet.applayer.com/), enter your testnet address there and click on the "Request Tokens" button. + +**Please note that MetaMask might take up to 5 minutes to update your balance.** If the faucet displays "Tokens requested and transferred successfully", the operation has been successful, so please be patient. + +Check this example video showing how to claim APPL tokens using a MetaMask address: + +{% embed url="https://drive.google.com/file/d/1_uqgwqgiQHAbyU7qKwoM6KOcSO93abSJ/view?usp=drive_link" %} \ No newline at end of file diff --git a/getting-started-with-applayer-testnet.md b/getting-started/applayer-testnet/deploying-contracts.md similarity index 62% rename from getting-started-with-applayer-testnet.md rename to getting-started/applayer-testnet/deploying-contracts.md index a88696d..e355080 100644 --- a/getting-started-with-applayer-testnet.md +++ b/getting-started/applayer-testnet/deploying-contracts.md @@ -1,36 +1,12 @@ --- -description: A hands-on guide for interacting with the AppLayer Testnet +description: A hands-on guide for interacting with contracts in the AppLayer Testnet --- -# Getting started with AppLayer Testnet +# Deploying contracts (Testnet) -Here's a simple guide on how to start developing with AppLayer and the Blockchain Development Kit (BDK). We'll be using the [Remix IDE](https://remix.ethereum.org) as a tool to aid us in deploying contracts in the network. +Here's a simple guide on how to deploy both precompiled (C++) and EVM (Solidity) contracts on the AppLayer Testnet. Check the Contracts section for deeper details on how it all works. -### Step 1 - Switching to the AppLayer Testnet - -First, make sure you have set your preferred Web3 frontend (e.g. MetaMask) to connect to the AppLayer Testnet. Configuration is as follows: - -* **Network Name**: AppLayer Testnet -* **RPC URL**: https://testnet-api.applayer.com/ -* **Chain ID**: 75338 -* **Currency Symbol**: APPL -* **Block Explorer URL**: https://testnet-explorer.applayer.com/ - -Here's a video showing the configuration on MetaMask as an example: - -{% file src=".gitbook/assets/applayer_step1.mp4" %} - -### Step 2 - Claiming your AppLayer Tokens - -To claim your APPL tokens on the testnet, open our [testnet faucet website](https://testnet-faucet.applayer.com/), enter your testnet address there and click on the "Request Tokens" button. - -**Please note that MetaMask might take up to 5 minutes to update your balance.** If the faucet displays "Tokens requested and transferred successfully", the operation has been successful, so please be patient. - -Check this example video showing how to claim APPL tokens using a MetaMask address: - -{% file src=".gitbook/assets/applayer_step2.mp4" %} - -### Step 3 - Deploying C++ contracts +## Deploying C++ contracts To deploy a C++ contract on the testnet, open Remix IDE and then compile this Solidity interface in it: @@ -55,17 +31,17 @@ interface ContractManager { } ``` -This will allow you to call the `ContractManager` precompiled contract and use it to deploy any of the available precompiles on the blockchain. Precompile deploys cost 100,000 gas each. +This will allow you to call the `ContractManager` precompiled contract and use it to deploy any of the available precompiled contracts on the blockchain. Precompiled contract deploys cost 100,000 gas each. To find out the address of your deployed contract, call the `getDeployedContractsForCreator()` function, passing the `ContractManager` address itself as the argument. The address for ContractManager is hardcoded to `0x0001cb47ea6d8b55fe44fdd6b1bdb579efb43e61`. See the example video that deploys an ERC20 contract: -{% file src=".gitbook/assets/applayer_step3.mp4" %} +{% embed url="https://drive.google.com/file/d/1zYKqBOCqS_CoL2HM-jV1BzVWuirP2NkO/view?usp=drive_link" %} -### Step 4 - Deploying EVM contracts +## Deploying EVM contracts -Deploying a Solidity/EVM contract on the testnet is done just like with Ethereum. The AppLayer EVM is set to "Shanghai", so be sure to set Remix to the same version before compiling. Check the video below for an example on the following contract: +Deploying a Solidity/EVM contract on the testnet is done just like Ethereum. **The AppLayer EVM is set to "Shanghai"**, so be sure to set Remix to the same version before compiling. Check the video below for an example on the following contract: ```solidity // contracts/GLDToken.sol @@ -81,9 +57,9 @@ contract AnotherTestToken is ERC20 { } ``` -{% file src=".gitbook/assets/applayer_step4.mp4" %} +{% embed url="https://drive.google.com/file/d/1Q1yV8J37fhn8CTfBlgYYCOXMz3FhHO3o/view?usp=drive_link" %} -### Step 5 (Optional) - Using randomness in EVM contracts +## (Optional) Using randomness in EVM contracts You can also use one of our on-chain precompiles called `BDKPrecompile` to fetch random numbers generated on the fly. It is only accessed by the EVM through the following Solidity interface: @@ -126,10 +102,10 @@ contract RandomnessTest { } ``` -Compile this code in Remix IDE (with the EVM set to "Shanghai", like done in the previous step) and deploy the `RandomnessTest` contract. Once it is deployed call the `setRandom` function to initialize it, and then call the `getRandom` function to get a random number. +Compile this code in Remix IDE (**remember to set the EVM to "Shanghai"**, like in the previous step) and deploy the `RandomnessTest` contract. Once it is deployed call the `setRandom` function to initialize it, and then call the `getRandom` function to get a random number. **If you are using the interface in a view function, be aware that two different executions will always result in a different value (if called by RPC).** This is done on purpose, as the random value is only decided when the transaction is included in a block and it is generated in a cryptographically secure manner, making it impossible to predict the value. For the `RandomnessTest` contract specifically, due to how it is coded, if you want a new random number you must call `setRandom` again before calling `getRandom` the next time. See the following video as an example: -{% file src=".gitbook/assets/applayer_step5.mp4" %} +{% embed url="https://drive.google.com/file/d/1m1d7P2ibQTogSnq_72JTZjDvHat4Tnpd/view?usp=drive_link" %} diff --git a/getting-started/developing-with-bdk/README.md b/getting-started/developing-with-bdk/README.md new file mode 100644 index 0000000..8684c09 --- /dev/null +++ b/getting-started/developing-with-bdk/README.md @@ -0,0 +1,15 @@ +--- +description: A hands-on guide for compiling AppLayer's BDK and running your own blockchain +--- + +# Developing with BDK + +This subchapter explains how to set up AppLayer's BDK, our open-core blockchain SDK project, to start creating and deploying your own blockchain network and contracts in it. You'll be able to tweak almost everything related to the BDK. We offer pre-existing solutions for all of those, but you are free to hack into them as you wish: + +* Consensus +* Block processing +* Transaction processing +* Contract processing +* Communication between nodes, etc. + +This is an overview/"more approachable" version of the BDK's own README.md file - be sure to read it as well for a summed up version. Current instructions are specific to the only existing implementation of BDK, which is coded in C++. Future implementations may get their specific subsections when required. diff --git a/getting-started/developing-with-bdk/compilation-and-deployment.md b/getting-started/developing-with-bdk/compilation-and-deployment.md new file mode 100644 index 0000000..1e8364e --- /dev/null +++ b/getting-started/developing-with-bdk/compilation-and-deployment.md @@ -0,0 +1,68 @@ +--- +description: Putting those bytes to work +--- + +# Compilation and deployment + +After ensuring your environment is properly set up, you can now compile and deploy your own local testnet. This is strongly recommended, as it will ensure your network and contracts compile and work as they should before deploying your project on a more serious environment. + +## Compiling + +The following commands will build the project out of tree within a folder called `build_local_testnet`, which will be used later by the deploying script included in the project (see "Manual deploy" below - this is not entirely necessary as the script will rebuild the entire project if needed, but we recommend doing it this way to avoid having two separate builds taking up space). + +First, create and move into the build folder: + +```bash +mkdir build_local_testnet && cd build_local_testnet +``` + +Then, configure CMake to build inside the folder (the `..` points to the root folder's `CMakeLists.txt`): + +```bash +cmake -DDEBUG=ON -DBUILD_TESTS=ON .. +``` + +The `DEBUG` flag enables debug symbols and the compiler's address sanitizer. It is `ON` by default. For release builds, this should be set to `OFF` as debug builds take a significant hit on performance. You can also set the `BUILD_TESTS` flag (`ON` by default) to toggle the compilation of unit tests, which can help with compilation times and/or RAM usage if not needed. Check the `CMakeLists.txt` file for more flags. + +Finally, build the project. Adjust `-j$(nproc)` accordingly to your system's CPU cores and/or memory limits if necessary, as some parts of the project can get really heavy RAM-wise during compilation: + +```bash +cmake --build . -- -j$(nproc) +``` + +## Running unit tests + +After building, assuming `BUILD_TESTS` is `ON`, you can optionally run a test bench with the following command: `./src/bins/bdkd-tests/bdkd-tests -d yes` (the `-d yes` parameter will give a verbose output). + +You can also use filter tags to test specific parts of the project (e.g. `./src/bins/bdkd-tests/bdkd-tests -d yes [utils]` will test all the components inside the `src/utils` folder, `[utils][tx]` will test only the transaction-related components inside that same folder, etc.). You can check all the available tags by doing a `grep -rw "\"\[.*\]\""` in the `tests` subfolder. You can also read the "BDK implementation" section for more details on how the project is structured. + +## Documentation + +The project also has built-in local documentation powered by [Doxygen](https://doxygen.nl). Make sure you have it installed in your system and run `doxygen` in the project's root folder to generate it. You can find the generated docs in the `docs` folder and open it in your browser of choice. + +## Deploying + +There are two ways to deploy an AppLayer node: *dockerized* and *manual*. Go back to the project's root folder and check the `scripts` subfolder - there are two main scripts there used for deploying the node. You can pick whichever one you prefer, depending on your needs. + +### Dockerized deploy + +You can deploy a node with Docker by running `./scripts/auto.sh`. Make sure you have both `docker` and `docker-compose` installed, as the script requires both to work. The script itself accepts several parameters. Running `./scripts/auto.sh help` will give you more info on each parameter. + +### Manual deploy + +To manually deploy a node, run `./scripts/AIO-setup.sh`. Make sure `tmux` is installed, as the script needs it to work. The script will create two folders at the project's root - `build_local_testnet` and `local_testnet`, used respectively for building and deploying a fresh new instance of a local testnet. + +Running the script again will stop the testnet, rebuild it, update the binaries and restart it on the spot. If you wish to manually stop the testnet for some reason, run `tmux kill-server`. You can also read the script to find out the specific names of the tmux sessions to manually restart or stop accordingly. + +**NOTE**: when re-deploying the testnet, if your wallet or RPC client keeps track of account nonce data, you must reset it as a network reset would set back their nonces to 0. [Here's how to do it in MetaMask, for example](https://support.metamask.io/hc/en-us/articles/360015488891-How-to-clear-your-account-activity-reset-account). + +You can use the following flags when calling the manual script to customize deployment: + +| Flag | Description | Default Value | +| --------------- | ------------------------------------------------ | ----------------- | +| --clean | Clean the build folder before building | false | +| --no-deploy | Only build the project, don't deploy the network | false | +| --debug=\ | Build in debug mode | true | +| --cores=\ | Number of cores to use for building | Maximum available | + +As an example, `./scripts/AIO-setup.sh --clean --no-deploy --debug=false --cores=4` will clean the build folder, only build the project, build in release mode and use 4 cores for building. Remember that GCC uses around 1.5-2GB of RAM per core, so we recommend adjusting the number of cores according to the available RAM on your system for more stability. \ No newline at end of file diff --git a/getting-started/developing-with-bdk/connecting-the-wallet.md b/getting-started/developing-with-bdk/connecting-the-wallet.md new file mode 100644 index 0000000..591cdea --- /dev/null +++ b/getting-started/developing-with-bdk/connecting-the-wallet.md @@ -0,0 +1,32 @@ +--- +description: Now it's time to take it for a ride! +--- + +# Connecting the wallet + +With your local testnet node running, it is now possible to configure and connect your preferred Web3 wallet to it and play around with an AppLayer-powered blockchain. As said at the beginning, we recommend using [Metamask](https://metamask.io) as it is the most popular one, but you're free to use any other client you wish. + +As an example, here's how to configure MetaMask to connect to your local testnet: + +| Field | Value | +| --------------- | ----------------------------------------------- | +| Network Name | AppLayer Local Testnet | +| New RPC URL | [http://127.0.0.1:8090](http://127.0.0.1:8090/) | +| Chain ID | 808080 | +| Currency Symbol | APPL | + +
+ +Once you're connected, import the following private key for the chain owner account: + +``` +0xe89ef6409c467285bcae9f80ab1cfeb3487cfe61ab28fb7d36443e1daa0c2867 +``` + +This account contains a huge number of APPL tokens from the get go and is able to call the `ContractManager` contract, deployed at the address: + +``` +0x0001cb47ea6d8b55fe44fdd6b1bdb579efb43e61 +``` + +Other details about the deployed testnet chain can be found in the project's README.md file. \ No newline at end of file diff --git a/understanding-contracts/contract-tester.md b/getting-started/developing-with-bdk/contract-tester.md similarity index 76% rename from understanding-contracts/contract-tester.md rename to getting-started/developing-with-bdk/contract-tester.md index c0abab6..7f54d1e 100644 --- a/understanding-contracts/contract-tester.md +++ b/getting-started/developing-with-bdk/contract-tester.md @@ -1,42 +1,41 @@ --- -description: A web application for testing your contracts. +description: A web application for testing your contracts --- # Contract Tester -We have developed a web application called [Contract Tester](https://github.com/AppLayerLabs/contract-tester), for testing the logic of customized contracts in a simple, fast and intuitive way. +We have also developed a web application called [Contract Tester](https://github.com/AppLayerLabs/contract-tester), for testing the logic of customized contracts in a simple, fast and intuitive way. -## Cloning the repository +It is assumed that you have read the previous sections before this one, have your environment setup and a local testnet already running in your machine. Some parts of the page may also require that you have read the Contract sections as it deals with deploying custom contracts. -First, clone the repository to your local device: +## Cloning and deploying + +First, clone the repository to your local device and move to it: ```bash git clone https://github.com/AppLayerLabs/contract-tester cd contract-tester ``` -## Deploying a local testnet - -As you're testing the interaction of your customized contracts with the AppLayer testnet, you should configure your environment and run a local testnet. See "Setting up the development environment" for more info. - -**Deploying the application** - -You can deploy the web application in two ways: _using Docker_, or _manually_. - ### Using Docker (recommended) -Install Docker on your system (if you don't have it installed already). Instructions for your system can be found at the links below: +Similar to the BDK, you can deploy the web application in two ways: _using Docker_, or _manually_. We recommend using Docker, so make sure it is installed in your system: * [Docker for Windows](https://docs.docker.com/docker-for-windows/install/) * [Docker for Mac](https://docs.docker.com/docker-for-mac/install/) * [Docker for Linux](https://docs.docker.com/desktop/install/linux-install/) -After installing Docker, build the image and run the container: +On Linux, you may need to run Docker commands as `sudo`, or you can follow [this post-install](https://docs.docker.com/engine/install/linux-postinstall/) so you don't have to. Command examples will be shown without `sudo` for simplicity purposes. + +After installing Docker, build the image: ```bash -# Build the image docker build -t contract-tester . -# Run the container +``` + +Then run the container: + +```bash docker run -p 3000:3000 contract-tester ``` @@ -44,13 +43,13 @@ docker run -p 3000:3000 contract-tester If you use VSCode as code editor, you can integrate it with the container. To do so, you need to install the [Docker extension](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-docker) and configure it to use the container. After installing it, there is a `docker-compose.yml` file on the root of the repository that you can use to build and run the container. -Right-click on it and select `Compose Up` to build and run the container, so the application will be deployed on `http://localhost:3000` (or the port you chose in the `docker-compose.yml` file). You can also use the `Compose Restart` option to restart the container, and `Compose Down` to stop and remove the container. +Right-click on it and select `Compose Up` (or run `docker compose up` in a terminal) to build and run the container, so the application will be deployed on `http://localhost:3000` (or the port you chose in the `docker-compose.yml` file). You can also use the `Compose Restart` option to restart the container, and `Compose Down` to stop and remove the container.

VSCode Docker integration

### Manual setup -If you want to run the project manually, you will need to install the following dependencies: +If you want to deploy the web application manually, you will need to install [NPM](https://www.npmjs.com/) along with a few dependencies: ```bash sudo apt install npm @@ -58,11 +57,9 @@ sudo npm install --global yarn npm install next react@latest react-dom ``` -Then, clone the repository, install the yarn dependencies and run the project: +Then, install the yarn dependencies and run the project: ```bash -git clone https://github.com/AppLayer/contract-tester.git -cd contract-tester yarn install yarn build yarn start @@ -70,7 +67,7 @@ yarn start ## Using the application -You should now be able to access the contract tester through `http://localhost:3000` (or the port you chose in the previous step). The application should look like this: +You should now be able to access the Contract Tester through `http://localhost:3000` (or the port you chose in the previous step). The application should look like this:

Main page of the application

@@ -95,7 +92,7 @@ After that, you should wait for the transaction to be mined and the contract to If you want to create a custom version of `ContractManager`, you will need to follow these steps: -1. Deploy the custom contract to the network (see "Creating a Dynamic Contract (Simple) > Deploying and testing" for more info on how to do it) +1. Deploy the custom contract to the network (see the Contract sections for more info on how to do it) 2. Take the custom contract address and put it at the top, in the “Contract Manager Address” field 3. Add the custom contract's ABI by clicking the button next to the "Enter your own JSON ABI" label, choosing your JSON ABI file, and then clicking "Upload" 4. Finally, click the “Get Functions” button to show the custom contract's functions and input fields @@ -116,7 +113,7 @@ First, make sure the contract is defined and added to the Contract Manager so yo **NOTE**: If you are passing an array as input values, just separate them using commas and the application will make all the necessary changes. -Let's use the `SimpleContract` example because we already registered it earlier in the Contract Manager section. First, develop the contract and deploy it to the network. After that, you can use the Contract Tester to interact with it. The solidity code for the contract is: +Let's use the `SimpleContract` example because it is already built-in to BDK and registered in `ContractManager`. First, develop the contract and deploy it to the network. After that, you can use the Contract Tester to interact with it. The Solidity code for the contract is: ```solidity // SPDX-License-Identifier: MIT @@ -156,7 +153,7 @@ contract SimpleContract { } ``` -We can use our own ABI generator tool described in the "Deploying and testing" section of the SimpleContract docs, or a third-party one like [Remix IDE](https://remix.ethereum.org) to generate the contract ABI. If using Remix, first create a Solidity file (.sol extension) and paste the code above. After that, you can compile the contract by clicking on the "Compile" button on the left sidebar. You should see something like this: +We can use our own ABI generator tool (see the Contract sections), or a third-party one like [Remix IDE](https://remix.ethereum.org) to generate the contract ABI. If using Remix, first create a Solidity file (.sol extension) and paste the code above. After that, you can compile the contract by clicking on the "Compile" button on the left sidebar. You should see something like this:

Ethereum Remix IDE

@@ -265,4 +262,4 @@ Now, you can use the Contract Tester to interact with the deployed `SimpleContra

Testing calls to a custom contract

-Now you can interact with the contract. You can set the name and value by using the "setName" and "setValue" functions, and get the name and value by using the "getName" and "getValue" functions. You can also check the transactions on Metamask. +Now you can interact with the contract. You can set the name and value by using the "setName" and "setValue" functions, and get the name and value by using the "getName" and "getValue" functions. You can also check the transactions on MetaMask. diff --git a/getting-started/developing-with-bdk/environment-setup.md b/getting-started/developing-with-bdk/environment-setup.md new file mode 100644 index 0000000..5a564ad --- /dev/null +++ b/getting-started/developing-with-bdk/environment-setup.md @@ -0,0 +1,136 @@ +--- +description: Prepping up the space +--- + +# Environment setup + +Before compiling BDK from source, we must ensure all of its dependencies are installed. Most of those dependencies come from the operating system (as in "directly from a Linux distro's repos"), but some of them are external and compiled alongside the project itself. + +## Forking + +Head over to the [BDK repository on Github](https://github.com/AppLayerLabs/bdk-cpp) and click the "Fork" button. You now have your own copy of the project. After that, clone your forked repository to your local machine with `git clone https://github.com/YOUR_USER_NAME/bdk-cpp.git`. Now you're ready to start developing your own local blockchain. + +

Go on and fork it!

+ +## Setup + +You can setup your local environment in two ways: _using Docker_, or _manually_. Manual setup has instructions for APT-based distros (e.g. Debian, Ubuntu, Mint, etc.), but other distros should work as long as you meet the minimum version requirements for all installed dependencies. + +### Docker (recommended) + +Using Docker is the recommended way to develop with the BDK. It will ensure that you have the correct environment to build and deploy the network, without worrying about dependencies or which host distro you're using. + +First, install Docker on your system (if you don't have it installed already). Instructions vary depending on the operating system you're using: + +* [Docker for Windows](https://docs.docker.com/docker-for-windows/install/) +* [Docker for Mac](https://docs.docker.com/docker-for-mac/install/) +* [Docker for Linux](https://docs.docker.com/desktop/install/linux-install/) + +On Linux, you may need to run Docker commands as `sudo`, or you can follow [this post-install](https://docs.docker.com/engine/install/linux-postinstall/) so you don't have to. Command examples will be shown without `sudo` for simplicity purposes. + +Once Docker is installed, go to the root directory of your cloned repository (where the `Dockerfile` is located), and run the following command: + +```bash +docker build -t bdk-cpp-dev:latest . +``` + +This will build the image and tag it as `bdk-cpp-dev:latest`. You can change the tag to whatever you want, but remember to change it at the next step. + +After building the image, run a container with the following command: + +```bash +# For Linux/Mac +docker run -it --name bdk-cpp -v $(pwd):/bdk-volume -p 8080-8099:8080-8099 -p 8110-8111:8110-8111 bdk-cpp-dev:latest +# For Windows +docker run -it --name bdk-cpp -v %cd%:/bdk-volume -p 8080-8099:8080-8099 -p 8110-8111:8110-8111 bdk-cpp-dev:latest +``` + +where: + +* `--name bdk-cpp` is an optional label for easier handling of the container (instead of using its ID directly) +* `$(pwd)` or `%cd%` is the absolute/full path to your repository's folder +* `:/bdk-volume` is the path inside the container where the BDK will be mounted. This volume is synced with the `bdk-cpp` folder inside the container +* The `-p` flags expose the ports used by the nodes. The example exposes the default ports 8080-8099 and 8110-8111 - if you use different ports, change them accordingly + +When running the container, you will be logged in as `root` and will be able to develop, build and deploy the network within the container. Remember that we are using our local repo as a volume, so every change in the local folder will be reflected to the container in real time, and vice-versa (so you can develop outside and use the container only for build and deploy). You can also integrate the container with your favorite IDE or editor. + +#### VSCode + Docker extension + +To integrate the container with VSCode, you need to install the [Docker extension](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-docker) and configure it to use the container. After installing it, there is a `docker-compose.yml` file in the prject's root folder that you can use to build and run the container. The only thing that you need to do is to change the `volumes` section to point to your local SDK folder: + +```yaml +volumes: + - /path/to/your/sdk:/orbitersdk-volume +``` + +After editing the `docker-compose.yml` file, right-click on it and select `Compose Up` (or run `docker compose up` in a terminal) to build and run the container so you can start developing on it. Click on the Docker extension icon on the left side of the VSCode window and you will see the container running. You can also right-click on the container and select `Attach Shell` to open a terminal on the container. + +
+ +### Manual setup + +If you don't want to use Docker for some reason, you can also build the project manually in your own system. Make sure your system provides at least the following dependencies: + +#### Toolchain binaries + +* **git** +* **GCC** with support for **C++23** or higher +* **Make** +* **CMake 3.19.0** or higher +* **Protobuf** (protoc + grpc_cpp_plugin) +* **tmux** (for deploying) +* (optional) **ninja** if you prefer it over make +* (optional) **mold** if you prefer it over ld +* (optional) **doxygen** for generating docs +* (optional) **clang-tidy** for linting + +#### Libraries + +* **Boost 1.83** or higher (components: *chrono, filesystem, program-options, system, thread, nowide*) +* **OpenSSL 1.1.1** / **libssl 1.1.1** or higher +* **libzstd** +* **CryptoPP 8.2.0** or higher +* **libscrypt** + * **libc-ares** +* **gRPC** (libgrpc and libgrpc++) +* **secp256k1** +* **ethash** + **keccak** +* **EVMOne** + **EVMC** +* **Speedb** + +Dependency versions should suffice out-of-the-box for at least the following distros (or greater, including their derivatives): + +* **Debian 13 (Trixie)** +* **Ubuntu 24.04 LTS (Noble Numbat)** +* **Linux Mint 22 (Wilma)** +* **Fedora 40** +* Any rolling release distro from around **May 2024** onwards (check the repos to be sure) + +#### The deps.sh script + +We provide a script called `scripts/deps.sh` which automates the process of checking and installing the project's dependencies on the system. We strongly recommend using it if building manually, unless you know what you're doing. + +The script accepts the following arguments (they should be run one at a time, e.g. `deps.sh --arg`): + +* `--check`: scans the system and confirms which dependencies are properly installed +* `--install`: installs missing dependencies not found during check +* `--cleanext`: clean up external dependencies (those compiled alongside the project), in case rebuilding them from a clean slate is necessary + +The script expects dependencies to be installed either on `/usr` or `/usr/local`, giving preference to the latter if it finds anything there. That way you can use a higher version of a dependency while still keeping your distro's default one. + +**Please note that installing most dependencies through the script only works on APT-based distros** (Debian, Ubuntu and derivatives) - you can still check the dependencies on any distro and install the few ones labeled as "external" (those are fetched through `git`), but if you're on a distro with another package manager and/or a distro older than one of the minimum ones listed above, you're on your own. + +#### Debian and GCC caveat + +For Debian specifically, you can (and should) use `update-alternatives` to register and set your GCC version to a more up-to-date build if required. + +If you're using a self-compiled GCC build out of the system path (e.g. `--prefix=/usr/local/gcc-X.Y.Z` instead of `--prefix=/usr/local`), don't forget to export its installation paths in your `PATH` and `LD_LIBRARY_PATH` env vars (to prevent e.g. `version GLIBCXX_.../CXXABI_... not found` errors). Put something like this in your `~/.bashrc` file for example, changing the version accordingly to whichever one you have installed: + +```bash +# For GCC in /usr/local +export LD_LIBRARY_PATH=/usr/local/lib64:$LD_LIBRARY_PATH + +# For self-contained GCC outside /usr/local +export PATH=/usr/local/gcc-14.2.0/bin:$PATH +export LD_LIBRARY_PATH=/usr/local/gcc-14.2.0/lib64:$LD_LIBRARY_PATH +``` \ No newline at end of file diff --git a/getting-started/setting-up-a-wallet.md b/getting-started/setting-up-a-wallet.md new file mode 100644 index 0000000..fb30aed --- /dev/null +++ b/getting-started/setting-up-a-wallet.md @@ -0,0 +1,61 @@ +--- +description: First things first +--- + +# Setting up a wallet + +Obviously, to interact with the Web3 ecosystem, you need a crypto wallet that integrates with it. You can use any modern Web3 wallet you prefer, but we recommend MetaMask as it is the most established and known one. + +Further instructions will assume MetaMask is being used for the sake of simplicity. If you already have your wallet installed and running or know how to do it yourself, you can skip this section entirely and go to the next one. If you're new to Web3 or crypto altogether, we'll guide you through installing MetaMask and setting up your wallet for the first time. + +## Installing MetaMask + +MetaMask is available both as an extension on desktop browsers (Firefox-based and Chromium-based) and as a native application on mobile platforms (Android and iOS). + +Go to the [MetaMask](https://metamask.io/) website and click or tap on "Get MetaMask". You will be redirected to your platform's app or extension store. If you wish, you can also get it directly from the [Firefox Add-On Store](https://addons.mozilla.org/pt-BR/firefox/addon/ether-metamask/) or [Chrome Web Store](https://chromewebstore.google.com/detail/metamask/nkbihfbeogaeaoehlefnkodbefgpgknn) (for desktop), [Google Play](https://play.google.com/store/apps/details?id=io.metamask) or the [Apple App Store](https://apps.apple.com/us/app/metamask-crypto-wallet/id1438144202) (for mobile). + +## Creating a MetaMask wallet + +After installation, open MetaMask and you'll be greeted with a welcome screen and two buttons - one for creating a new wallet, and another for importing an existing wallet using a backup seed. + +People familiar with crypto might already have a backup seed of their wallet - they would ideally just choose "Import an existing wallet", import their seed and skip straight to the next section. Since we're assuming the perspective of a new user who has never used a crypto wallet before, we'll create a new wallet from scratch. + +The first step is to check the "I agree to MetaMask's Terms of use" box and choose the "Create a new wallet" option, as shown here: + +

MetaMask on first boot

+ +MetaMask will then ask if you want to help with improving their software by sending basic usage and diagnostics data to them. You can do as you please here, either click on "I agree" or "No thanks". + +### Creating the wallet's password + +The next step is creating a strong password for your wallet. ***PLEASE NOTE IT DOWN AND STORE IT ON A SAFE AND SECURE PLACE!*** In the land of crypto, you're fully responsible for your wallet's security (this includes both the password *and* the seed, which we'll get to later). + +Once you have a good enough password, type it on the box, re-type it to confirm, carefully read and check the checkbox below, then proceed with creating your wallet: + +

Register your password and never lose it!

+ +### Storing the wallet's backup seed + +MetaMask will then ask you to secure your wallet by noting down your secret recovery phrase (also known as a "seed" or "backup seed"). We ***STRONGLY*** recommend you do this, as that's the ***ONLY*** way you can recover your wallet in case something bad happens to it. ***NEVER*** show or share it to ***ANYONE*** either, as that'll give them direct access to your funds. Just like with your password, you should note it down and store it on a safe and secure place only you know. + +We can't stress this enough. Always keep in mind: *"Not your keys, not your coins"*. + +After watching the video and reading the rest of the info on the screen, choose the "Secure my wallet" option: + +

This is very important! Do not skip!

+ +The next screen will show you a box where your backup seed is written. Click or tap on "Reveal Secret Recovery Phrase" (after ensuring you're totally isolated from prying eyes 👀) and note the words down in the order they're in: + +

This is meant for my eyes only...

+ +

...and I mean it!

+ +Once you're done, choose "Next". MetaMask will ask you to complete your own seed with the missing words to make sure you've properly noted it down. Type the missing words in order accordingly and choose "Confirm": + +

You know the words, right? ...right? 😬

+ +### Finishing + +You're done! The wallet is now created. Be sure to read what MetaMask has to say in the next few screens and proceed to your wallet's main screen. + +Now you're ready to add AppLayer's network to your wallet. We have separate instructions for our mainnet and testnet respectively - check the following subchapters according to where you want to go next. diff --git a/glossary.md b/glossary.md index 655fb3e..e1c96cb 100644 --- a/glossary.md +++ b/glossary.md @@ -1,5 +1,5 @@ --- -description: Definitions of some technical terms used throughout the documentation. +description: Definitions of some technical terms used throughout the documentation --- # Glossary diff --git a/how-applayer-works/README.md b/how-applayer-works/README.md deleted file mode 100644 index a593832..0000000 --- a/how-applayer-works/README.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -description: Exploring the functional elements of AppLayer. ---- - -# How AppLayer works - -This chapter aims to show an overview on the more conceptual parts of the AppLayer system and how they work. Check the subchapters for more details. - -AppLayer’s functionality is contained in five primary components: *Validators*, *Sentinels*, *AppLayers*, *Chain Abstraction* and *rdPoS*, the latter of which is separated into its own chapter. \ No newline at end of file diff --git a/how-applayer-works/appchains.md b/how-applayer-works/appchains.md deleted file mode 100644 index c47cfdc..0000000 --- a/how-applayer-works/appchains.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -description: How Application Chains work on AppLayer. ---- - -# Application Chains - -An application chain, also known as an AppLayer™, are blockchains built using AppLayer's Blockchain Development Kit (BDK) and are deployed on AppLayer's Chain Abstraction Layer. An AppLayer™ primarily enables developers to create a chain dedicated to a particular application, with the chain’s rules tailored to the security, performance, speed, decentralization and other service delivery needs of the application. These AppLayers are responsible for securing their own ledger of accounts and balances along with their own execution. - -AppLayer's BDK currently supports C++ and Solidity for development with plans for other languages such as Rust, C#, Golang, and more. These application chains compile to a binary to enable efficient execution in parallel to Solidity byte code. Additionally, whoever sets up an application chain is in charge of validating the transactions on it and attracting others to run validators. - diff --git a/how-applayer-works/bridging/README.md b/how-applayer-works/bridging/README.md deleted file mode 100644 index ebf7c05..0000000 --- a/how-applayer-works/bridging/README.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -description: How bridging works on AppLayer. ---- - -# Bridging - -There are inherent flexibility issues with **native** and **application-specific** chains when compared to traditional EVM chains. An application-specific chain is limited as it can not support a complete service that involves interaction between more than one application. Those problems could heavily damage the reputation of a project built on them. - -Our solution to this issue is to allow AppLayer-enabled blockchains to natively communicate with each other by using the Chain Abstraction Network (hereby denominated **CAN**) as a middleman, where AppLayer serves as an intermediary between two dApp chains trying to communicate with each other. We call that **bridging**. - -It's possible to bridge *arbitrary data* and *tokens*, both *between AppLayer nodes* and *between AppLayer and external networks*. - -## How is safety ensured? - -The Validator and Sentinel nodes that read from a given chain are determined using RandomGen, the trustless decentralized randomness generator developed by AppLayer Labs. We ensure to keep a fair selection of nodes, however, there is a possibility of a 51% attack. - -For example, in a network with 100 nodes, if a malicious user controls 51 of them, and all of them get selected for driving a cross-chain request and a block, they could collude and forward any message they want. - -We avoid this by introducing Sentinels to the network. Sentinels are AppLayer Labs-powered Validators that ensure this collusion does not happen. Sentinels can not create new blocks, but rather work together with Validators to fortify the network’s security. - diff --git a/how-applayer-works/bridging/applayer-to-applayer-token-bridging.md b/how-applayer-works/bridging/applayer-to-applayer-token-bridging.md deleted file mode 100644 index 5a257dd..0000000 --- a/how-applayer-works/bridging/applayer-to-applayer-token-bridging.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: How native chains exchange tokens on AppLayer. ---- - -# AppLayer-to-AppLayer Token Bridging - -The same method for arbitrary data bridging is used for token bridging, but there are extra checkups to make sure that a given chain is not minting another chain's tokens. - -Due to how the network was designed, when doing a cross-chain transaction we can only ensure that the data *exists*, not that it is *valid in context*. That breach allows a given chain to mint the native token of another chain, because the Chain Abstraction Network does not verify if the token is valid inside that network. - -We avoid this problem by keeping a "token table", which is just a "spreadsheet" of chains and their external token balances. Of course, this doesn't include the given chain's own native token, since it can freely mint its own token itself and does its own internal validations to avoid invalid minting conditions. - -For example, we have chains A, B, and C, each one with tokens of each other. The Chain Abstraction Network keeps track of: - -* How many B's and C's exist on A -* How many A's and C's exist on B -* How many A's and B's exist on C - -When bridging another chain's tokens, the Chain Abstraction Network checks if that chain has enough balance to do so. When bridging your own tokens, the Chain Abstraction Network only has to increase the balance at the target chain, since the `exit` transaction from your chain has to be included in one of your blocks, which means it has been verified and validated inside your own network, so there's no need to do it again from the outside. - -This method of bridging follows mint/burn mechanisms. - diff --git a/how-applayer-works/bridging/applayer-to-external-bridging.md b/how-applayer-works/bridging/applayer-to-external-bridging.md deleted file mode 100644 index 4fcc811..0000000 --- a/how-applayer-works/bridging/applayer-to-external-bridging.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: How AppLayer bridges with other blockchains. ---- - -# AppLayer-to-External Bridging (Ethereum, Solana, etc.) - -There are multiple edge cases related to external bridging. For example, it's not possible to natively push data into these chains without paying transaction fees, and those external networks are also limited on both processing power and how much signature verification can be done. - -Knowing this, at least for now, the current bridging implementation for them is handled by Sentinels and owned by AppLayer Labs and its most trusted partners to ensure operational safety. The contract checks signatures of Validators and Sentinels but only the Sentinels can write into the contracts on these chains. - -This method of bridging follows lock/release mechanisms, unless a token is fully integrated with mint/burn bridging. - diff --git a/how-applayer-works/sentinels.md b/how-applayer-works/sentinels.md deleted file mode 100644 index 0030dfd..0000000 --- a/how-applayer-works/sentinels.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: How Sentinels work on AppLayer. ---- - -# Sentinels - -Sentinels are similar to Validators, except they cannot create blocks nor act on their own. It is required that both randomly selected Validators and Sentinels send the same data to the requester, otherwise they will be reported to the network as a malicious node. AppLayer Labs and its partners host them to ensure that will not happen. - -Sentinels are open for trusted 3rd parties to host and are KYC'd entities with a high level of scrutiny. Eventually as the network and underlying technology matures, Sentinels will be phased out in favor of a fully permissionless system. - -Sentinels require 200,000 $APPL tokens as a prerequisite and are only accepted by the network after a whitelisting by AppLayer Labs. - diff --git a/how-applayer-works/validators.md b/how-applayer-works/validators.md deleted file mode 100644 index 532ec2a..0000000 --- a/how-applayer-works/validators.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: How validators work on AppLayer ---- - -# Validators - -A Validator is a computer typically housed in a datacenter and commonly referred to as a server, with a sole purpose of executing and securing a blockchain network. - -Validators require a token lockup of at least 200,000 $APPL through the `addValidator` transaction to be accepted for validation in the AppLayer Network, and to register 3rd-party stateful pre-compiles. They create blocks, generate the "randomness" seed used to select the next block creator, and are responsible for gathering and signing data on bridging and blocks. - -Those deploying an AppLayer, more commonly known as an application chain, will register their chain through a transaction with fees paid to AppLayer's Network. These tokens are distributed to the managing company currently developing AppLayer and will be allocated to the foundation treasury at a future date. - diff --git a/introducing-applayer/README.md b/introducing-applayer/README.md index 23e1e89..e653d95 100644 --- a/introducing-applayer/README.md +++ b/introducing-applayer/README.md @@ -1,8 +1,8 @@ --- -description: A quick introduction to what AppLayer does +description: A quick introduction to what AppLayer is and what it does --- # Introducing AppLayer -This chapter aims to introduce the core concepts behind AppLayer, such as the context it's being built in, the problems with existing projects, what are the solutions we perceive (as well as their caveats), and how AppLayer is meant to bridge those gaps to bring forward new ways for developing in the blockchain space, as well as the potential use cases it could cover. +This chapter introduces the core concepts behind AppLayer, such as the context it's being built in, the problems with existing projects, what are the solutions we perceive (as well as their caveats), and how AppLayer is meant to bridge those gaps to bring forward new ways for developing in the blockchain space, as well as the potential use cases it could cover. diff --git a/introducing-applayer/a-primer-on-smart-contracts-and-evms.md b/introducing-applayer/a-primer-on-smart-contracts-and-evms.md new file mode 100644 index 0000000..f58e2ab --- /dev/null +++ b/introducing-applayer/a-primer-on-smart-contracts-and-evms.md @@ -0,0 +1,37 @@ +--- +description: Detailing the building blocks of DeFi and the hydra we're trying to slay +--- + +# A Primer on Smart Contracts and EVMs + +Blockchains like [Ethereum](https://ethereum.org), [Avalanche](https://support.avax.network/en/articles/5417030-what-is-the-ethereum-virtual-machine-evm), [Tron](https://developers.tron.network/v4.4.2/docs/tvm) and [Bitcoin Cash](https://cashscript.org/) support an ecosystem of *smart contracts* - pieces of code that run on the network and enforce certain rules or specific logic for given operations. They act much like real-world contracts, but are inherently programmable, allowing them to do much more than traditional paper contracts. + +Smart contracts act as a foundation to build decentralized applications and concepts such as DeFi and blockchain gaming, while eliminating the necessity of "trust" usually provided by a third-party on a traditional real-world contract. You can read more about the topic in [Ethereum's website](https://ethereum.org/en/smart-contracts/). The most widely known implementation of such ecosystem is Ethereum's Virtual Machine ([EVM](https://ethereum.org/en/developers/docs/evm) for short), which pioneered and popularized the concept. + +## Limitations of the EVM and current solutions + +One of the biggest pain points of blockchain development since its inception is related to *performance*. Blockchains that use virtual machines share the same conceptual problem: contracts built on top of them often have limited speed and flexibility. This happens not only due to the contracts being coded in higher-level languages (as opposed to lower-level languages which are by nature more performant, although harder to handle), but also due to the nature of the virtual machines themselves, as they're built to be "generic computers with limited throughput". + +This is common on Ethereum and other EVM-based chains - most major scaling solutions use the same unsustainable technology, which introduces the same issues of scalability in every deployed layer. Having to share a "generic computer" with the whole world is *inefficient by design*. Forcing a chain to be both generic and decentralized at the same time puts heavy limits on which types of applications you can decentralize and how much you can decentralize them. + +As an example, in the case of the Ethereum Virtual Machine, you can *not* do any of the following: + +* Loop a function more than 50 times due to block gas limit constraints; +* Have a stack size larger than 16 variables due to constraints on the EVM itself; +* Parallelize multiple contract calls (every time a new block has multiple transactions that interact with multiple different contracts, you have to load the contract, parse and save changes to the database for *each* single one of these contracts, *in order*). + +As quoted by [Itamar](https://github.com/itamarcps): *"The biggest problem is that everyone is sharing the same computer, and that computer is a Commodore 64"*. + +## Seeking a new solution + +If the problem is inherently tied to the EVM, then... *why don't we just get rid of it?* This seems like a reasonable solution, but it can't be the only one. + +Given the current blockchain landscape is primarily dominated by the Solidity EVM, a widely established standard that has spanned over a decade, it is unrealistic for many Web3 companies and developers to completely remove it from existing applications. However, to properly scale a Web3 based application, we must venture outside the virtual machine. + +Thus, we'te taking an approach that is a hybrid of old and new: a modular blockchain that not only allows for *natively-coded contracts*, but also a natively-coded, performance-centric EVM with [*stateful pre-compiles*](https://medium.com/@AppLayerLabs/stateful-precompiles-evm-game-changers-or-another-overhyped-complexity-b064145b290e). In other words, a blockchain that doesn't *need* a virtual machine to run smart contracts, but still has support for existing contracts to be deployed as-is. Running an EVM with stateful pre-compiles unlocks performance optimizations to existing Solidity contracts and accelerates them with pre-compiled functions within the state, programmed in common performance-driven development languages such as C++, C#, Rust, and more. + +This approach brings a tremendous advantage, however, it also comes with its own set of problems: + +* It can be tricky to push new smart contract code into the network as a pre-compile. If you want to add or remove logic from your contract in the network, you have to force a mandatory update to all node operators - this could take *days*, which is a pretty big deal depending on how mission-critical the update is +* An application-specific pre-compiled contract might be too expensive to be natively executed on 1st party validation +* If the pre-compile is not hard-coded into the network and validated from a 3rd party, it must be ensured that this communication remains secure \ No newline at end of file diff --git a/introducing-applayer/a-primer-on-smart-contracts.md b/introducing-applayer/a-primer-on-smart-contracts.md deleted file mode 100644 index 282a6be..0000000 --- a/introducing-applayer/a-primer-on-smart-contracts.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -description: The building blocks of decentralized finance ---- - -# A Primer on Smart Contracts - -In the field of cryptocurrencies, *smart contracts* are pieces of code that can run on a blockchain and enforce certain rules or logic, much like real-world contracts. However, due to the inherent programmability provided by blockchain technology, smart contracts can go much further than that, acting as a foundation to build decentralized applications and concepts such as DeFi, while eliminating the necessity of "trust" usually provided by a third-party on a traditional real-world contract. - -Although a fairly old concept in theory, smart contracts in practice have been popularized by the [Ethereum](https://ethereum.org) blockchain and its implementation of a virtual machine - the "Ethereum Virtual Machine", or [EVM](https://ethereum.org/en/developers/docs/evm) - that made it possible to deploy and interact with them. Other coins such as [Avalanche](https://support.avax.network/en/articles/5417030-what-is-the-ethereum-virtual-machine-evm), [Tron](https://developers.tron.network/v4.4.2/docs/tvm) and [Bitcoin Cash](https://cashscript.org/) have followed along with their own implementations as time went on. - -[Ethereum's website](https://ethereum.org/en/smart-contracts/) has a good explanation of smart contracts in EVM and most major scaling solutions for Ethereum use the same unsustainable technology, introducing the same issues of scalability in every deployed layer. diff --git a/introducing-applayer/enter-applayer.md b/introducing-applayer/enter-applayer.md new file mode 100644 index 0000000..b326476 --- /dev/null +++ b/introducing-applayer/enter-applayer.md @@ -0,0 +1,50 @@ +--- +description: The light at the end of the tunnel +--- + +# Enter AppLayer + +What if there was a blockchain that incorporated stateful pre-compiles, allowing third parties to deploy and natively support these contracts within a single network that shares its state across all nodes? + +This is exactly what AppLayer does. It is a modular blockchain with multiple layers, including an EVM, stateful pre-compiles, and chain abstraction. + +The AppLayer Network is made up of three parts: + +* A Blockchain Development Kit (hereby denominated [**BDK**](https://github.com/AppLayerLabs/bdk-cpp)), with extensive documentation for developers to easily build their own app-specific chains with unprecedented freedom +* An EVM network built on top of the BDK which enables builders to deploy EVM smart contracts and scale with C++ stateful pre-compiles +* A network that allows bridging data and assets between app-specific chains and external chains called the Chain Abstraction Network (**CAN**) + +Therefore, blockchains built using the BDK are able to communicate with each other through AppLayer. + +## Key Features + +AppLayer brings a myriad of options for blockchain developers such as: + +* **Natively-coded contracts** - developers can code their contracts directly in the blockchain's native language, bypassing extra layers of abstraction to leverage the full potential of AppLayer +* **Out-of-the-box support for Solidity contracts** - those who already have contracts coded in Solidity can use AppLayer's built-in, natively-coded EVM as a drop-in replacement +* **Stateful pre-compiles** - Solidity contracts can use stateful pre-compiles from our built-in EVM to achieve a higher performance compared to conventional EVMs +* **State-of-the-art consensus protocol** - our consensus engine is currently powered by [CometBFT](https://cometbft.com/), but we have plans for expanding beyond it (see the Future Plans section for more info) +* **Validators** - our network is executed and secured by a type of node we call "Validator", responsible for creating, gathering and signing data on blocks, as well as generating a "randomness" seed used to select the next block creator in the chain + +## Potential use cases + +AppLayer provides all the essential tools to build a wide range of products, powering applications such as: + +* **DeFi**: build financial products with security and ease. Leveraging AppLayer's performance network enables a whole new generation of DeFi products such as facilitating millions of trades per second +* **Data Storage**: AppLayer makes data storage easier, more affordable, and secure. Builders can backup whole ledgers or fully decentralize any form of a database natively in the AppLayer network +* **GameFi**: gaming projects can use a performance-driven infrastructure to build game engines capable of leveraging a pure blockchain solution with a whole new range of in-game features + +Other potential use cases include (but not limited to): + +* Multiplayer games/servers +* Decentralized exchange +* Decentralized and hyper-available caching for dApp databases +* Decentralized e-mail +* HR portal +* VPN +* Cloud services +* Video rendering +* E-commerce +* Arbitrage bot +* Blockchain-enabled utilities such as water and power +* Supply chain and logistics diff --git a/introducing-applayer/the-problem-with-evms.md b/introducing-applayer/the-problem-with-evms.md deleted file mode 100644 index f0a0f9c..0000000 --- a/introducing-applayer/the-problem-with-evms.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: The hydra we're trying to slay ---- - -# The Problem With EVMs - -Chains with virtual machines share the same problem: contracts built on top of them often have limited speed and flexibility due to the nature of the virtual machines themselves, as they're built to be "generic computers with limited throughput". - -This is common on Ethereum and other EVM-based chains - having to share a "generic computer" with the whole world is *inefficient by design*. Forcing a chain to be both generic and decentralized at the same time puts heavy limits on which types of applications you can decentralize and how much you can decentralize them. - -## Limitations of the EVM - -For example, in the case of the Ethereum Virtual Machine, you can not do any of the following: - -* Loop a function more than 50 times due to block gas limit constraints; -* Have a stack size larger than 16 variables due to constraints on the EVM itself; -* Parallelize multiple contract calls (as in, every time a new block has multiple transactions that interact with multiple different contracts, you have to load the contract, parse and save changes to the database for *each* single one of these contracts, *in order*). - -As quoted by [Itamar](https://github.com/itamarcps): *"The biggest problem is that everyone is sharing the same computer, and that computer is a Commodore 64"*. - -## The solution - -If the problem is inherently tied to the EVM, then... *why don't we just get rid of it?* This seems like a reasonable solution, but it can't be the only one either. - -The current blockchain landscape is primarily dominated by the Solidity EVM, and for many Web3 companies it is unrealistic to completely remove it from existing applications. However, to properly scale a Web3 based application, we must venture outside the virtual machine. - -Our solution is a hybrid of old and new: a modular blockchain that not only allows for *natively-coded contracts*, but also a natively-coded, performance-centric EVM with *stateful pre-compiles*. In other words, a blockchain that doesn't *need* a virtual machine to run smart contracts, but still has support for existing contracts to be deployed as-is. - -Having an EVM running in parallel of stateful pre-compiles unlocks performance optimizations to existing Solidity contracts and accelerates them with pre-compiled functions within the state, programmed in common performance-driven development languages, such as C++, C#, Rust, and more. - -## The caveats (and how we're solving them) - -There’s a tremendous advantage to using native stateful pre-compiles, however, this approach also comes with its own set of problems: - -* It can be tricky to push new smart contract code into the network as a pre-compile. If you want to add or remove logic from your contract in the network, you have to force a mandatory update to all node operators - this could take *days*, which is a pretty big deal depending on how mission-critical the update is -* An application-specific pre-compiled contract might be too expensive to be natively executed on 1st party validation -* If the pre-compile is not hard-coded into the network and validated from a 3rd party, it must be ensured that this communication remains secure - -What if there was a blockchain system that incorporated stateful pre-compiles, allowing third parties to deploy and natively support these contracts within a single network that shares its state? - -This is exactly what AppLayer does. AppLayer is a modular blockchain of multiple layers, including an EVM, stateful pre-compiles, and chain abstraction. - diff --git a/introducing-applayer/what-is-applayer.md b/introducing-applayer/what-is-applayer.md deleted file mode 100644 index d298be3..0000000 --- a/introducing-applayer/what-is-applayer.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -description: The light at the end of the tunnel ---- - -# What is AppLayer? - -The AppLayer Network is made up of three parts: - -* A Blockchain Development Kit (hereby denominated [**BDK**](https://github.com/AppLayerLabs/bdk-cpp)), with extensive documentation for developers to easily build their own AppLayer's with unprecedented freedom -* An EVM network built on top of the Blockchain Development Kit which enables builders to deploy EVM smart contracts and scale with C++ stateful pre-compiles -* A network that allows the bridging of data and assets between these app-specific chains and external chains - we call it the Chain Abstraction Network (**CAN**) - -Therefore, blockchains built using the BDK are able to communicate with each other through AppLayer. - -## Potential use cases - -AppLayer provides all the essential tools to build a wide range of products for decentralized finance, data storage, gaming, and much more. It can power applications such as: - -* **DeFi**: build financial products with security and ease. Leveraging AppLayer's performance network enables a whole new generation of DeFi products such as facilitating millions of trades per second. -* **Data Storage**: AppLayer makes data storage easier, more affordable, and secure. Builders can store backups of whole ledgers or fully decentralize any form of a database natively in the AppLayer network. -* **GameFi**: gaming projects now have performance infrastructure to build game engines capable of leveraging a pure blockchain solution with a whole new range of in-game features. - -Other potential use cases include (but not limited to): - -* Multiplayer games/servers -* Decentralized exchange -* Decentralized and hyper available caching for dApps’ databases -* Decentralized e-mail -* HR portal -* VPN -* Cloud services -* Video rendering -* E-commerce -* Arbitrage bot -* Blockchain-enabled utilities such as water and power -* Supply chain and logistics - diff --git a/join-our-community.md b/join-our-community.md index bd32a10..09b4d36 100644 --- a/join-our-community.md +++ b/join-our-community.md @@ -1,5 +1,5 @@ --- -description: Where to follow AppLayer on social media. +description: Where to follow AppLayer on social media --- # Join our Community diff --git a/precompiled-contracts/README.md b/precompiled-contracts/README.md deleted file mode 100644 index 8da55d9..0000000 --- a/precompiled-contracts/README.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -description: A primer on natively-coded smart contracts in the AppLayer ecosystem. ---- - -# Precompiled contracts - -Precompiled contracts (also known as "native contracts" or "stateful pre-compiles" - "stateful" since they can maintain a state, and "pre-compile" because they're compiled machine code not interpreted by a virtual machine) are contracts coded with the blockchain's native language, with things like transaction parsing methods and arguments, as well as management and storage of the contract's variables in a database, being manually coded in the blockchain's native language (e.g. bdk-cpp in C++) to be tightly integrated with the blockchain itself. - -Similar to Solidity contracts, they can be used to employ any type of logic within the network, but unlike Solidity, they aren’t subject to EVM constraints. This means we can take advantage of that fact and have full control of the contract's logic, unleashing blazing fast performance, flexibility and power. - -The contract templates provided by AppLayer's BDK (in the `src/contract/templates` folder) are based on OpenZeppelin contracts, maintaining the same operational standards known in the Solidity ecosystem, but on native C++ (in the case of bdk-cpp). diff --git a/precompiled-contracts/dynamic-and-protocol-contracts.md b/precompiled-contracts/dynamic-and-protocol-contracts.md deleted file mode 100644 index b81fcd0..0000000 --- a/precompiled-contracts/dynamic-and-protocol-contracts.md +++ /dev/null @@ -1,156 +0,0 @@ ---- -description: Which kinds of pre-compiled contracts can be coded and managed with the BDK. ---- - -# Dynamic and Protocol Contracts - -AppLayer's BDK offers two main classes of contracts: **Dynamic Contracts** and **Protocol Contracts**. The differences between both classes come from how they are created and managed within the BDK. - -## Dynamic Contracts (recommended) - -* Are managed by the state and can only be created by `ContractManager`, which enables the chain owner to create an unlimited number of Dynamic Contracts -* Can use special types called **SafeVariables** (explained further) - an additional layer of protection that allows better control over whether variable changes are committed to the state or automatically reverted when necessary (e.g. when a transaction fails) -* Can only be called when a block is being processed -* Are directly loaded into memory and work very similarly to Solidity contracts - -AppLayer's BDK provides ready-to-use templates for the following Dynamic Contracts: - -* `ERC20` (template for an ERC20 token) -* `ERC20Wrapper` (template for an ERC20 wrapper) -* `ERC721` (template for an ERC721 token) -* `NativeWrapper` (template for a native asset wrapper) -* `DEXV2Factory` (template for a DEX factory) -* `DEXV2Library` (namespace for commonly used DEX functions) -* `DEXV2Pair` (template for a DEX contract pair) -* `DEXV2Router02` (template for a DEX contract router) -* `UQ112x112` (namespace for dealing with fixed point fractions in DEX contracts) - -Some contracts were converted directly from their OpenZeppelin counterparts and are also available for use: - -* `ERC721URIStorage` -* `Ownable` - -There are also specific contracts that only exist for internal testing purposes and are not meant to be used as templates: - -* `SimpleContract` (what it says on the tin - a simple contract, used for both testing and teaching purposes) -* `RandomnessTest` (contract for testing random number generation) -* `ERC721Test` (derivative contract meant to test the capabilities of the ERC721 template) -* `TestThrowVars` (contract meant to test SafeVariable commit/revert functionality using exception throwing) -* `ThrowTestA/B/C` (contracts meant to test nested call revert functionality) -* `SnailTracer` / `SnailTracerOptimized` (C++ conversions of the [SnailTracer](https://github.com/karalabe/snailtracer) contract, used for benchmarking purposes) -* `Pebble` (contract meant to be used in the testnet as a little NFT mining game) - -## Protocol Contracts - -* Are directly integrated into the blockchain, therefore not linked to the `ContractManager` class and not contained by the state, which removes some restrictions but adds others (explained further) -* Can be designed to process information beyond transaction calls, communicate with other nodes, access files within the current system, and even automatically call themselves when certain conditions are met - anything is possible, as long as you don't break your own blockchain (more on that later) -* Cannot use SafeVariables nor emit events, as they only work with Dynamic Contracts - so it's all up to the developer (for the most part) as to where to place the contract's variables and their commit/revert logic within the blockchain's source code - -## Managing contracts - -Contracts in the AppLayer network are managed by a few classes working together (check the `src/core` and `src/contract` folders for more info on each class): - -* `State` (`src/core/state.h`) is responsible for owning all the Dynamic Contracts registered in the blockchain (which you can get a list of by calling the class' `getCppContracts()` and/or `getEVMContracts()` functions, depending on which ones you need), as well as their global variables (name, address, owner, balances, etc.) -* `Storage` (`src/core/storage.h`) is responsible for properly storing contract data, such as emitted events and the transactions that triggered them -* `ContractManager` (`src/contract/contractmanager.h`) is responsible for solely creating and registering the contracts, then passing them to State (with the `ContractFactory` namespace providing helper functions to do so) -* `ContractHost` (`src/contract/contracthost.h`) is responsible for allowing contracts to interact with each other and to enable them to modify balances - this kind of inter-communication is done by intercepting calls of functions from registered contracts (if the functor/signature matches), which is done through either an `eth_call` request or a transaction processed from a block -* `ContractStack` (`src/contract/contractstack.h`) is responsible for managing alterations of data, like contract variables and account balances during nested contract call chains, by keeping a stack of changes made during a contract call, and automatically committing or reverting them in the account state when required (for non-view functions) - -The `ContractManager` class is itself a _Protocol Contract_, but it does not own or create any Protocol Contracts. Instead, they are created during blockchain initialization, and a reference to each of them is stored within the class, allowing it to access them directly. Dynamic Contracts, however, are fully owned and stored by State in an internal map. This ensures that each contract has a unique address, which is derived using a similar scheme as an EVM. - -Because of this, we don't necessarily need to know the type of the contract stored within the pointer, only during either creating it or loading it from the database. This is why all contracts inherit the `BaseContract` class, as it contains a `name_` variable specifying the name of the contract. This name must be the same as the name of the class, as it is used to identify the contract during loading and creation. - -Using `ContractHost` for contract inter-communication instead of accessing State directly allows us to do it in an isolated and secure way. For example, if we want to send tokens from a contract to another address, we don't access the balance directly from the State. Every time a function is called by a **user** (not another contract), the balances map available for contracts is empty, only populating what is currently being accessed and not previously available. When modifying the balance, only the mapping within `ContractStack` is modified, therefore allowing for multiple nested contract functions to revert in an atomic fashion while not affecting either the State or other contracts altogether. - -## Example of a contract - -Given the example Solidity contract: - -```cpp -// SPDX-License-Identifier: MIT -pragma solidity ^0.8.10; - -contract ExampleContract { - mapping(address => uint256) values; - function setValue(address addr, uint256 value) external { - values[addr] = value; - return; - } -} -``` - -The transpiled code should look similar to this (this is only an example, read further for more details): - -* Declaration (`ExampleContract.h`) - -```cpp -#include <...> -class ExampleContract : public DynamicContract { - private: - std::unordered_map values; // or boost::unordered_flat_map for example - // Const-reference as they are not changed by the function. - void setValue(const Address& addr, const uint256_t& value); - public: - // Constructor for creating the contract the first time. - // "address" is where the contract will be deployed. - // "creator" is the address that created the contract. - // "chainId" is the chain ID where the contract will operate. - ExampleContract( - const Address& address, const Address& creator, const uint64_t& chainId - ); - // Constructor for loading the already-created contract from the database. - ExampleContract( - const Address& address, const DB& db - ); - void callContractWithTransaction(const Tx& transaction); -} -``` - -* Definition (`ExampleContract.cpp`) - -```cpp -#include "ExampleContract.h" - -ExampleContract( - const Address& address, const Address& creator, const uint64_t& chainId -) : DynamicContract("ExampleContract", address, creator, chainId) { - // Read the "values" variables from DB - // Code generated by the transpiler from all local variables - // of the solidity contract, on the ExampleContract, you have values as an address => uint256 mapping - ... -} - -ExampleContract::ExampleContract( - const Address& address, - const DB& db -) : DynamicContract(address, db) { - // Same thing but the contract already exists in the database so you just load it entirely from there - ... -} - -void ExampleContract::setValue(const Address &addr, const uint256 &value) { - this->values[addr] = value; - return; -} - -void ExampleContract::callContractWithTransaction(const Tx& transaction) { - // CODE GENERATED BY THE TRANSPILLER - // USED TO ROUTE AND DECODE TRANSACTIONS - // THE IF USED HERE IS FOR EXAMPLE PURPOSES - // THE GENERATED CODE WILL BE USING DIFFERENT STRING ALGORITHMS IN ORDER TO MATCH - // FUNCTOR AND ARGUMENTS TO CONTRACT FUNCTION. - std::string_view txData = transaction.getData(); - auto functor = txData.substr(0,8); - // Keccak256("setValue(address,uint256)") - if (functor == Utils::hexToBytes("0x48461b56")) { - this->setValue(ABI::Decoder::decodeAddress(txData, 8), ABI::Decoder::decodeUint256(txData, 8 + 32)); - } - return; -} -``` - -## The BaseContract class - -The `BaseContract` class, declared in `src/contract/contract.h`, is the base class which all contracts derive from. This class holds all the [Solidity global variables](https://docs.soliditylang.org/en/v0.8.17/units-and-global-variables.html), besides variables common among these contracts (such as contract address). Have a look at the header file for further reference on its structure. - -Regarding the `callContractWithTransaction` and the `ethCallContract` functions, `callContractWithTransaction` is used by the State when calling from `processNewBlock()`, while `ethCallContract` is used by RPC to answer for `eth_call`. Strings returned by `ethCallContract` are hex strings encoded with the desired function result. diff --git a/precompiled-contracts/types-of-precompiled-contracts.md b/precompiled-contracts/types-of-precompiled-contracts.md deleted file mode 100644 index e5069da..0000000 --- a/precompiled-contracts/types-of-precompiled-contracts.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -description: How AppLayer differentiates precompiled contracts. ---- - -# Types of precompiled contracts - -AppLayer frequently references precompiled contracts in three ways: *1st-party*, *3rd-party*, and *AppChains/AppLayers*. - -**1st-party** contracts are contracts provided by AppLayer itself as ready-to-use templates in the `src/contract/templates` subfolder (e.g. ERC20, ERC721, etc.), officially supported and leveraging all the features of the blockchain. Those contracts are also available in the AppLayer EVM chain. - -**3rd-party** contracts act like a "slim Layer 2", in the sense that they are only processed, but have no concepts of consensus, blocks, transactions, etc. like a normal contract would. They essentially act like a "daemon" of sorts, running atop the main chain (just like a Layer 2), reading transactions made on it and reacting accordingly if said transaction happens to call it. The transaction is executed and the contract publishes the results back on the main chain (e.g. a user sent tokens to a 3rd-party exchange contract, once the transaction is confirmed on the main chain the contract will read the data field and execute its own logic, then send the exchanged tokens back on the main chain as its own transaction). - -**AppChains/AppLayers** act as a "full Layer 2" instead, having their own blocks, transactions, consensus, etc., but still depending on the main chain to execute their separate logic. - diff --git a/understanding-contracts/internal-and-external-contract-calls.md b/understanding-contracts/internal-and-external-contract-calls.md deleted file mode 100644 index 7707090..0000000 --- a/understanding-contracts/internal-and-external-contract-calls.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -description: Not all contract calls are equal. ---- - -# Internal and external contract calls - -In the context of interacting with smart contracts, calls can be classified into two types: *external* and *internal*. - -**External calls** occur when a contract is invoked from an external source, which can be either a user transaction (e.g. a block and its transactions are being processed) or a remote procedure call (RPC, e.g. an `eth_call` request). External calls serve as entry points into the smart contract from the outside world, initiating a transaction or a query. Within AppLayer's BDK, to process an external call, a `ContractHost` object is created, which establishes the execution context based on the specified parameters. - -**Internal calls** occur when a smart contract, already running as part of an ongoing transaction/chain of execution, calls another contract or a function within another contract. This is considered an internal process, as it happens within the chain of execution started by an external call. In AppLayer's BDK, internal calls utilize the same `ContractHost` object created for the initial external call. This maintains consistency and control over the execution environment, allowing for functions like commits or rollbacks within the same transactional context. - -Basically, an external call creates a new "chain of execution", while internal calls only exist within that same given chain of execution, thus they cannot create new chains by themselves. A chain of execution is a sequence of calls that are executed in a given external call, always composed of an initial external call and a set of internal calls (if applicable). diff --git a/understanding-contracts/setting-up-the-development-environment.md b/understanding-contracts/setting-up-the-development-environment.md deleted file mode 100644 index 01fbd21..0000000 --- a/understanding-contracts/setting-up-the-development-environment.md +++ /dev/null @@ -1,218 +0,0 @@ ---- -description: How to prep up for compiling AppLayer's BDK and running your own blockchain. ---- - -# Setting up the development environment - -This subchapter explains how to set up AppLayer's BDK, our open-core blockchain SDK project, to start creating and deploying your contracts in it. This is an overview/"more approachable" version of the project's README.md file - be sure to read it as well. - -You're able to tweak almost everything related to the BDK. We offer pre-existing solutions for all of those, but you are free to hack into them as you wish: - -* Consensus -* Block processing -* Transaction processing -* Contract processing -* Communication between nodes, etc. - -## Forking - -Head over to the [GitHub repository](https://github.com/AppLayerLabs/bdk-cpp) and click the "Fork" button. After that, you can clone your forked repository with `git clone` and start developing on your own local blockchain. - -

Go on and fork it!

- -## Setup - -You can setup the environment in two ways: _using Docker_, or _manually_. Manual setup has instructions for APT-based distros (e.g. Debian, Ubuntu, Mint, etc.), but other distros should work as long as you have all the dependencies installed. - -### Docker (recommended) - -Using the Docker image is the recommended way to develop on the BDK. It will ensure that you have the correct environment to build and deploy the network, without worrying about dependencies or which host distro you're using. - -Fork the project and clone your forked repository: - -```bash -# Clone your repository -git clone https://github.com/YOUR_USER_NAME/bdk-cpp.git -# Go to the project directory -cd bdk-cpp -# Switch to a branch for contract development on latest release (main branch) -git checkout -b contract-development main -``` - -Then, install Docker on your system (if you don't have it installed already). Instructions can be found at the links below: - -* [Docker for Windows](https://docs.docker.com/docker-for-windows/install/) -* [Docker for Mac](https://docs.docker.com/docker-for-mac/install/) -* [Docker for Linux](https://docs.docker.com/desktop/install/linux-install/) - -Once Docker is installed, go to the root directory of your cloned repository (where the `Dockerfile` is located), and run the following command: - -```bash -docker build -t bdk-cpp-dev:latest . -``` - -This will build the image and tag it as `bdk-cpp-dev:latest`. You can change the tag to whatever you want, but remember to change it at the next step. - -After building the image, run a container with the following command: - -```bash -# For Linux/Mac -docker run -it --name bdk-cpp -v $(pwd):/bdk-volume -p 8080-8099:8080-8099 -p 8110-8111:8110-8111 bdk-cpp-dev:latest -# For Windows -docker run -it --name bdk-cpp -v %cd%:/bdk-volume -p 8080-8099:8080-8099 -p 8110-8111:8110-8111 bdk-cpp-dev:latest -``` - -where: - -* `--name bdk-cpp` is an optional label for easier handling of the container (instead of using its ID directly) -* `$(pwd)` or `%cd%` is the absolute/full path to your repository's folder -* `:/bdk-volume` is the path inside the container where the BDK will be mounted. This volume is synced with the `bdk-cpp` folder inside the container -* The `-p` flags expose the ports used by the nodes - the example exposes the default ports 8080-8099 and 8110-8111, if you happen to use different ports, change them accordingly - -When running the container, you will be logged in as the root user and will be able to develop, build and deploy the network within the container. Remember that we are using our local repo as a volume, so every change in the local folder will be reflected to the container in real time, and vice-versa (so you can develop outside and use the container only for build and deploy). You can also integrate the container with your favorite IDE or editor. - -#### VSCode + Docker extension - -To integrate the container with VSCode, you need to install the [Docker extension](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-docker) and configure it to use the container. After installing it, there is a `docker-compose.yml` file on the root of the repository that you can use to build and run the container. The only thing that you need to do is to change the `volumes` section to point to your local SDK folder: - -```yaml -volumes: - - /path/to/your/sdk:/orbitersdk-volume -``` - -After editing the `docker-compose.yml` file, right-click on it and select `Compose Up` to build and run the container so you can start developing on it. Click on the Docker extension icon on the left side of the VSCode window and you will see the container running. You can also right-click on the container and select `Attach Shell` to open a terminal on the container. - -
- -### Manual setup - -You can follow these steps to build the BDK in your own system. Dependencies are divided logically into *toolchain binaries* and *libraries*: - -* *Toolchain binaries*: - * **git** - * **GCC** with support for **C++23** or higher - * **Make** - * **CMake 3.19.0** or higher - * **Protobuf** (protoc + grpc_cpp_plugin) - * **tmux** (for deploying) - * (optional) **ninja** if you prefer it over make - * (optional) **mold** if you prefer it over ld - * (optional) **doxygen** for generating docs - * (optional) **clang-tidy** for linting -* *Libraries*: - * **Boost 1.83** or higher (components: *chrono, filesystem, program-options, system, thread, nowide*) - * **OpenSSL 1.1.1** / **libssl 1.1.1** or higher - * **libzstd** - * **CryptoPP 8.2.0** or higher - * **libscrypt** - * **libc-ares** - * **gRPC** (libgrpc and libgrpc++) - * **secp256k1** - * **ethash** + **keccak** - * **EVMOne** + **EVMC** - * **Speedb** - -The versions of those dependencies should suffice out-of-the-box for at least the following distros (or greater, including their derivatives): - -* **Debian 13 (Trixie)** -* **Ubuntu 24.04 LTS (Noble Numbat)** -* **Linux Mint 22 (Wilma)** -* **Fedora 40** -* Any rolling release distro from around **May 2024** onwards (check their repos to be sure) - -#### Tips for dependencies - -There is a script called `scripts/deps.sh` which you can use to check if you have those dependencies installed (`deps.sh --check`), install them in case you don't (`deps.sh --install`), and clean up the external ones for reinstalling (`deps.sh --cleanext`). The script expects dependencies to be installed either on `/usr` or `/usr/local`, giving preference to the latter if it finds anything there (so you can use a higher version of a dependency while still keeping your distro's default one). - -**Please note that installing most dependencies through the script only works on APT-based distros** (Debian, Ubuntu and derivatives) - you can still check the dependencies on any distro and install the few ones labeled as "external" (those are fetched through `git`), but if you're on a distro with another package manager and/or a distro older than one of the minimum ones listed above, you're on your own. - -For Debian specifically, you can (and should) use `update-alternatives` to register and set your GCC version to a more up-to-date build if required. - -If you're using a self-compiled GCC build out of the system path (e.g. `--prefix=/usr/local/gcc-X.Y.Z` instead of `--prefix=/usr/local`), don't forget to export its installation paths in your `PATH` and `LD_LIBRARY_PATH` env vars (to prevent e.g. "version `GLIBCXX_...'/`CXXABI_...` not found" errors). Put something like this in your `~/.bashrc` file for example, changing the version accordingly to whichever one you have installed: - -```bash -# For GCC in /usr/local -export LD_LIBRARY_PATH=/usr/local/lib64:$LD_LIBRARY_PATH - -# For self-contained GCC outside /usr/local -export PATH=/usr/local/gcc-14.2.0/bin:$PATH -export LD_LIBRARY_PATH=/usr/local/gcc-14.2.0/lib64:$LD_LIBRARY_PATH -``` - -## Compiling - -After forking the project, you can now setup your own local testnet. This is strongly recommended, as it will ensure your environment is properly setup and that you are able to compile the project with your contracts in it. - -Clone your forked repository by following the steps below: - -```bash -# Clone your repository -git clone https://github.com/YOUR_USER_NAME/bdk-cpp.git -# Go to the project directory -cd bdk-cpp -# Switch to a branch for contract development on latest release (main branch) -git checkout -b contract-development main -``` - -After cloning, the following commands will build the project within the folder which `scripts/AIO-setup.sh` (a script that automatically deploys a local testnet) will use later. - -```bash -# Create the folder and enter it -mkdir build_local_testnet && cd build_local_testnet -# Configure cmake (DEBUG=ON will enable debug symbols and address sanitizer) -cmake -DDEBUG=ON .. -# Build the project - you can use either one of the lines below -make -j$(nproc) -# or... -cmake --build . -- -j$(nproc) -``` - -Adjust `-j$(nproc)` accordingly to your system if necessary, as some parts of the project can get really heavy RAM-wise during compilation. - -After building, you can optionally run a test bench with the following command: `./src/bins/bdkd-tests/bdkd-tests -d yes` (the `-d yes` parameter will give a verbose output). - -You can also use filter tags to test specific parts of the project (e.g. `./src/bins/bdkd-tests/bdkd-tests -d yes [utils]` will test all the components inside the `src/utils` folder, `[utils][tx]` will test only the transaction-related components inside utils, etc.). You can check all the available tags by doing a `grep -rw "\"\[.*\]\""` in the `tests` subfolder. - -## Deploying - -Currently there are two ways to deploy an AppLayer node: *dockerized* and *manual*. Go back to the project's root folder and check the `scripts` subfolder - there are two main scripts there used for deploying the node. You can pick whichever one you prefer, depending on your needs. - -### Dockerized deploy - -You can deploy a node using Docker by running `./scripts/auto.sh`. Make sure you have both `docker` and `docker-compose` installed, as the script requires both to work. The script itself accepts several parameters. Running `./scripts/auto.sh help` will give you more info on each parameter. - -### Manual deploy - -To manually deploy a node, run `./scripts/AIO-setup.sh`. Make sure `tmux` is installed, as the script needs it to work. The script will create two folders at the project's root - `build_local_testnet` and `local_testnet` - and build and deploy a fresh new instance of a local testnet. - -Running the script again will stop the testnet, rebuild it, replace it and restart it on the spot. If you wish to manually stop the testnet for some reason, run `tmux kill-server`. You can also read the script to find out the specific names of the tmux sessions to manually restart or stop accordingly. - -Note that, when re-deploying, if your wallet or RPC client keeps track of account nonce data, you must reset it as a network reset would set back their nonces to 0. [Here's how to do it in MetaMask, for example](https://support.metamask.io/hc/en-us/articles/360015488891-How-to-clear-your-account-activity-reset-account). - -You can use the following flags when calling the manual script to customize deployment: - -| Flag | Description | Default Value | -| --------------- | ------------------------------------------------ | ----------------- | -| --clean | Clean the build folder before building | false | -| --no-deploy | Only build the project, don't deploy the network | false | -| --debug=\ | Build in debug mode | true | -| --cores=\ | Number of cores to use for building | Maximum available | - -As an example, `./scripts/AIO-setup.sh --clean --no-deploy --debug=false --cores=4` will clean the build folder, only build the project, build in release mode and use 4 cores for building. Remember that GCC uses around 1.5GB of RAM per core, so we recommend adjusting the number of cores according to the available RAM on your system for more stability. - -## MetaMask config - -After deploying your node, you can configure and connect your preferred Web3 client/frontend to the network. We recommend using [Metamask](https://metamask.io) as it is the most popular one, but you're free to use any other client you wish. - -As an example, here's how to configure MetaMask to connect to your local testnet: - -| Field | Value | -| --------------- | ----------------------------------------------- | -| Network Name | AppLayer Local Testnet | -| New RPC URL | [http://127.0.0.1:8090](http://127.0.0.1:8090/) | -| Chain ID | 808080 | -| Currency Symbol | APPL | - -
- -Once you're connected, import the following private key for the chain owner account: `0xe89ef6409c467285bcae9f80ab1cfeb3487cfe61ab28fb7d36443e1daa0c2867`. This account contains 1000 APPL Tokens from the get go and is able to call the `ContractManager` contract. Other details about the deployed testnet chain can be found in the project's README.md file. diff --git a/understanding-rdpos/README.md b/understanding-rdpos/README.md deleted file mode 100644 index 5384fe1..0000000 --- a/understanding-rdpos/README.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -description: How AppLayer uses Random Deterministic Proof of Stake. ---- - -# Understanding rdPoS - -To keep consensus on such a blazing fast network without tripping up and/or having to deal with rollbacks, AppLayer uses a *random deterministic block creation* that allows only one given node to create a block for a given time, eliminating the risk of a block race condition in the network. - -This is implemented in AppLayer as its own consensus algorithm, called **rdPoS (Random Deterministic Proof of Stake)**, which empowers Validators and Sentinels to deal with block congestion and random number generation. This chapter aims to explain in-depth the rdPoS algorithm used by the AppLayer protocol, from concept to implementation. diff --git a/understanding-rdpos/blockchains-overview.md b/understanding-rdpos/blockchains-overview.md deleted file mode 100644 index a00ecda..0000000 --- a/understanding-rdpos/blockchains-overview.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: How existing blockchains handle rollbacks. ---- - -# Blockchains overview - -One of the biggest problems of blockchain development is handling block rollbacks. For example, on the Bitcoin chain, assuming there is a latest block that has another block after it. If a node receives a block that replaces the latest block, the next block and all the transactions in it are replaced too, which results in a rollback of the blockchain’s state by one block. - -The Bitcoin blockchain and other derivatives follow the "longest lived chain" rule (the chain with the most accumulated proof of work is the main chain). However, rollbacks unearth problems in that rule. For instance, when a developer is building dApps where they have to deal with such special conditions, it could take a greater effort depending on the size and/or complexity of the application. - -

Example of block rollback

- -In the diagram above, block C was replaced by block D followed by block E, rolling back the transactions made in block C. - -The solution to the problem is avoiding the rollback condition altogether. This can be done by deterministically defining which network node can create a block, thereby eliminating the block race condition and keeping everyone in the network synchronized to the same latest block. - -AppLayer implements this concept as **Random Deterministic Proof of Stake (rdPoS)**, which pairs a block congestion system and a random number generator system, allowing only one Validator to create a block at any given time, thus avoiding rollbacks and achieving consensus on ultra-fast networks. diff --git a/understanding-rdpos/how-rdpos-works.md b/understanding-rdpos/how-rdpos-works.md deleted file mode 100644 index a8b70e2..0000000 --- a/understanding-rdpos/how-rdpos-works.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -description: The logical flux of the rdPoS algorithm. ---- - -# How rdPoS works - -The heart of rdPoS is RandomGen, a deterministic uint256\_t generator used for almost everything related to consensus. This deterministic randomness ensures that every node has a chance to respond to a given request (block, randomness, bridging, etc.), while making sure that the nodes selected from the network are truly random and not problematic nodes operated by a malicious actor. - -For RandomGen to be viable, it needs to be seeded with a truly random number. RandomGen works as follows: - -* Every time a new block is about to be created, 16 random nodes are selected using RandomGen with the previous block’s randomness seed. -* These nodes make a 32-byte random string (`RandomnessSeed`) and hash it (`RandomnessHash`), then sign the hash and publish it to the network. -* After all the nodes have signed and published their hashes to the network, they can publish the real data, verifying that no one is trying to manipulate the end result. -* After the data is published and included in the block, the randomness seeds are concatenated and hashed, and the resulting hash is used to seed the next block creation. - -We have to pay attention to the current state of RandomGen to ensure that all nodes are always in the same internal state so they can properly synchronize with each other. - -#### Flux of the rdPoS algorithm during block creation - -A block in an rdPoS network is created by the following rules: - -* A list of network Validators is randomly generated and sorted using the "randomness" seed from the previous block. - -

New random Validator list being created

- -* The first Validator from the list will be the block creator, while at least 4 others will create a random 32-byte string and make two transactions with it: one containing the hash of said string, and another containing the string itself, both signed. - -

Validators performing a hash transaction broadcast

- -

Validators performing a random transaction broadcast

- -* The hashes are verified to make sure they match their respective random strings. - -

Transactions are checked against each other

- -* A new block is created by the first Validator, concatenating and hashing the other Validators' random strings to create a new "randomness" seed that will be used at the next block. - -

New randomness seed is generated

- -

New block is created with randomness seed, Validator signature and the transactions

- -* The block is signed and published to the network by the first Validator, while the other Validators verify that all transaction signatures (random and hashed) correspond with the list created at the start. -* The genesis block (the very first block in the chain) enforces a given fixed randomness to be valid, since there is no previous block before genesis to derive the randomness from. Additionally, at least five hardcoded Validators are needed to bootstrap the network, since each block requires at least 4 Validators to confirm the string and hash transaction signatures, and one for signing the block itself. - -As quoted by [Supra](https://github.com/Jean-Lessa): *"It's like playing poker but everyone hashes their hands first before showing the real cards"*. diff --git a/understanding-rdpos/slashing.md b/understanding-rdpos/slashing.md deleted file mode 100644 index da5f685..0000000 --- a/understanding-rdpos/slashing.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -description: Cutting out malicious nodes, carefully. ---- - -# Slashing - -What happens when a Validator answers with a "randomness" hash that does not match its own hash? Or when a Validator creates an invalid block with invalid transactions? Or when a Validator can't create a block before reaching the network's time limit? - -Misbehaving nodes suffer consequences. Since Validator signatures are required at protocol level, if a Validator tries to break the rules, it's possible to know who it is thanks to the signature, and "slash" it from the network. - -At the moment the biggest problem is a group of Validators being "slashed" and halting network activity. This can be solved by adding extra conditions to the network - for example, if the network wants to change the current block creator (in case it's been "slashed"), at least 90% of the Validators in the network have to sign a transaction consenting with the change, always maintaining the majority's consensus. diff --git a/understanding-rdpos/validator-implementations.md b/understanding-rdpos/validator-implementations.md deleted file mode 100644 index dd292e3..0000000 --- a/understanding-rdpos/validator-implementations.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -description: How to add Validators to the network. ---- - -# Validator implementations - -Developers get to choose how Validators are added to the network, but there are three pre-established implementation options: *permissionless*, *permissioned*, and *semi-permissioned*. - -## Permissionless - -In a permissionless implementation, all Validators have to participate in block creation to ensure that there’s no collusion. A totally permissionless network using rdPoS may face problems when the number of Validators in the network grows massively (e.g. 10,000 Validators), because the latency between those nodes could increase significantly. - -To solve this problem, the block time in a permissionless network should be bigger (e.g. 15-30 seconds) so all the nodes have enough time to respond. Validators can be added to a permissionless network by locking a certain amount of tokens in the contract that contains the rdPoS logic. - -## Permissioned - -In a permissioned implementation, every network has a "master address" that can add as many Validators as desired. The developer who sets up this implementation is responsible for keeping the chain up and running. The recommended number of nodes for this implementation is at least 32, but you can use more or less nodes depending on the application’s needs. - -## Semi-permissioned - -In a semi-permissioned implementation, both Validator types are used: - -* A normal Validator, simply called "Validator", similar to the permissionless one and added to the network the same way (by locking tokens); and -* A type of Validator called "Sentinel", similar to the permissioned one and added to the network the same way (with a master address). - -This implementation is different in that neither Validators nor Sentinels can create a block on their own. The randomness requires at least one of the transactions from a Sentinel and whoever publishes the block has to follow the Validator list order set during rdPoS processing. - -This means the network can have a smaller number of Validators (e.g. 16 instead of 32), requiring less computing power for verification but remaining highly secure. Since Sentinels take part in the process, every extra byte in the concatenated randomness seed will change the resulting hash, thus ensuring that any attempt of tampering is easily detected and dealt with. -