Scripts for uploading and setting record

.env_template

@@ -0,0 +1,5 @@
+SEED_PHRASE="one two three four five six seven eight nine ten eleven twelve"
+SITE_ADDRESS="0:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
+DOMAIN_ADDRESS="0:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
+GQL_TRANSPORT="https://mainnet.evercloud.dev/XXXXXX/graphql"
+JRPC_TRANSPORT="https://jrpc.everwallet.net"

.gitignore

@@ -0,0 +1,4 @@
+.idea
+__pycache__/
+.env
+sites/

README.md

@@ -1 +1,102 @@
+# Scripts for uploading html-sites to blockchain
 
+It uses [.ever domains](https://app.evername.io/) for storing sites.
+Every .ever domain is an NFT-contract with a set of records.
+There are two options for storing sites:
+1. Store directly in domain NFT-contract (record 1004)
+2. Store in separate contract and link it to domain (record 1005)
+
+Scripts are written in Python 3.
+
+## Install dependencies
+
+```shell
+python3 -m pip install -r requirements.txt
+```
+
+## Prepare the environment
+
+`.env` is used to configure these scripts. 
+You should create it like [env_template](.env_template). 
+Fill in the fields with your data.
+
+| Field            | Required | Description                                          |
+|------------------|----------|------------------------------------------------------|
+| `SEED_PHRASE`    | **yes**  | Seed phrase of the wallet that pays for transactions |
+| `SITE_ADDRESS`   | **yes**  | Address of the site contract                         |
+| `DOMAIN_ADDRESS` | **yes**  | Address of the .ever domain NFT                      |
+| `GQL_TRANSPORT`  | no       | Address of GraphQL transport                         |
+| `JRPC_TRANSPORT` | no       | Address of JSON-RPC transport                        |
+
+Take in mind that the account of the provided seed phrase has to be the owner of the domain and deployed site.
+
+### Transport
+
+Recommended way is to use GraphQL transport. 
+To get the right url of transport you should visit [everclod](https://www.evercloud.dev/), register and create a new project.
+Then you will get the url of transport.
+
+You have to specify at least one transport in the environment, so either `GQL_TRANSPORT` or `JRPC_TRANSPORT` should be set.
+
+## Store site directly
+
+This way is strongly limited to store **only 889 bytes**! Use it to store tiny sites or html-formatted text.
+Script tries to minify your html before storing, but it does not guarantee that it will fit.
+You could use `set_record.py` script for this with the `1004` record key.
+
+```shell
+python3 -m set_record -r 1004 sites/my_site.html
+```
+
+*Tip* You could store your html in `sites` folder as it is added to `.gitignore`.
+
+## Upload site to special contract
+
+### 1. Deploy site contract
+
+You could deploy [site contract from special repository](https://github.com/chumschat/eversite-contract).
+You will get the contract address and use it in the next steps.
+
+### 2. Upload site
+
+After deploying the site contract you could upload html to it.
+You could use `upload_site.py` script for this. 
+This script also tries to minify your html.
+
+```shell
+python3 -m upload_site sites/my_site.html
+```
+
+*Tip* You could store your html in `sites` folder as it is added to `.gitignore`.
+
+### 3. Link domain to site
+
+After uploading the site you should link it to the domain.
+You could use `set_record.py` script for this with the `1005` record key (default).
+
+```shell
+python3 -m set_record -r 1005
+```
+
+## Troubleshooting
+
+### RuntimeError: Message expired
+
+If you can't upload a site with constantly error `RuntimeError: Message expired`,
+and your transaction is not accepted, that often means that the contract hasn't enough gas on its balance.
+
+Script `upload_site.py` uses external messages to send data to the contract. 
+It is the cheapest way to send *large* amount of data to the contract.
+Such messages come without gas, so it is consumed from the site contract.
+Script divides html into chunks of 60 000 symbols, you could see it in the logs.
+Each of such chunk requires about 0.27-0.3 EVER for gas that should be on the contract balance.
+I.e. for 5 chunks you should have about 1.5 EVER on the contract balance.
+
+### Shrink amount of chunks
+
+Currently, if you upload a site with multiple chunks, you wouldn't be able to upload it again with the smaller number of chinks.
+Old high-indexed chunks will stay in the contract, and it's overall content will be corrupted.
+You cannot do anything without it. You could only destroy old site using it's `destruct` method and redeploy new contract.
+By doing this you could at least get the remaining gas back.
+
+Consider onchain sites as immutable.

abi/domain.abi.json

@@ -0,0 +1,58 @@
+{
+  "ABI version": 2,
+  "version": "2.2",
+  "header": ["pubkey", "time", "expire"],
+  "functions": [
+    {
+      "name": "supportsInterface",
+      "inputs": [
+        {"name":"answerId","type":"uint32"},
+        {"name":"interfaceID","type":"uint32"}
+      ],
+      "outputs": [
+        {"name":"support","type":"bool"}
+      ]
+    },
+    {
+      "name": "getRecords",
+      "inputs": [
+        {"name":"answerId","type":"uint32"}
+      ],
+      "outputs": [
+        {"name":"records","type":"map(uint32,cell)"}
+      ]
+    },
+    {
+      "name": "setRecord",
+      "inputs": [
+        {"name":"key","type":"uint32"},
+        {"name":"value","type":"cell"}
+      ],
+      "outputs": [
+      ]
+    },
+    {
+      "name": "deleteRecord",
+      "inputs": [
+        {"name":"key","type":"uint32"}
+      ],
+      "outputs": [
+      ]
+    },
+    {
+      "name": "getInfo",
+      "inputs": [
+        {"name":"answerId","type":"uint32"}
+      ],
+      "outputs": [
+        {"name":"id","type":"uint256"},
+        {"name":"owner","type":"address"},
+        {"name":"manager","type":"address"},
+        {"name":"collection","type":"address"}
+      ]
+    }
+  ],
+  "data": [],
+  "events": [],
+  "fields": []
+}

abi/eversite.abi.json

@@ -0,0 +1,44 @@
+{
+  "ABI version": 2,
+  "version": "2.3",
+  "header": ["pubkey", "time", "expire"],
+  "functions": [
+    {
+      "name": "upload",
+      "inputs": [
+        {"name":"part","type":"cell"},
+        {"name":"index","type":"uint8"}
+      ],
+      "outputs": [
+      ]
+    },
+    {
+      "name": "getDetails",
+      "inputs": [
+      ],
+      "outputs": [
+        {"name":"value0","type":"map(uint8,cell)"}
+      ]
+    },
+    {
+      "name": "owner",
+      "inputs": [
+      ],
+      "outputs": [
+        {"name":"owner","type":"uint256"}
+      ]
+    }
+  ],
+  "data": [
+    {"key":1,"name":"nonce","type":"uint16"}
+  ],
+  "events": [
+  ],
+  "fields": [
+    {"name":"_pubkey","type":"uint256"},
+    {"name":"_timestamp","type":"uint64"},
+    {"name":"_constructorFlag","type":"bool"},
+    {"name":"nonce","type":"uint16"},
+    {"name":"content","type":"map(uint8,cell)"}
+  ]
+}

config.py

@@ -0,0 +1,49 @@
+import os
+
+from dotenv import load_dotenv
+from pydantic import BaseModel, Field, model_validator
+from typing_extensions import Self
+import nekoton as nt
+
+
+class Configuration(BaseModel):
+    seed_phrase: str = Field(..., min_length=1)
+    site_address: str = Field(..., min_length=1)
+    domain_address: str = Field(..., min_length=1)
+    gql_transport: str
+    jrpc_transport: str
+
+    @model_validator(mode="after")
+    def check_model(self) -> Self:
+        if not nt.Address.validate(self.site_address):
+            raise ValueError("Invalid site address is provided in environment!")
+        if not nt.Address.validate(self.domain_address):
+            raise ValueError("Invalid domain address is provided in environment!")
+        return self
+
+    async def get_transport(self) -> nt.Transport:
+        if self.gql_transport:
+            transport = nt.GqlTransport([self.gql_transport])
+            await transport.check_connection()
+            return transport
+        elif self.jrpc_transport:
+            transport = nt.JrpcTransport(endpoint=self.jrpc_transport)
+            await transport.check_connection()
+            return transport
+        else:
+            raise ValueError("No valid transport is provided in environment!")
+
+    def get_keypair(self) -> nt.KeyPair:
+        seed = nt.Bip39Seed(self.seed_phrase)
+        return seed.derive()
+
+
+def load_config() -> Configuration:
+    load_dotenv()
+    return Configuration(
+        seed_phrase=os.getenv("SEED_PHRASE"),
+        site_address=os.getenv("SITE_ADDRESS"),
+        domain_address=os.getenv("DOMAIN_ADDRESS"),
+        gql_transport=os.getenv("GQL_TRANSPORT", ""),
+        jrpc_transport=os.getenv("JRPC_TRANSPORT", "")
+    )

domain.py

@@ -0,0 +1,33 @@
+import nekoton as nt
+from typing import Optional
+
+
+class Domain:
+
+    def __init__(self, transport: nt.Transport, address: nt.Address):
+        self._address = address
+        self._transport = transport
+
+        with open("abi/domain.abi.json") as f:
+            domain_abi_text = f.read()
+            domain_abi = nt.ContractAbi(domain_abi_text)
+
+        self._getInfo = domain_abi.get_function("getInfo")
+        assert self._getInfo is not None
+        self._setRecord = domain_abi.get_function("setRecord")
+        assert self._setRecord is not None
+
+    async def _get_account_state(self) -> Optional[nt.AccountState]:
+        return await self._transport.get_account_state(self._address)
+
+    async def get_info(self) -> Optional[dict]:
+        state = await self._get_account_state()
+
+        if state is None:
+            return None
+        else:
+            return self._getInfo.call(state, input={'answerId': 0}).output
+
+    async def get_set_record_body(self, record_key: int, content: nt.Cell) -> nt.Cell:
+        input_params = {"key": record_key, "value": content}
+        return self._setRecord.encode_internal_input(input_params)

eversite.py

@@ -0,0 +1,62 @@
+import nekoton as nt
+from typing import Optional
+
+
+class EverSite:
+
+    def __init__(self, transport: nt.Transport, address: nt.Address, keypair: nt.KeyPair):
+        self._address = address
+        self._transport = transport
+        self._keypair = keypair
+
+        with open("abi/eversite.abi.json") as f:
+            eversite_abi_text = f.read()
+            eversite_abi = nt.ContractAbi(eversite_abi_text)
+
+        self._upload = eversite_abi.get_function("upload")
+        assert self._upload is not None
+        self._getDetails = eversite_abi.get_function("getDetails")
+        assert self._getDetails is not None
+        self._owner = eversite_abi.get_function("owner")
+        assert self._owner is not None
+
+    async def _get_account_state(self) -> Optional[nt.AccountState]:
+        return await self._transport.get_account_state(self._address)
+
+    async def get_details(self) -> Optional[nt.ExecutionOutput]:
+        state = await self._get_account_state()
+
+        if state is None:
+            return None
+        else:
+            return self._getDetails.call(state, input={})
+
+    async def get_owner(self) -> Optional[nt.PublicKey]:
+        state = await self._get_account_state()
+
+        if state is None:
+            return None
+        else:
+            public_key_int = self._owner.call(state, input={}).output["owner"]
+            return nt.PublicKey.from_int(public_key_int)
+
+    async def build_upload(self, content: nt.Cell, index: int) -> nt.Cell:
+        input_params = {"index": index, "part": content}
+        return self._upload.encode_internal_input(input_params)
+
+    async def send_chunk(self, content: nt.Cell, index: int) -> nt.Transaction:
+        signature_id = await self._transport.get_signature_id()
+
+        external_message = self._upload.encode_external_message(
+            self._address,
+            input={
+                "index": index,
+                "part": content
+            },
+            public_key=self._keypair.public_key
+        ).sign(self._keypair, signature_id)
+
+        tx = await self._transport.send_external_message(external_message)
+        if tx is None:
+            raise RuntimeError("Message expired")
+        return tx

requirements.txt

@@ -0,0 +1,5 @@
+nekoton==0.1.18
+typing~=3.7.4.3
+htmlmin~=0.1.12
+python-dotenv==1.0.0
+pydantic~=2.1.1