From 32abfb2ceb8e9e4f9aef8cd0a0d3b0782a1cf02c Mon Sep 17 00:00:00 2001 From: "7. Sun" Date: Thu, 2 May 2024 13:10:19 +0100 Subject: [PATCH 1/3] Update README.md --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index da80cda..79ad6c6 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ -# testnet-doc -Documentation on node and validators +# Decentralised AI Training Platform by FLock.io +Documentation on node and validators for train.flock.io ## Fed Ledger ## Training Node From 14864df079a015a52a274b6330778679359a26e4 Mon Sep 17 00:00:00 2001 From: "7. Sun" Date: Thu, 2 May 2024 13:10:36 +0100 Subject: [PATCH 2/3] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 79ad6c6..7c628fa 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ # Decentralised AI Training Platform by FLock.io -Documentation on node and validators for train.flock.io +Documentation on node and validators for `train.flock.io` ## Fed Ledger ## Training Node From b23ed94a141116ade27ad33331594e1a45230f3e Mon Sep 17 00:00:00 2001 From: elihylui Date: Thu, 2 May 2024 13:29:30 +0100 Subject: [PATCH 3/3] add small edits --- README.md | 4 ++++ guides/validator-guide.md | 17 +++++++++-------- 2 files changed, 13 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 7c628fa..f7f3647 100644 --- a/README.md +++ b/README.md @@ -68,3 +68,7 @@ curl --location 'https://fed-ledger-staging.flock.io/api/v1/tasks/update-validat "status": "failed" }' ``` + +## Troubleshooting + +- You should be able to view and generate your API keys at `train.flock.io` once you have staked $FML. If you are not able to do so, please try disconnet your wallet, clear your cookies, and connect with your wallet again. Please also make sure you are connected with the wallet which you used to stake for tasks. diff --git a/guides/validator-guide.md b/guides/validator-guide.md index a8772cb..f92f291 100644 --- a/guides/validator-guide.md +++ b/guides/validator-guide.md @@ -2,8 +2,7 @@ - Web3 Wallet - You will use this to connect to the web app at train.flock.io -- $FML - FLock’s testnet tokens required for staking. You can check out the contract [here](https://sepolia.basescan.org/address/0xeF39985160FEdd389B46E6dB14d6aEa26cb5981A\)). - +- $FML - FLock’s testnet tokens required for staking. You can check out the contract [here](https://sepolia.basescan.org/address/0xeF39985160FEdd389B46E6dB14d6aEa26cb5981A\)). You should first get whitelisted [here](https://blog.flock.io/news/trainflock), then obatin $FML at our faucet [here](https://train.flock.io/faucet). - Base Sepolia ETH - FLock.io testnet is deployed on Base Sepolia. You will need Base Sepolia ETH for gas to execute the staking transactions. @@ -101,9 +100,9 @@ After receiving a validation assignment (see Step 3) you must run the validation ```bash FLOCK_API_KEY="" python validate.py \ --model_name_or_path \ ---template_name \ +--base_model \ --eval_file ./data/dummy_data.jsonl \ ---max_seq_length 128 \ +--context_length 128 \ --assignment_id \ --validation_args_file ``` @@ -114,7 +113,7 @@ FLOCK_API_KEY="" python validate.py \ 4. Replace `` with the corresponding value from the response body in Step 3 -5. Replace `` with the `validation_set_url` value from the response body in Step 3 +5. For ``, please navigate to the url in `validation_set_url` in the response body in Step 3 and download the file. `` should be pointing to where you save this file. Note that this link expires in 60 minutes so be sure to download it in time. 6. Replace ` `with the ` `value from the response body in Step 3 @@ -124,14 +123,16 @@ FLOCK_API_KEY="" python validate.py \ 2. `Validation_config_cpu.json.example `to run on CPU -8. Run the command +8. Go to https://github.com/FLock-io/llm-loss-validator, git clone the repository and run the validation script from there. -The `eval_loss` value is the loss score you’ve submitted for this validation assignment. +You should get a response similar to the following: ```bash -{"id":1,"task_submission":{"task_id":23,"data":{"hg_repo_id":"Nicoooolasweee/qwen-1.5-4b-function","base_model":"qwen1.5"},"id":11},"assigned_at":"2024-04-24T18:21:45.047503","status":"completed","data":{"loss":0.17}}%  +evaluate result is {'eval_loss': 1.5328044891357422, 'eval_runtime': 1091.4278, 'eval_samples_per_second': 0.916, 'eval_steps_per_second': 0.458} ``` +The `eval_loss` value is the loss score you’ve submitted for this validation assignment. + _NOTE: If you couldn't complete the validation task for this submission, you can manually submit a failed result to Fed Ledger so that you can continue on other validation assignments._ ```bash