diff --git a/README.md b/README.md index da80cda..f7f3647 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ -# testnet-doc -Documentation on node and validators +# Decentralised AI Training Platform by FLock.io +Documentation on node and validators for `train.flock.io` ## Fed Ledger ## Training Node @@ -68,3 +68,7 @@ curl --location 'https://fed-ledger-staging.flock.io/api/v1/tasks/update-validat "status": "failed" }' ``` + +## Troubleshooting + +- You should be able to view and generate your API keys at `train.flock.io` once you have staked $FML. If you are not able to do so, please try disconnet your wallet, clear your cookies, and connect with your wallet again. Please also make sure you are connected with the wallet which you used to stake for tasks. diff --git a/guides/validator-guide.md b/guides/validator-guide.md index a8772cb..f92f291 100644 --- a/guides/validator-guide.md +++ b/guides/validator-guide.md @@ -2,8 +2,7 @@ - Web3 Wallet - You will use this to connect to the web app at train.flock.io -- $FML - FLock’s testnet tokens required for staking. You can check out the contract [here](https://sepolia.basescan.org/address/0xeF39985160FEdd389B46E6dB14d6aEa26cb5981A\)). - +- $FML - FLock’s testnet tokens required for staking. You can check out the contract [here](https://sepolia.basescan.org/address/0xeF39985160FEdd389B46E6dB14d6aEa26cb5981A\)). You should first get whitelisted [here](https://blog.flock.io/news/trainflock), then obatin $FML at our faucet [here](https://train.flock.io/faucet). - Base Sepolia ETH - FLock.io testnet is deployed on Base Sepolia. You will need Base Sepolia ETH for gas to execute the staking transactions. @@ -101,9 +100,9 @@ After receiving a validation assignment (see Step 3) you must run the validation ```bash FLOCK_API_KEY="" python validate.py \ --model_name_or_path \ ---template_name \ +--base_model \ --eval_file ./data/dummy_data.jsonl \ ---max_seq_length 128 \ +--context_length 128 \ --assignment_id \ --validation_args_file ``` @@ -114,7 +113,7 @@ FLOCK_API_KEY="" python validate.py \ 4. Replace `` with the corresponding value from the response body in Step 3 -5. Replace `` with the `validation_set_url` value from the response body in Step 3 +5. For ``, please navigate to the url in `validation_set_url` in the response body in Step 3 and download the file. `` should be pointing to where you save this file. Note that this link expires in 60 minutes so be sure to download it in time. 6. Replace ` `with the ` `value from the response body in Step 3 @@ -124,14 +123,16 @@ FLOCK_API_KEY="" python validate.py \ 2. `Validation_config_cpu.json.example `to run on CPU -8. Run the command +8. Go to https://github.com/FLock-io/llm-loss-validator, git clone the repository and run the validation script from there. -The `eval_loss` value is the loss score you’ve submitted for this validation assignment. +You should get a response similar to the following: ```bash -{"id":1,"task_submission":{"task_id":23,"data":{"hg_repo_id":"Nicoooolasweee/qwen-1.5-4b-function","base_model":"qwen1.5"},"id":11},"assigned_at":"2024-04-24T18:21:45.047503","status":"completed","data":{"loss":0.17}}%  +evaluate result is {'eval_loss': 1.5328044891357422, 'eval_runtime': 1091.4278, 'eval_samples_per_second': 0.916, 'eval_steps_per_second': 0.458} ``` +The `eval_loss` value is the loss score you’ve submitted for this validation assignment. + _NOTE: If you couldn't complete the validation task for this submission, you can manually submit a failed result to Fed Ledger so that you can continue on other validation assignments._ ```bash