Skip to content

Edits #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
elihylui wants to merge 4 commits into
base: mattflock-patch-1
Choose a base branch
from
Open

Edits #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
32abfb2
Update README.md
sjhddh May 2, 2024
14864df
Update README.md
sjhddh May 2, 2024
52f7a71
Merge branch 'mattflock-patch-1' into edits
elihylui May 2, 2024
b23ed94
add small edits
elihylui May 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# testnet-doc
Documentation on node and validators
# Decentralised AI Training Platform by FLock.io
Documentation on node and validators for `train.flock.io`

## Fed Ledger
## Training Node
Expand Down Expand Up @@ -68,3 +68,7 @@ curl --location 'https://fed-ledger-staging.flock.io/api/v1/tasks/update-validat
"status": "failed"
}'
```

## Troubleshooting

- You should be able to view and generate your API keys at `train.flock.io` once you have staked $FML. If you are not able to do so, please try disconnet your wallet, clear your cookies, and connect with your wallet again. Please also make sure you are connected with the wallet which you used to stake for tasks.
17 changes: 9 additions & 8 deletions guides/validator-guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@

- Web3 Wallet - You will use this to connect to the web app at train.flock.io

- $FML - FLock’s testnet tokens required for staking. You can check out the contract [here](https://sepolia.basescan.org/address/0xeF39985160FEdd389B46E6dB14d6aEa26cb5981A\)).

- $FML - FLock’s testnet tokens required for staking. You can check out the contract [here](https://sepolia.basescan.org/address/0xeF39985160FEdd389B46E6dB14d6aEa26cb5981A\)). You should first get whitelisted [here](https://blog.flock.io/news/trainflock), then obatin $FML at our faucet [here](https://train.flock.io/faucet).


- Base Sepolia ETH - FLock.io testnet is deployed on Base Sepolia. You will need Base Sepolia ETH for gas to execute the staking transactions.
Expand Down Expand Up @@ -101,9 +100,9 @@ After receiving a validation assignment (see Step 3) you must run the validation
```bash
FLOCK_API_KEY="<your-api-key>" python validate.py \
--model_name_or_path <hg_repo_id> \
--template_name <base_model> \
--base_model <base_model> \
--eval_file ./data/dummy_data.jsonl \
--max_seq_length 128 \
--context_length 128 \
--assignment_id <assignment-id> \
--validation_args_file <validation_config.json.example OR validation_config_cpu.json.example>
```
Expand All @@ -114,7 +113,7 @@ FLOCK_API_KEY="<your-api-key>" python validate.py \

4. Replace `<base_model>` with the corresponding value from the response body in Step 3

5. Replace `<eval_file>` with the `validation_set_url` value from the response body in Step 3
5. For `<eval_file>`, please navigate to the url in `validation_set_url` in the response body in Step 3 and download the file. `<eval_file>` should be pointing to where you save this file. Note that this link expires in 60 minutes so be sure to download it in time.

6. Replace `<assignment-id> `with the `<id> `value from the response body in Step 3

Expand All @@ -124,14 +123,16 @@ FLOCK_API_KEY="<your-api-key>" python validate.py \

2. `Validation_config_cpu.json.example `to run on CPU

8. Run the command
8. Go to https://github.com/FLock-io/llm-loss-validator, git clone the repository and run the validation script from there.

The `eval_loss` value is the loss score you’ve submitted for this validation assignment.
You should get a response similar to the following:

```bash
{"id":1,"task_submission":{"task_id":23,"data":{"hg_repo_id":"Nicoooolasweee/qwen-1.5-4b-function","base_model":"qwen1.5"},"id":11},"assigned_at":"2024-04-24T18:21:45.047503","status":"completed","data":{"loss":0.17}}% 
evaluate result is {'eval_loss': 1.5328044891357422, 'eval_runtime': 1091.4278, 'eval_samples_per_second': 0.916, 'eval_steps_per_second': 0.458}
```

The `eval_loss` value is the loss score you’ve submitted for this validation assignment.

_NOTE: If you couldn't complete the validation task for this submission, you can manually submit a failed result to Fed Ledger so that you can continue on other validation assignments._

```bash
Expand Down