Skip to content

Commit 1113e1a

Browse files
committed
fix server config
1 parent acf00e3 commit 1113e1a

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

cookbook/client/tinker/megatron/server_config.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@ applications:
6363
target_ongoing_requests: 16
6464
ray_actor_options:
6565
num_cpus: 0.1
66+
num_gpus: 1
6667

6768
# 3. Sampler Service - Runs inference / sampling using vLLM engine
6869
# Used for generating text from the model (e.g., evaluating LoRA results).

0 commit comments

Comments
 (0)