We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
1 parent acf00e3 commit 1113e1aCopy full SHA for 1113e1a
cookbook/client/tinker/megatron/server_config.yaml
@@ -63,6 +63,7 @@ applications:
63
target_ongoing_requests: 16
64
ray_actor_options:
65
num_cpus: 0.1
66
+ num_gpus: 1
67
68
# 3. Sampler Service - Runs inference / sampling using vLLM engine
69
# Used for generating text from the model (e.g., evaluating LoRA results).
0 commit comments