Support Node Pool within Task Group Stanza

[arodd]

Support node_pool at the Task Group level in jobs

Summary

Allow jobs to specify node_pool at the task group level (in addition to the existing job-level setting) so that different groups within the same job can target different node pools.

---

Background / Problem Statement

Today, node_pool can be defined at the job level, but placement is ultimately decided per allocation, and allocations map to task groups. This creates a practical limitation: a single job cannot schedule multiple groups into different node pools.

As a result, users must split what is logically one application/job into multiple Nomad jobs just to target different node pools. This becomes painful when groups need coordinated rollout via deployments, because Nomad does not provide a mechanism to coordinate deployment dependencies across separate jobs the way it can within a single job.

---

Goals

• Enable specifying node_pool at the group level so a single job can contain multiple groups targeting different node pools.
• Preserve existing behavior for jobs that only specify node_pool at the job level.
• Make placement intent clearer by aligning configuration with the allocation/task-group boundary.

---

Proposed Behavior

• Add node_pool (or equivalent) configuration at the task group stanza.
• If group.node_pool is set, it takes precedence for allocations created from that group.
• If group.node_pool is not set, fall back to job.node_pool (existing behavior).
• If neither is set, behavior remains unchanged (default scheduling rules).

---

Acceptance Criteria

1. Group-level targeting works

   • Given a job with two task groups groupA and groupB:

     • groupA configured with node_pool = "pool-a"
     • groupB configured with node_pool = "pool-b"

   • Allocations for groupA are only placed on nodes in pool-a, and allocations for groupB are only placed on nodes in pool-b.

2. Backwards compatibility

   • A job that specifies node_pool only at the job level behaves exactly as it does today.
   • Existing job specs continue to validate and run without modification.

3. Precedence rules

   • When both job-level and group-level node_pool are present, the group-level value is used for that group’s allocations.
   • This precedence is documented and covered by tests.

4. User feedback on invalid configuration

   • If a group references a node pool that does not exist (or is not available), the job fails validation or produces a clear, actionable scheduler/plan error (whichever is consistent with existing Nomad behavior for node pool validation).

5. Documentation

   • Job specification docs are updated to describe group-level node_pool, including examples and precedence with job-level node_pool.

---

Example Use Case

A single application job defines:

• api group that must run in a “services” node pool
• worker group that must run in a “batch” node pool

Today this requires two separate Nomad jobs, making coordinated changes harder. Group-level node_pool enables this to remain one cohesive job.

---

[martinmcnulty]

Funnily enough I just hit this today. We have a system job that runs a monitoring agent on every node in the cluster. Currently we achieve this with two Nomad jobs (one for the default node pool, and one for a different node pool), and I was about to try to combine them into a single job with two task groups when I realised it wasn't possible.

[dani]

For system jobs, you can use node_pool = "all" to target every hosts