Skip to content

Workflow: Spawner should prioritize work items when maxConcurrency limits selectionΒ #435

@axon-agent

Description

@axon-agent

πŸ€– Axon Agent @gjkim42

Summary

When a TaskSpawner discovers more work items than maxConcurrency allows, it creates Tasks for whichever items the source returns first β€” with no priority awareness. For GitHub issues, this means the API's default sort order (created descending) determines which issues get worked on. A priority/critical-urgent issue created last week can be starved by three priority/backlog issues created today.

This proposal adds priority-aware work item selection to the spawner so that higher-priority items are always scheduled first when concurrency is limited.

Problem

Current behavior

Code reference: cmd/axon-spawner/main.go:204-216

The spawner iterates newItems in discovery order and creates Tasks until maxConcurrency is reached:

for _, item := range newItems {
    if maxConcurrency > 0 && int32(activeTasks) >= maxConcurrency {
        log.Info("Max concurrency reached, skipping remaining items")
        break
    }
    // ... create Task ...
}

For GitHub sources, Discover() returns issues in the GitHub API's default order (created descending). There is no sort or direction parameter set (internal/source/github.go:260-276).

Why this matters

With maxConcurrency: 3 (Axon's own self-development config), if there are 5 eligible issues:

The spawner creates Tasks for #500, #400, #300 (most recently created) and skips #200 and #100 β€” the critical-urgent issue is starved.

Real-world evidence

Axon's self-development uses priority labels (priority/critical-urgent, priority/imporant-soon, priority/import-longterm, priority/backlog) applied by the triage agent or maintainer. But the axon-workers spawner ignores these labels entirely. The triage classification work (done by axon-triage) is effectively wasted for scheduling purposes.

Currently there are 12+ open issues with actor/axon and triage-accepted that have different priority levels. When axon/needs-input is removed from several at once, the spawner has no way to pick the most important ones first.

Proposed Solution

Option A: priorityLabels field on GitHubIssues (recommended)

Add a configurable label-to-priority mapping in the GitHubIssues spec:

type GitHubIssues struct {
    // ... existing fields ...

    // PriorityLabels maps label prefixes to priority order.
    // Items are sorted by the first matching label, from highest
    // priority (index 0) to lowest. Items without a matching label
    // are scheduled last.
    // Example: ["priority/critical-urgent", "priority/imporant-soon",
    //           "priority/import-longterm", "priority/backlog"]
    // +optional
    PriorityLabels []string `json:"priorityLabels,omitempty"`
}

Spawner changes (cmd/axon-spawner/main.go):

After filtering newItems, sort by priority before creating Tasks:

if len(priorityLabels) > 0 {
    sort.SliceStable(newItems, func(i, j int) bool {
        pi := priorityIndex(newItems[i].Labels, priorityLabels)
        pj := priorityIndex(newItems[j].Labels, priorityLabels)
        return pi < pj
    })
}

Where priorityIndex returns the index of the first matching priority label (lower = higher priority), or len(priorityLabels) for items with no matching label.

Option B: Sort at the source level

Instead of a new API field, the GitHubSource.Discover() could accept a sort configuration. However, GitHub's API only supports sorting by created, updated, or comments β€” not by arbitrary labels. Priority sorting must happen client-side after discovery.

Option C: Sort in the spawner generically

Add a sort field to TaskTemplate or TaskSpawnerSpec that applies to all source types:

type TaskSpawnerSpec struct {
    // ... existing fields ...

    // Sort defines how discovered items are ordered before task
    // creation. When maxConcurrency limits how many tasks are
    // created per cycle, higher-priority items are created first.
    // +optional
    Sort *SortConfig `json:"sort,omitempty"`
}

type SortConfig struct {
    // LabelPriority sorts items by the first matching label.
    // Index 0 is highest priority.
    // +optional
    LabelPriority []string `json:"labelPriority,omitempty"`
}

This is more general but adds complexity. Option A is simpler and covers the primary use case.

Self-development configuration change

With Option A, axon-workers.yaml would add:

spec:
  when:
    githubIssues:
      labels:
        - actor/axon
      excludeLabels:
        - axon/needs-input
      priorityLabels:
        - priority/critical-urgent
        - priority/imporant-soon
        - priority/import-longterm
        - priority/backlog

Implementation scope

Files to change

File Change
api/v1alpha1/taskspawner_types.go Add PriorityLabels to GitHubIssues
internal/source/source.go Add SortByLabelPriority utility function
cmd/axon-spawner/main.go Sort newItems before task creation loop
internal/source/source_test.go Tests for priority sorting
cmd/axon-spawner/main_test.go Integration test for priority ordering
self-development/axon-workers.yaml Add priorityLabels config

Backward compatibility

  • PriorityLabels is optional; when empty, behavior is unchanged (items processed in discovery order)
  • No CRD version change needed (additive field)
  • Existing TaskSpawners continue to work without modification

Related

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions