Add error categorization proto schema and executor classifier

[dejanzele]

What type of PR is this?

Feature (PR 1 of 4)

What this PR does / why we need it

Adds the proto schema and shared building blocks for error categorization in Armada.

• Adds FailureInfo message to the Error proto with fields: exit_code, termination_message, categories, container_name
• Adds errormatch package (internal/common/errormatch/) with shared matching primitives: ExitCodeMatcher (In/NotIn operators), RegexMatcher, and Kubernetes condition constants (OOMKilled, Evicted, DeadlineExceeded)
• Adds categorizer package (internal/executor/categorizer/) - configurable classifier that matches pod failures against rules (exit codes, termination messages, Kubernetes conditions) and assigns named categories, with optional containerName scoping per rule
• Adds ExtractFailureInfo() in pod_status.go to extract exit code, termination message, and container name from Kubernetes pod status into the FailureInfo proto
• Adds ErrorCategories config field under ApplicationConfiguration for defining category rules

Nothing is wired into the event reporting path yet - this PR provides the building blocks that PR #4745 connects.

Which issue(s) this PR fixes

Part of #4713 (Error Categorization)

Special notes for your reviewer

• This is PR 1 of 4: Proto + classifier (this) -> Wire into executor (Wire error categories into executor event reporting #4745) -> Store in lookout DB + UI (Store FailureInfo in lookout DB and expose via API #4755) -> e2e tests (Add e2e test cases for error categorization #4760)
• The errormatch package is in internal/common/ since it will be reused by the lookout ingester
• The categorizer has thorough doc.go explaining config format and validation
• Exit code 0 containers are skipped during classification (only failures are categorized)
• Rules within a category are OR'd; categories are evaluated independently

[greptile-apps]

Greptile Summary

This PR lays the foundational building blocks for error categorization in Armada: a FailureInfo proto message, a shared errormatch package of matching primitives, a configurable categorizer classifier, and ExtractFailureInfo() in pod_status.go. Nothing is wired into the event-reporting path yet; that comes in PR #4745.

The implementation is well-structured with thorough tests and good validation at construction time. Several issues flagged in earlier review rounds have been addressed (exit-code-0 filtering in failedContainers, empty-rules validation in NewClassifier, nil guard in MatchExitCode).

Findings:

• AppError in categorizer/doc.go — The package doc lists AppError as a valid OnConditions value, but it is not present in errormatch.KnownConditions. Any operator who follows this documentation and adds \"AppError\" to their config will get a runtime error from NewClassifier, blocking executor startup.
• containerName silently ignored for pod-level conditions — When a CategoryRule specifies both containerName and an onConditions of Evicted or DeadlineExceeded, the container filter has no effect because pod-level conditions are matched against pod.Status.Reason, not the container list. A user expecting to scope an eviction/deadline rule to a specific container will get unexpected false positives with no error or warning.

Confidence Score: 4/5

Safe to merge after fixing the AppError documentation error in categorizer/doc.go — it will cause executor startup failures for any operator who follows the documented config.

One P1 documentation bug (AppError listed as valid condition) would directly cause executor startup failures for users following the godoc. The rest of the implementation is solid: previously-flagged bugs around exit-code-0 filtering, empty-rules validation, and the nil guard are all fixed. One P2 behavioral gap (containerName silently ignored for pod-level conditions) is noted for awareness.

internal/executor/categorizer/doc.go — remove AppError from the valid conditions list. internal/executor/categorizer/classifier.go — consider validating or documenting the containerName + pod-level condition interaction.

Important Files Changed

Filename	Overview
internal/executor/categorizer/doc.go	Package documentation incorrectly lists AppError as a valid OnConditions value — it's not in KnownConditions and will be rejected at runtime by NewClassifier.
internal/executor/categorizer/classifier.go	Core classifier implementation; exit-code-0 filtering and empty-rules validation are correctly implemented. Minor behavioral gap: containerName filter is silently ignored when combined with pod-level conditions (Evicted, DeadlineExceeded).
internal/executor/categorizer/classifier_test.go	Thorough test coverage for all matchers, multi-category matches, container scoping, init containers, and validation errors.
internal/common/errormatch/match.go	Clean implementation of MatchExitCode (with nil guard and exit-code-0 exclusion) and MatchPattern; no issues found.
internal/common/errormatch/types.go	Defines ExitCodeMatcher, RegexMatcher, condition constants, and KnownConditions map; well-structured with correct set of pod-observable conditions.
internal/executor/util/pod_status.go	Adds ExtractFailureInfo() and replaces local reason-string constants with errormatch package constants; logic is correct.
pkg/armadaevents/events.proto	Adds FailureInfo message and failure_info field (tag 15) to Error; generated .pb.go is in sync with the proto source.
internal/executor/configuration/types.go	Adds ErrorCategories []categorizer.CategoryConfig field to ApplicationConfiguration with appropriate YAML tag and documentation.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Pod fails] --> B[ExtractFailureInfo pod_status.go]
    A --> C[Classifier.Classify]

    B --> D[Iterate GetPodContainerStatuses]
    D --> E{First container\nwith exitCode != 0?}
    E -->|Yes| F[Set ExitCode, TerminationMessage, ContainerName in FailureInfo]
    E -->|No| G[ExitCode stays 0]

    C --> H[failedContainers\nfilter exitCode == 0]
    H --> I[For each CategoryConfig]
    I --> J[categoryMatches: OR rules]
    J --> K{Rule matcher type?}
    K -->|onConditions OOMKilled| L[Check container.reason]
    K -->|onConditions Evicted/Deadline| M[Check pod.Status.Reason]
    K -->|onExitCodes| N[MatchExitCode]
    K -->|onTerminationMessage| O[MatchPattern regex]
    L & M & N & O --> P{Matched?}
    P -->|Yes| Q[Append category name]
    P -->|No| R[Next rule]

    F --> S[FailureInfo proto\nCategories + ExitCode + TerminationMessage + ContainerName]
    Q --> S
    G --> S

Loading

_{Reviews (25): Last reviewed commit: "Merge branch 'master' into error-categor..." | Re-trigger Greptile}

[dejanzele]

@greptile

[dejanzele]

@greptile

[dejanzele]

@greptile

[dejanzele]

Changed the default from FAILURE_CONDITION_USER_ERROR to FAILURE_CONDITION_UNSPECIFIED - at large scale various infra/hardware/k8s errors appear that aren't the user's fault, so the catch-all should be honest about not knowing. The USER_ERROR enum value stays in the proto for operators who want to explicitly classify via errorCategories config.

[dejanzele]

@greptileai