[FEAT] New control: securityJobsMustNotBeWeakened: detect allow_failure, rules bypass, and when:manual on security jobs

[Joseph94m]

Is your feature request related to a problem? Please describe.

GitLab's security scanning templates (SAST, Secret Detection, Container Scanning, Dependency Scanning, DAST) are designed to run automatically in pipelines. However, because GitLab CI allows overriding any property of an included job, a developer (or attacker) can silently weaken or completely disable security checks in the .gitlab-ci.yml without removing the include: line. The pipeline still "includes" the security template, giving a false sense of compliance, but the actual scanning is neutralized.

There are multiple ways to achieve this, and they should all be caught by a single umbrella control:

Sub-control 1: allow_failure on security jobs

GitLab's own templates already ship with allow_failure: true on analyzer jobs (see .sast-analyzer, .secret-analyzer, .ds-analyzer, container_scanning). This is GitLab's design choice, they don't want scanners to block pipelines by default. But for organizations that need security checks to be blocking, someone can explicitly keep or re-add allow_failure: true on an overridden job, making failures invisible.

# Security scanner silently becomes non-blocking
include:
  - template: Security/SAST.gitlab-ci.yml

semgrep-sast:
  allow_failure: true  # Scanner fails? Pipeline still green.

Sub-control 2: rules: override that disables or makes jobs manual

By overriding the rules: block of a security job, a developer can prevent the job from running at all, or make it manual (meaning it never runs unless someone clicks a button):

include:
  - template: Security/SAST.gitlab-ci.yml

semgrep-sast:
  rules:
    - when: never  # Scanner never runs

# Or even sneakier:
secret_detection:
  rules:
    - when: manual  # Scanner only runs if someone manually triggers it
      allow_failure: true

Sub-control 3: when: manual at job level

Similar to sub-control 2, but set at job level instead of inside rules::

include:
  - template: Security/SAST.gitlab-ci.yml

semgrep-sast:
  when: manual  # Will only run if manually triggered effectively disabled

---

Note on script: overrides: Overriding the script: block of an included security job (e.g., replacing /analyzer run with echo "skip") is already covered by existing controls (pipelineMustNotIncludeHardcodedJobs and the upcoming #50 Detect Overridden Included templates/components). This control intentionally does not duplicate that detection.

All three patterns above share the same outcome: the pipeline looks compliant (the security templates are included) but the actual scanning is weakened or disabled. This maps to OWASP CICD-SEC-4 (Poisoned Pipeline Execution).

Describe the solution you'd like

Add a single umbrella control securityJobsMustNotBeWeakened with configurable sub-controls. The control inspects jobs that originate from included security templates and flags any override that weakens their intended behavior.

Configuration in .plumber.yaml

controls:
  securityJobsMustNotBeWeakened:
    enabled: true

    # List of job name patterns considered "security jobs"
    # Plumber auto-detects jobs from security templates, but this
    # list lets you add custom security jobs too
    securityJobPatterns:
      - "*-sast"
      - "secret_detection"
      - "container_scanning"
      - "*_dependency_scanning"
      - "gemnasium-*"
      - "dast"
      - "dast_*"
      - "license_scanning"

    # Sub-controls; each can be individually disabled if needed
    subControls:
      # Flag security jobs with allow_failure: true
      # (Note: GitLab templates ship with this by default,
      # set this to detect when it's NOT been hardened to false)
      allowFailureMustBeFalse:
        enabled: false  # Off by default;  opt-in for orgs wanting blocking security

      # Flag security jobs whose rules: are overridden
      # to include when:never or when:manual
      rulesMustNotBeRedefined:
        enabled: true

      # Flag security jobs with when:manual at job level
      whenMustNotBeManual:
        enabled: true

Expected output

securityJobsMustNotBeWeakened:
  ✗ semgrep-sast: allow_failure is true (should be false for blocking security)
  ✗ secret_detection: rules overridden with 'when: never' job will not run
  ✗ container_scanning: when set to 'manual' job requires manual trigger
  ✓ gemnasium-dependency_scanning: no weakening detected

Implementation Hints

These are just ideas. Feel free to change the implementation.

1. Data source: The PipelineOriginData collector already provides the merged CI config (MergedConf) and the job map (JobMap). Crucially, it also tracks which jobs come from includes vs. hardcoded this is the key to detecting overrides. The unmerged Conf (raw .gitlab-ci.yml) shows what the user explicitly wrote, while MergedConf shows the final result after template merging. Comparing the two reveals overrides.
2. Detecting security jobs: Two complementary approaches:
   • By origin: Jobs whose OriginType is template and whose include Location matches known security template paths (Jobs/SAST.gitlab-ci.yml, Security/Secret-Detection.gitlab-ci.yml, Security/Container-Scanning.gitlab-ci.yml, Jobs/Dependency-Scanning.gitlab-ci.yml, DAST.gitlab-ci.yml)
   • By name pattern: Match against securityJobPatterns in config for custom security jobs
3. New control file: Create control/controlGitlabSecurityJobsWeakened.go.
4. Logic for each sub-control:
   • allowFailureMustBeFalse: Parse the GitlabJob for matching jobs in the merged config. Check if AllowFailure is true. Note: since GitLab templates ship with allow_failure: true by default, this sub-control flags the absence of hardening (i.e., the user didn't override it to false). This is why it defaults to enabled: falseonly orgs that actively want blocking security opt in.
   • rulesMustNotBeRedefined: Check if the user's raw .gitlab-ci.yml (unmerged Conf) redefines the rules: key on any security job. If the overridden rules contain when: never or when: manual, flag it.
   • whenMustNotBeManual: Check if the job-level When field is set to manual in the merged config for any security job.
5. Compliance: Per sub-control, 0% if any security job is weakened, 100% if none are. Overall compliance is the minimum across enabled sub-controls.

Files Touched

• control/controlGitlabSecurityJobsWeakened.go (new control with sub-control logic)
• control/types.go (add SecurityJobsWeakenedResult field to AnalysisResult, including per-sub-control results)
• control/task.go (wire the new control in RunAnalysis())
• configuration/plumberconfig.go (add config struct with sub-controls)
• .plumber.yaml (add default config section)
• cmd/analyze.go (add output formatting)

Why It's Valuable

This control catches the gap between "including a security template" and "actually enforcing security scanning." Today, Plumber can verify that required templates are included (pipelineMustIncludeRequiredTemplates), and it can detect when included jobs are overridden at the script level (#50). But neither catches the subtler weakening patterns allow_failure: true, when: manual, or rules: [{when: never}] which leave the job technically present but functionally useless.

The problem is real and widespread: GitLab's own issue tracker has a long-standing discussion about SAST jobs shipping with allow_failure: true, and the override mechanism is by design. The umbrella approach with sub-controls gives organizations the granularity to enforce what matters to them; some may only care about when: never (job completely disabled), while security-mature orgs want full hardening including blocking failures.