Skip to content

Add LLM-based SemVer validation for breaking changes detection #508

@cursor

Description

@cursor

Problem

Currently there's no automated way to verify whether a plugin's version increment matches SemVer guidelines based on the actual changes introduced. This is particularly important for managed plugin updates where breaking changes need to be properly communicated.

Proposed Solution

Leverage the existing LLM checks infrastructure to detect:

  • Whether a change introduces breaking changes
  • Whether the version increment matches SemVer conventions (major for breaking, minor for features, patch for fixes)
  • Flag mismatches between detected breaking changes and version bump

Use Cases

  1. Managed plugin updates: Ensure version increments accurately reflect the nature of changes
  2. Breaking change detection: Automatically identify API changes, removed features, or behavioral changes that could break existing users
  3. Review assistance: Help reviewers quickly identify potential SemVer violations

Implementation Notes

  • Could build on the existing CodeDiff validator which already analyzes changes between versions
  • New LLM models may be capable of accurately detecting breaking changes from diffs
  • Should provide clear feedback on what breaking changes were detected

Context

This would help ensure plugins follow proper versioning practices, making updates safer for Grafana users relying on these plugins.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    📬 Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions