Skip to content

Update CI to label-driven testing#450

Open
Micky774 wants to merge 10 commits intodevfrom
zain/ci
Open

Update CI to label-driven testing#450
Micky774 wants to merge 10 commits intodevfrom
zain/ci

Conversation

@Micky774
Copy link
Contributor

@Micky774 Micky774 commented Feb 17, 2026

Description

With this PR, testing is mediated by labels such as: ci-level {1, 2, 3}. No tests will be run without at least one label present. At any given time, the highest-level label will take precedence, and only its corresponding level of tests will be run. The workflow triggers on labeling, unlabeling, and pushing commits. When removing a label, the workflow will check to see if there are any labels remaining and if there are, it will dispatch to the highest of them.

Fixes # (issue)

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refactoring

Changes

Please list the changes introduced in this PR:

  • Change A
  • Change B

Checklist:

  • I have read and followed the contributing guidelines
  • The functionality is complete
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Copy link
Collaborator

@ipanfilo ipanfilo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe make approach more generic - ci_level% label specifies CI level to use.

@Micky774 Micky774 changed the title Add label-specific workflow for level 3 testing Update CI to label-driven testing Feb 18, 2026
const toCancel = runs.data.workflow_runs.filter(run => {
const prMatch = (run.pull_requests || []).some(pr => pr.number === prNumber);
const active = run.status === 'queued' || run.status === 'in_progress';
if (action === 'synchronize') {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reopened action does not seem handled in this code

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now handled as an early-skip

// If a label was added, cancel runs with a lower level than the added label
if (action === 'labeled') {
const isLowerThanAdded = runLevel > 0 && runLevel < addedLevel;
return prMatch && active && isLowerThanAdded && run.id !== currentRunId;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can is LowerThanAdded be if run.id === currentRunId?

.map(label => label.name.toLowerCase());

// Determine if a CI level label was removed and what level it was
const removedLabel = (context.payload.label?.name || '').toLowerCase();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it removedLabel? For labeled event it is addedLabel.
Maybe unlabeled events are not needed at all? If higher lablel is added - CI with new level is triggered. Otherwise the last CI run should still be relevant.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a PR has labels ci-level 1 and ci-level 3 but then ci-level 3 is removed, then we still need to run ci-level 1 tests.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a PR has labels ci-level 1 and ci-level 3 but then ci-level 3 is removed, then we still need to run ci-level 1 tests.

It ran Level 3 before, why does it need run Level 1 now? The next time something is changed, yes, but why now?

Copy link
Contributor Author

@Micky774 Micky774 Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Originally it was in case the highest-level test wasn't finished, in which case the commit still wouldn't necessarily have a passed test on the next-highest level yet. I've adjusted the script to explicitly check for tests of at least the new level, and skip dispatch if it exists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants