Skip to content

Change Review Process#3659

Queued
Phlip79 wants to merge 11 commits intoNVIDIA:mainfrom
Phlip79:philip/change-review-process
Queued

Change Review Process#3659
Phlip79 wants to merge 11 commits intoNVIDIA:mainfrom
Phlip79:philip/change-review-process

Conversation

@Phlip79
Copy link
Member

@Phlip79 Phlip79 commented Mar 2, 2026

What does this PR do ?

Changing the review process to:

  • force all new PRs be drafts
  • no longer using "Expert Review" label
  • PRs that have all necessary approvals have label "Approved" automatically added
  • automated the review process steps
  • updated all documentation to reflect this new process

Before this PR is merged in I will change all open PRs that do not have the "Expert Review" or "Final Review" process to draft PRs.

Contribution process

flowchart LR
    A[Pre-checks] --> B[PR Tests]
    subgraph Code Review/Approval
        C1[Expert Review] --> C2[Final Review]
    end
    B --> C1
    C2 --> D[Merge]
Loading

Pre-checks

  • I want this PR in a versioned release and have added the appropriate Milestone (e.g., Core 0.8)
  • I have added relevant unit tests
  • I have added relevant functional tests
  • I have added proper typing to my code Typing guidelines
  • I have added relevant documentation
  • I have run the autoformatter.sh on my PR

Code review

The following process is enforced via the CODEOWNERS file for changes into megatron/core. For changes outside of megatron/core, it is up to the PR author whether or not to tag the Final Reviewer team.

For MRs into `main` branch

Feel free to message or comment the @mcore-oncall to help accelerate your merge into main. The less complex your PR is, the faster it will be approved and merged!

(Step 1): Add PR label Expert Review

(Step 2): Collect the expert reviewers reviews

  1. Attach the Expert Review label when your PR is ready for review.
  2. GitHub auto-assigns expert reviewers based on your changes. They will get notified and pick up your PR soon.

⚠️ Only proceed to the next step once all reviewers have approved, merge-conflict are resolved and the CI is passing.
Final Review might get declined if these requirements are not fulfilled.

(Step 3): Final Review

  1. Add Final Review label
  2. GitHub auto-assigns final reviewers based on your changes. They will get notified and pick up your PR soon.

(Optional Step 4): Cherry-pick into release branch

If this PR also needs to be merged into core_r* release branches, after this PR has been merged, select Cherry-pick to open a new PR into the release branch.

For MRs into `dev` branch The proposed review process for `dev` branch is under active discussion.

MRs are mergable after one approval by either eharper@nvidia.com or zijiey@nvidia.com.

Merging your PR

Any member of core-adlr and core-nemo will be able to merge your PR.

@Phlip79 Phlip79 requested a review from a team as a code owner March 2, 2026 23:56
@copy-pr-bot
Copy link

copy-pr-bot bot commented Mar 2, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@svcnvidia-nemo-ci svcnvidia-nemo-ci requested a review from a team March 2, 2026 23:56
@Phlip79 Phlip79 requested a review from ericharper March 2, 2026 23:59
- **Expert reviewers should review within 1 business day.** Message the assigned reviewer if it is taking longer. The reviewer either needs to review the PR or suggest an alternate reviewer.
- If the reviewer is not responding after 2 business days, escalate to the reviewer's manager.
- Add the “Final Review” label after experts approve
- If the reviewer is not responding after 2 business days, escalate to the reviewers manager.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who is this note for? Maybe it’s obsolete nowadays?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I.e. how do we do that if we cannot reach out to everyone? Maybe via MCore-oncall?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the idea, the oncall is supposed to reach out directly to reviewer.

@Phlip79
Copy link
Member Author

Phlip79 commented Mar 3, 2026

/claude review

@Phlip79
Copy link
Member Author

Phlip79 commented Mar 4, 2026

/claude review

Comment on lines +21 to +27
GH_TOKEN: ${{ secrets.PAT }}
run: |
gh pr ready --undo ${{ github.event.pull_request.number }} --repo ${{ github.repository }}

- name: Add comment explaining draft policy
env:
GH_TOKEN: ${{ github.token }}

This comment was marked as resolved.

@Phlip79
Copy link
Member Author

Phlip79 commented Mar 4, 2026

/claude review

@Phlip79
Copy link
Member Author

Phlip79 commented Mar 4, 2026

/ok to test f5b8205

@svcnvidia-nemo-ci svcnvidia-nemo-ci added this to the Core 0.16 milestone Mar 4, 2026
@Phlip79
Copy link
Member Author

Phlip79 commented Mar 4, 2026

/claude review

@Phlip79
Copy link
Member Author

Phlip79 commented Mar 4, 2026

/claude review

@claude
Copy link

claude bot commented Mar 4, 2026

Code Review

Bug (inline comment posted): TypeError in auto_reminder_github.py

See inline comment on line 253. days_since(ready_date) returns None for pre-existing non-draft PRs that lack a ready_for_review event, and the subsequent "stage_days > 3" comparison raises TypeError in Python 3. The new label-agnostic query (is:open is:pr -is:draft) returns PRs that the old label-filtered query never would, exposing this path during migration.


Potential gap: milestone checkbox removed from PR template

The "I want this PR in a versioned release and have added the appropriate Milestone" checklist item was removed from pull_request_template.md. The auto-reminder bot queries PRs by milestone -- PRs without a milestone are silently skipped. If milestone assignment is now handled another way that is fine, but worth confirming PRs cannot fall through reminder coverage.


Missing tests for refactored swap_pr_labels.py

The label-swap logic was significantly rewritten into a 3-stage state machine (_handle_expert_review_stage, _handle_final_review_stage, _handle_approved_stage). No tests were added or updated. Given the edge cases (revert paths, race-condition guard, needs_final_review logic), unit tests covering the main state transitions would help catch regressions.

@Phlip79
Copy link
Member Author

Phlip79 commented Mar 4, 2026

/claude review

# Assign to PR author
reviewer_emails = [self.get_user_email(pr.user.login)]
action_message = "All Expert Reviewers approved the PR. Please attach the Final Review label to proceed with the review."
action_message = "All Expert Reviewers have approved the PR."
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This message fires whenever pending_reviewers is empty in the EXPERT_REVIEW stage. With the query now including all non-draft milestone PRs (not just labeled ones), this will also trigger for a PR that just became ready-for-review before CODEOWNERS/oncall assignment completes — resulting in a spurious "All Expert Reviewers have approved" message to the author.

swap_pr_labels.py already handles this race condition with a has_any_reviewers guard (step 8). A similar guard is needed here, e.g.:

if not (pending_individuals or pending_teams_slugs or approvers or non_approving_reviewers):
    logger.info(f"PR #{pr.number} has no reviewers yet. Skipping.")
    return [], ""

This was referenced Mar 4, 2026
@Phlip79 Phlip79 enabled auto-merge March 5, 2026 00:20
@Phlip79 Phlip79 added this pull request to the merge queue Mar 5, 2026
@svcnvidia-nemo-ci
Copy link

🔄 Merge queue validation started!

You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/22700334343

@svcnvidia-nemo-ci
Copy link

🔄 Merge queue validation started!

You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/22703252875

@svcnvidia-nemo-ci
Copy link

🔄 Merge queue validation started!

You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/22705408467

@svcnvidia-nemo-ci
Copy link

🔄 Merge queue validation started!

You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/22709214191

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Approved All necessary approvals have been made

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants