Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 15 additions & 31 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,8 @@

## Contribution process

```mermaid
flowchart LR
A[Pre-checks] --> B[PR Tests]
subgraph Code Review/Approval
C1[Expert Review] --> C2[Final Review]
end
B --> C1
C2 --> D[Merge]
```

### Pre-checks

- [ ] I want this PR in a versioned release and have added the appropriate Milestone (e.g., `Core 0.8`)
- [ ] I have added relevant unit tests
- [ ] I have added relevant functional tests
- [ ] I have added proper typing to my code [Typing guidelines](https://docs.python.org/3/library/typing.html)
Expand All @@ -26,41 +15,36 @@ flowchart LR

### Code review

The following process is enforced via the CODEOWNERS file for changes into `megatron/core`. For changes outside of `megatron/core`, it is up to the PR author whether or not to tag the Final Reviewer team.
Feel free to message or comment the [@mcore-oncall](https://github.com/orgs/NVIDIA/teams/mcore-oncall) to help accelerate your merge into main. The less complex your PR is, the faster it will be approved and merged!

<details>
<summary>For MRs into `main` branch</summary>
All PRs start as **draft**. If you open a non-draft PR, it will be automatically converted to draft.

Feel free to message or comment the @mcore-oncall to help accelerate your merge into main. The less complex your PR is, the faster it will be approved and merged!
#### Step 1: Mark PR as "Ready for Review"

#### (Step 1): Add PR label `Expert Review`
1. When your PR is ready, click **Ready for Review**.
2. An oncall reviewer is auto-assigned and expert reviewers are notified based on your changes.
- Some PRs may jump straight to step 2. This is determined by `.github/CODEOWNERS`.

#### (Step 2): Collect the expert reviewers reviews
:warning: Only mark as ready once merge-conflicts are resolved and the CI is passing.
Final Review might get declined if these requirements are not fulfilled.

1. Attach the `Expert Review` label when your PR is ready for review.
2. GitHub auto-assigns expert reviewers based on your changes. They will get notified and pick up your PR soon.
#### Step 2: Final Review

:warning: Only proceed to the next step once all reviewers have approved, merge-conflict are resolved and the CI is passing.
Final Review might get declined if these requirements are not fulfilled.
For PRs that change `megatron/core`, once all expert reviewers have approved, the `Final Review` label is applied **automatically** and final reviewers are assigned.

#### (Step 3): Final Review
For PRs outside `megatron/core`, this step is skipped.

1. Add `Final Review` label
2. GitHub auto-assigns final reviewers based on your changes. They will get notified and pick up your PR soon.
#### Step 3: Approved

#### (Optional Step 4): Cherry-pick into release branch
Once all required reviewers have approved, the `Approved` label is applied **automatically**.

If this PR also needs to be merged into `core_r*` release branches, after this PR has been merged, select `Cherry-pick` to open a new PR into the release branch.
### Merge

</details>
Any member of [mcore-engineers](https://github.com/orgs/NVIDIA/teams/mcore-engineers) will be able to merge your PR.

<details>
<summary>For MRs into `dev` branch</summary>
The proposed review process for `dev` branch is under active discussion.

MRs are mergable after one approval by either `eharper@nvidia.com` or `zijiey@nvidia.com`.
</details>

### Merging your PR

Any member of [core-adlr](https://github.com/orgs/teams/NVIDIA/core-adlr) and [`core-nemo`](https://github.com/orgs/teams/NVIDIA/core-nemo) will be able to merge your PR.
14 changes: 12 additions & 2 deletions .github/workflows/auto-swap-labels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ name: Auto Swap Labels
on:
pull_request_review:
types: [submitted]
pull_request_target:
types: [ready_for_review, synchronize]
branches:
- main

permissions:
pull-requests: write
Expand All @@ -12,7 +16,13 @@ permissions:
jobs:
check-approval:
runs-on: ubuntu-latest
if: github.event.review.state == 'approved' && github.repository == 'NVIDIA/Megatron-LM'
if: >-
github.repository == 'NVIDIA/Megatron-LM' &&
github.event.pull_request.base.ref == 'main' &&
(
(github.event_name == 'pull_request_review' && github.event.review.state == 'approved') ||
(github.event_name == 'pull_request_target' && !github.event.pull_request.draft)
)
steps:
- name: Check out repository code
uses: actions/checkout@v4
Expand All @@ -26,7 +36,7 @@ jobs:
run: |
pip install --no-cache-dir PyGithub slack-sdk

- name: Run Auto Reminder Bot
- name: Run Auto Swap Labels
run: |
export GH_TOKEN=${{ github.token }}
export PR_NUMBER=${{ github.event.pull_request.number }}
Expand Down
36 changes: 36 additions & 0 deletions .github/workflows/force-draft-pr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.

name: Force Draft PR

on:
pull_request_target:
types: [opened]
branches:
- main

permissions:
pull-requests: write

jobs:
force-draft:
runs-on: ubuntu-latest
if: ${{ !github.event.pull_request.draft && github.repository == 'NVIDIA/Megatron-LM' }}
steps:
- name: Convert PR to draft
env:
GH_TOKEN: ${{ secrets.PAT }}
run: |
gh pr ready --undo ${{ github.event.pull_request.number }} --repo ${{ github.repository }}

- name: Add comment explaining draft policy
env:
GH_TOKEN: ${{ github.token }}
Comment on lines +21 to +27

This comment was marked as resolved.

run: |
gh pr comment ${{ github.event.pull_request.number }} --repo ${{ github.repository }} --body \
"This PR has been automatically converted to **draft** because all PRs must start as drafts.

When you are ready for review, click **Ready for Review** to begin the review process. This will:
1. Add the oncall reviewer (optional reviewer)
2. Add required review teams based on your changes

See the [contribution guide](https://github.com/NVIDIA/Megatron-LM/blob/main/docs/developer/submit.md) for more details."
2 changes: 1 addition & 1 deletion .github/workflows/oncall-assign.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ name: Oncall Assign

on:
pull_request_target:
types: [opened, ready_for_review]
types: [ready_for_review]
branches:
- main

Expand Down
10 changes: 5 additions & 5 deletions docs/developer/oncall.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,13 @@ Below is the checklist that the oncall needs to go through for each PR.
- Do all tests pass?
- Oncall will need to kick off testing suite for external reviewers
- Comment “/ok to test commid_id” to kick off testing suite
- Add the “Expert Review” label
- Select an expert reviewer from each expert group as a reviewer. If you’re unsure who to select, pick a “maintainer” or manager.
- Expert reviewers are notified after the PR is marked “Ready for Review”
- **Expert reviewers should review within 1 business day.** Message the assigned reviewer if it is taking longer. The reviewer either needs to review the PR or suggest an alternate reviewer.
- If the reviewer is not responding after 2 business days, escalate to the reviewer's manager.
- Add the “Final Review” label after experts approve
- If the reviewer is not responding after 2 business days, escalate to the reviewers manager.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who is this note for? Maybe it’s obsolete nowadays?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I.e. how do we do that if we cannot reach out to everyone? Maybe via MCore-oncall?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the idea, the oncall is supposed to reach out directly to reviewer.

- For `megatron/core` PRs, the “Final Review” label is applied automatically once all expert reviewers approve
- Final reviewers should review within 1 business day. Message the assigned reviewer if it is taking longer.
- If the reviewer is not responding after 2 business days, escalate to the reviewer's manager.
- If the reviewer is not responding after 2 business days, escalate to the reviewer’s manager.
- The “Approved” label is applied automatically once all required reviewers have approved

## Issues and Discussion Questions

Expand Down
25 changes: 17 additions & 8 deletions docs/developer/submit.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,26 @@

# How to Submit a PR

## Step 1: Add PR label `Expert Review`
All PRs start as **draft**. If you open a non-draft PR, it will be automatically converted to draft.

## Step 2: Collect the expert reviewers reviews
## Step 1: Mark PR as "Ready for Review"

1. Attach the `Expert Review` label when your PR is ready for review.
2. GitHub auto-assigns expert reviewers based on your changes. They will get notified and pick up your PR soon.
1. When your PR is ready, click **Ready for Review**.
2. The oncall reviewer is auto-assigned and expert reviewers are notified based on your changes. They will get notified and pick up your PR soon.

:warning: Only proceed to the next step once all reviewers have approved, merge-conflict are resolved and the CI is passing.
:warning: Only mark as ready once all merge-conflicts are resolved and the CI is passing.
Final Review might get declined if these requirements are not fulfilled.

## Step 3: Final Review
## Step 2: Final Review (`megatron/core` only)

1. Add `Final Review` label
2. GitHub auto-assigns final reviewers based on your changes. They will get notified and pick up your PR soon.
For PRs that change `megatron/core`, once all expert reviewers have approved, the `Final Review` label is applied **automatically** and final reviewers are assigned.

For PRs outside `megatron/core`, this step is skipped.

## Step 3: Approved

Once all required reviewers have approved, the `Approved` label is applied **automatically**. The PR is now ready to merge.

## Step 4: Merge

Any member of [mcore-engineers](https://github.com/orgs/NVIDIA/teams/mcore-engineers) will be able to merge your PR.
49 changes: 41 additions & 8 deletions tests/test_utils/python_scripts/auto_reminder_github.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ class Reminder:
class PRReviewTracker:
EXPERT_REVIEW = "Expert Review"
FINAL_REVIEW = "Final Review"
APPROVED = "Approved"
EXCLUDED_TEAMS = {"core-adlr", "core-nemo"}

def __init__(
Expand Down Expand Up @@ -127,6 +128,11 @@ def get_label_date(self, pr, label: str):
]
return max(dates) if dates else None

def get_ready_for_review_date(self, pr):
"""Get the date a PR was marked as ready for review."""
dates = [e.created_at for e in pr.as_issue().get_events() if e.event == "ready_for_review"]
return max(dates) if dates else None

def days_since(self, date):
"""Calculate days since given date."""
if not date:
Expand All @@ -138,11 +144,22 @@ def days_since(self, date):
def get_stage(self, pr):
"""Get current review stage."""
labels = {l.name for l in pr.labels}
return self.FINAL_REVIEW if self.FINAL_REVIEW in labels else self.EXPERT_REVIEW
if self.APPROVED in labels:
return self.APPROVED
if self.FINAL_REVIEW in labels:
return self.FINAL_REVIEW
return self.EXPERT_REVIEW

def get_reviewers(self, pr):
"""Get filtered reviewer emails who haven't approved yet."""
stage = self.get_stage(pr)

if stage == self.APPROVED:
return (
[self.get_user_email(pr.user.login)],
"All reviewers have approved. Please merge the PR.",
)

org = self.github.get_organization(self.repo.organization.login)

# 1. Get the latest review state for everyone who has submitted a review
Expand Down Expand Up @@ -207,17 +224,27 @@ def get_reviewers(self, pr):
# 8. Handle the original edge cases
if len(reviewer_emails) == 0:
if stage == self.EXPERT_REVIEW:
# No reviewer activity yet — assignment hasn't completed (e.g. PR just became
# ready-for-review). Don't fire a spurious "all approved" message.
has_reviewer_activity = bool(
approvers
or non_approving_reviewers
or pending_individuals
or pending_teams_slugs
)
if not has_reviewer_activity:
return [], "Waiting for reviewers to be assigned."
# Assign to PR author
reviewer_emails = [self.get_user_email(pr.user.login)]
action_message = "All Expert Reviewers approved the PR. Please attach the Final Review label to proceed with the review."
action_message = "All Expert Reviewers have approved the PR."
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This message fires whenever pending_reviewers is empty in the EXPERT_REVIEW stage. With the query now including all non-draft milestone PRs (not just labeled ones), this will also trigger for a PR that just became ready-for-review before CODEOWNERS/oncall assignment completes — resulting in a spurious "All Expert Reviewers have approved" message to the author.

swap_pr_labels.py already handles this race condition with a has_any_reviewers guard (step 8). A similar guard is needed here, e.g.:

if not (pending_individuals or pending_teams_slugs or approvers or non_approving_reviewers):
    logger.info(f"PR #{pr.number} has no reviewers yet. Skipping.")
    return [], ""

elif stage == self.FINAL_REVIEW:
# Assign to mcore-reviewers who approved
try:
mcore_team = org.get_team_by_slug("mcore-reviewers")
mcore_members = {m.login for m in mcore_team.get_members()}
valid_approvers = approvers & mcore_members
reviewer_emails = sorted([self.get_user_email(u) for u in valid_approvers])
action_message = "All Final Reviewers approved the PR. Please ping an Expert or Final Reviewer to merge the PR."
action_message = "All Final Reviewers approved the PR. Please ping the @mcore-oncall to merge the PR."

except Exception as e:
logger.warning(
Expand All @@ -229,7 +256,14 @@ def get_reviewers(self, pr):
def create_reminder(self, pr):
"""Create reminder for PR."""
stage = self.get_stage(pr)
stage_days = self.days_since(self.get_label_date(pr, stage))
ready_date = self.get_ready_for_review_date(pr)
if stage == self.EXPERT_REVIEW:
stage_days = self.days_since(ready_date)
elif stage in (self.FINAL_REVIEW, self.APPROVED):
stage_days = self.days_since(self.get_label_date(pr, stage))
else:
stage_days = 0
total_review_days = self.days_since(ready_date)
author_email = self.get_user_email(pr.user.login)
reviewer_emails, action_message = self.get_reviewers(pr)
escaped_title = html.escape(pr.title, quote=False)
Expand All @@ -241,7 +275,7 @@ def create_reminder(self, pr):
author=self.get_slack_user_id(author_email),
priority="P0" if stage_days > 3 else "P1" if stage_days >= 1 else "P2",
review_stage=stage,
total_review_time=self.days_since(self.get_label_date(pr, self.EXPERT_REVIEW)),
total_review_time=total_review_days,
current_stage_time=stage_days,
reviewers=[self.get_slack_user_id(email) for email in reviewer_emails],
action_message=action_message,
Expand All @@ -256,12 +290,11 @@ def generate_reminders(self):

reminders = []
for milestone in milestones:
# Find issues with the 'Expert Review' or 'Final Review' label
# Find all open non-draft PRs with this milestone
query = (
f'repo:"{self.repo.full_name}" '
f'milestone:"{milestone.title}" '
f'is:open is:pr -is:draft '
f'label:"{self.EXPERT_REVIEW}","{self.FINAL_REVIEW}"'
f'is:open is:pr -is:draft'
)
try:
# Use search_issues for a more direct query instead of get_issues + filtering
Expand Down
Loading
Loading