Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 2 additions & 60 deletions .asf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,72 +54,14 @@ github:
# needs to be updated as part of the release process
# .asf.yaml doesn't support wildcard branch protection rules, only exact branch names
# https://github.com/apache/infrastructure-asfyaml?tab=readme-ov-file#branch-protection
# Keeping set of protected branches for future releases
# Meanwhile creating a prerelease script that will update the branch protection names
# automatically. Keep track on it https://github.com/apache/datafusion/issues/17134
# these branches protection blocks autogenerated during release process which is described in
# https://github.com/apache/datafusion/tree/main/dev/release#2-add-a-protection-to-release-candidate-branch
branch-50:
required_pull_request_reviews:
required_approving_review_count: 1
branch-51:
required_pull_request_reviews:
required_approving_review_count: 1
# branch-52:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-53:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-54:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-55:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-56:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-57:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-58:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-59:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-60:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-61:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-62:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-63:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-64:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-65:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-66:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-67:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-68:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-69:
# required_pull_request_reviews:
# required_approving_review_count: 1
# branch-70:
# required_pull_request_reviews:
# required_approving_review_count: 1
pull_requests:
# enable updating head branches of pull requests
allow_update_branch: true
Expand Down
48 changes: 28 additions & 20 deletions dev/release/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,11 @@ Patch releases are made on an adhoc basis, but we try and avoid them given the f
## Release Process Overview

New development happens on the `main` branch.
Releases are made from branches, e.g. `branch-37` for the `37.x.y` release series.
Releases are made from branches, e.g. `branch-50` for the `50.x.y` release series.

To prepare for a new release series, we:

- Create a new branch from `main`, such as `branch-37` in the Apache repository (not in a fork)
- Create a new branch from `main`, such as `branch-50` in the Apache repository (not in a fork)
- Continue merging new features changes to `main` branch
- Prepare the release branch for release:
- Update version numbers in `Cargo.toml` files and create `CHANGELOG.md`
Expand All @@ -55,7 +55,7 @@ these steps:

1. Find (or create) the issue for the incremental release ([example release issue]) and discuss the proposed change there with the maintainers.
2. Follow normal workflow to create PR to `main` branch and wait for its approval and merge.
3. After PR is squash merged to `main`, branch from most recent release branch (e.g. `branch-37`), cherry-pick the commit and create a PR targeting the release branch [example backport PR].
3. After PR is squash merged to `main`, branch from most recent release branch (e.g. `branch-50`), cherry-pick the commit and create a PR targeting the release branch [example backport PR].

For example, to backport commit `12345` from `main` to `branch-50`:

Expand Down Expand Up @@ -127,16 +127,35 @@ We then publish the code in the approved artifacts to crates.io.

First create a new release branch from `main` in the apache repository.

For example, to create the `branch-51` branch for the `51.x.y` release series:
For example, to create the `branch-50` branch for the `50.x.y` release series:

```shell
git fetch apache # make sure we are up to date
git checkout apache/main # checkout current latest development branch
git checkout -b branch-51 # create local branch
git push -u apache branch-51 # push branch to apache remote
git checkout -b branch-50 # create local branch
git push -u apache branch-50 # push branch to apache remote
```

### 2. Prepare PR to Update Changelog and the Release Version
### 2. Add a protection to release candidate branch

To protect a release candidate branch from accidental merges, run:

```shell
./dev/release/add-branch-protection.sh 50
```

The script will modify `.asf.yaml` and add following block:

```yaml
branch-50:
required_pull_request_reviews:
required_approving_review_count: 1
```

- Create a PR.
- Merge to `main`.

### 3. Prepare PR to Update Changelog and the Release Version

First, prepare a PR to update the changelog and versions to reflect the planned
release. See [#18173](https://github.com/apache/datafusion/pull/18173) for an example.
Expand Down Expand Up @@ -193,17 +212,6 @@ git commit -a -m 'Update version'

Remember to merge any fixes back to `main` branch as well.

### 3. Prepare a PR to Modify `asf.yaml`

Modify `asf.yaml` to protect future release candidate branch to prevent accidental merges:

```yaml
# needs to be updated as part of the release process
branch-50:
required_pull_request_reviews:
required_approving_review_count: 1
```

### 4. Prepare Release Candidate Artifacts

After the PR gets merged, you are ready to create release artifacts based off the
Expand Down Expand Up @@ -416,7 +424,7 @@ svn ls https://dist.apache.org/repos/dist/dev/datafusion
Delete a release candidate:

```shell
svn delete -m "delete old DataFusion RC" https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-38.0.0-rc1/
svn delete -m "delete old DataFusion RC" https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-50.0.0-rc1/
```

#### Deleting old releases from `release` svn
Expand All @@ -432,5 +440,5 @@ svn ls https://dist.apache.org/repos/dist/release/datafusion
Delete a release:

```shell
svn delete -m "delete old DataFusion release" https://dist.apache.org/repos/dist/release/datafusion/datafusion-37.0.0
svn delete -m "delete old DataFusion release" https://dist.apache.org/repos/dist/release/datafusion/datafusion-50.0.0
```
160 changes: 160 additions & 0 deletions dev/release/add-branch-protection.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
#!/usr/bin/env bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

set -e

# Script to add branch protection for a new release branch in .asf.yaml
#
# This script automates the process of adding branch protection rules to .asf.yaml
# for new release branches. It ensures the branch protection block doesn't already
# exist before adding it.
#
# Usage:
# ./dev/release/add-branch-protection.sh <release_number>
#
# Examples:
# ./dev/release/add-branch-protection.sh 52
# ./dev/release/add-branch-protection.sh 53
#
# The script will:
# 1. Validate the release number is a positive integer
# 2. Check if branch protection already exists for branch-<release_number>
# 3. Add the branch protection block to .asf.yaml if it doesn't exist
# 4. Error out if the block already exists

# Check if release number is provided
if [ $# -eq 0 ]; then
echo "Error: Release number is required"
echo "Usage: $0 <release_number>"
echo "Example: $0 52"
exit 1
fi

RELEASE_NUM=$1
BRANCH_NAME="branch-${RELEASE_NUM}"
ASF_YAML_FILE=".asf.yaml"

# Validate release number is a positive integer
if ! [[ "$RELEASE_NUM" =~ ^[0-9]+$ ]]; then
echo "Error: Release number must be a positive integer"
echo "Provided: $RELEASE_NUM"
echo "Example: ./dev/release/add-branch-protection.sh 52"
exit 1
fi

# Check if .asf.yaml exists
if [ ! -f "$ASF_YAML_FILE" ]; then
echo "Error: $ASF_YAML_FILE not found in current directory"
echo "Please run this script from the repository root"
exit 1
fi

# Check if the branch exists in the official Apache DataFusion repository
GITHUB_API_URL="https://api.github.com/repos/apache/datafusion/branches/${BRANCH_NAME}"
HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" "$GITHUB_API_URL")

if [ "$HTTP_STATUS" != "200" ]; then
echo "Error: Branch ${BRANCH_NAME} does not exist in the official Apache DataFusion repository"
echo "Please create the branch '${BRANCH_NAME}' first before adding branch protection"
echo ""
echo "To check existing branches, visit:"
echo " https://github.com/apache/datafusion/branches"
exit 1
fi

# Check if branch protection already exists for this release
if grep -q "^[[:space:]]*${BRANCH_NAME}:" "$ASF_YAML_FILE"; then
echo "Error: Branch protection for ${BRANCH_NAME} already exists in $ASF_YAML_FILE"
exit 1
fi

# Create a temporary file
TEMP_FILE=$(mktemp)

# Read the file and insert the new branch protection block
# We'll insert it after the last branch-XX block
awk -v branch="$BRANCH_NAME" '
/^[[:space:]]*branch-[0-9]+:/ {
last_branch_line = NR
last_branch_content = $0
}
{
lines[NR] = $0
}
END {
if (last_branch_line == 0) {
print "Error: No existing branch protection blocks found" > "/dev/stderr"
exit 1
}

# Print all lines up to and including the last branch block
for (i = 1; i <= last_branch_line; i++) {
print lines[i]
}

# Print the required_pull_request_reviews lines after the last branch
for (i = last_branch_line + 1; i <= NR; i++) {
print lines[i]
# After printing the required_approving_review_count line, insert new branch
if (lines[i] ~ /required_approving_review_count:/) {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This insertion relies on encountering required_approving_review_count: and peeking only one line ahead; if additional keys are added under the last branch block after this line, the insertion never triggers and the script exits with “Failed to add branch protection block” (also applies to future YAML layout changes).

🤖 Was this useful? React with 👍 or 👎

# Check if this belongs to the last branch block by looking ahead
next_non_empty = i + 1
while (next_non_empty <= NR && lines[next_non_empty] ~ /^[[:space:]]*$/) {
next_non_empty++
}
# If next non-empty line is not indented more than branch level, we found the end
if (next_non_empty > NR || lines[next_non_empty] !~ /^[[:space:]]{6,}/) {
print " " branch ":"
print " required_pull_request_reviews:"
print " required_approving_review_count: 1"
# Skip to next iteration to avoid double printing
for (j = i + 1; j <= NR; j++) {
i = j
if (j <= NR) print lines[j]
}
break
}
}
}
}
' "$ASF_YAML_FILE" > "$TEMP_FILE"

# Check if awk succeeded
if [ $? -ne 0 ]; then
rm -f "$TEMP_FILE"
exit 1
fi

# Verify the new content was added
if ! grep -q "^[[:space:]]*${BRANCH_NAME}:" "$TEMP_FILE"; then
echo "Error: Failed to add branch protection block"
rm -f "$TEMP_FILE"
exit 1
fi

# Replace the original file with the modified version
mv "$TEMP_FILE" "$ASF_YAML_FILE"

echo "Successfully added branch protection for ${BRANCH_NAME} to $ASF_YAML_FILE"
echo ""
echo "Added block:"
echo " ${BRANCH_NAME}:"
echo " required_pull_request_reviews:"
echo " required_approving_review_count: 1"
echo ""
echo "Please review the changes and commit them."
Loading