Skip to content

Commit 944f02c

Browse files
tyuchntyuchn
committed
Squashed 'release-tools/' changes from 984feece..dc4d0ae2
dc4d0ae2 Merge pull request #249 from jsafrane/use-go-version e681b170 Use .go-version to get Kubernetes go version b54c1ba4 Merge pull request #246 from xing-yang/go_1.21 5436c81e Change go version to 1.21.5 267b40e9 Merge pull request #244 from carlory/sig-storage b42e5a2d nominate self (carlory) as kubernetes-csi reviewer a17f536f Merge pull request #210 from sunnylovestiramisu/sidecar 011033de Use set -x instead of die 5deaf667 Add wrapper script for sidecar release f8c8cc4c Merge pull request #237 from msau42/prow b36b5bfd Merge pull request #240 from dannawang0221/upgrade-go-version adfddcc9 Merge pull request #243 from pohly/git-subtree-pull-fix c4650889 pull-test.sh: avoid "git subtree pull" error 7b175a1e Update csi-test version to v5.2.0 987c90cc Update go version to 1.21 to match k/k 2c625d41 Add script to generate patch release notes f9d5b9c0 Merge pull request #236 from mowangdk/feature/bump_csi-driver-host-path_version b01fd537 Bump csi-driver-host-path version up to v1.12.0 git-subtree-dir: release-tools git-subtree-split: dc4d0ae20a3dcce17fbfc745fb1f1e3b10cd9644
1 parent 607474e commit 944f02c

File tree

6 files changed

+286
-47
lines changed

6 files changed

+286
-47
lines changed

‎KUBERNETES_CSI_OWNERS_ALIASES‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ aliases:
1818
# when they are temporarily unable to review PRs.
1919
kubernetes-csi-reviewers:
2020
- andyzhangx
21+
- carlory
2122
- chrishenzie
2223
- ggriffiths
2324
- gnufied

‎SIDECAR_RELEASE_PROCESS.md‎

Lines changed: 37 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -46,54 +46,48 @@ naming convention `<hostpath-deployment-version>-on-<kubernetes-version>`.
4646
## Release Process
4747
1. Identify all issues and ongoing PRs that should go into the release, and
4848
drive them to resolution.
49-
1. Download the latest version of the
50-
[K8s release notes generator](https://github.com/kubernetes/release/tree/HEAD/cmd/release-notes)
51-
1. Create a
52-
[Github personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
53-
with `repo:public_repo` access
54-
1. Generate release notes for the release. Replace arguments with the relevant
55-
information.
56-
* Clean up old cached information (also needed if you are generating release
57-
notes for multiple repos)
58-
```bash
59-
rm -rf /tmp/k8s-repo
60-
```
61-
* For new minor releases on master:
62-
```bash
63-
GITHUB_TOKEN=<token> release-notes \
64-
--discover=mergebase-to-latest \
65-
--org=kubernetes-csi \
66-
--repo=external-provisioner \
67-
--required-author="" \
68-
--markdown-links \
69-
--output out.md
70-
```
71-
* For new patch releases on a release branch:
72-
```bash
73-
GITHUB_TOKEN=<token> release-notes \
74-
--discover=patch-to-latest \
75-
--branch=release-1.1 \
76-
--org=kubernetes-csi \
77-
--repo=external-provisioner \
78-
--required-author="" \
79-
--markdown-links \
80-
--output out.md
81-
```
82-
1. Compare the generated output to the new commits for the release to check if
83-
any notable change missed a release note.
84-
1. Reword release notes as needed. Make sure to check notes for breaking
85-
changes and deprecations.
86-
1. If release is a new major/minor version, create a new `CHANGELOG-<major>.<minor>.md`
87-
file. Otherwise, add the release notes to the top of the existing CHANGELOG
88-
file for that minor version.
89-
1. Submit a PR for the CHANGELOG changes.
90-
1. Submit a PR for README changes, in particular, Compatibility, Feature status,
91-
and any other sections that may need updating.
49+
1. Update dependencies for sidecars via
50+
[go-modules-update.sh](https://github.com/kubernetes-csi/csi-driver-host-path/blob/HEAD/release-tools/go-modules-update.sh),
51+
and get PRs approved and merged.
9252
1. Check that all [canary CI
9353
jobs](https://testgrid.k8s.io/sig-storage-csi-ci) are passing,
9454
and that test coverage is adequate for the changes that are going into the release.
9555
1. Check that the post-\<sidecar\>-push-images builds are succeeding.
9656
[Example](https://testgrid.k8s.io/sig-storage-image-build#post-external-snapshotter-push-images)
57+
1. Generate release notes.
58+
1. Download the latest version of the [K8s release notes generator](https://github.com/kubernetes/release/tree/HEAD/cmd/release-notes)
59+
1. Create a
60+
[Github personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
61+
with `repo:public_repo` access
62+
1. For patch release, use the script generate_patch_release_notes.sh. Read the instructions at the top of the
63+
script. The script also creates PRs for each branch.
64+
1. For new minor releases, follow these steps and replace arguments with the relevant
65+
information.
66+
* Clean up old cached information (also needed if you are generating release
67+
notes for multiple repos)
68+
```bash
69+
rm -rf /tmp/k8s-repo
70+
```
71+
* For new minor releases on master:
72+
```bash
73+
GITHUB_TOKEN=<token> release-notes \
74+
--discover=mergebase-to-latest \
75+
--org=kubernetes-csi \
76+
--repo=external-provisioner \
77+
--required-author="" \
78+
--markdown-links \
79+
--output out.md
80+
```
81+
1. Compare the generated output to the new commits for the release to check if
82+
any notable change missed a release note.
83+
1. Reword release notes as needed, ideally in the original PRs so that the
84+
release notes can be regnerated. Make sure to check notes for breaking
85+
changes and deprecations.
86+
1. If release is a new major/minor version, create a new `CHANGELOG-<major>.<minor>.md`
87+
file.
88+
1. Submit a PR for the CHANGELOG changes.
89+
1. Submit a PR for README changes, in particular, Compatibility, Feature status,
90+
and any other sections that may need updating.
9791
1. Make sure that no new PRs have merged in the meantime, and no PRs are in
9892
flight and soon to be merged.
9993
1. Create a new release following a previous release as a template. Be sure to select the correct

‎generate_patch_release_notes.sh‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
#!/bin/bash
2+
3+
# Copyright 2023 The Kubernetes Authors.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
17+
18+
# Usage: generate_patch_release_notes.sh
19+
#
20+
# Generates and creates PRs for kubernetes-csi patch releases.
21+
#
22+
# Required environment variables
23+
# CSI_RELEASE_TOKEN: Github token needed for generating release notes
24+
# GITHUB_USER: Github username to create PRs with
25+
#
26+
# Instructions:
27+
# 1. Login with "gh auth login"
28+
# 2. Copy this script to the kubernetes-csi directory (one directory above the
29+
# repos)
30+
# 3. Update the repos and versions in the $releases array
31+
# 4. Set environment variables
32+
# 5. Run script from the kubernetes-csi directory
33+
#
34+
# Caveats:
35+
# - This script doesn't handle regenerating and updating existing PRs yet.
36+
# It might work if you comment out the PR creation line
37+
38+
set -e
39+
set -x
40+
41+
releases=(
42+
# "external-attacher 4.4.1"
43+
# "external-provisioner 3.6.1"
44+
# "external-snapshotter 6.2.3"
45+
)
46+
47+
function gen_patch_relnotes() {
48+
rm out.md || true
49+
rm -rf /tmp/k8s-repo || true
50+
GITHUB_TOKEN="$CSI_RELEASE_TOKEN" \
51+
release-notes --discover=patch-to-latest --branch="$2" \
52+
--org=kubernetes-csi --repo="$1" \
53+
--required-author="" --markdown-links --output out.md
54+
}
55+
56+
for rel in "${releases[@]}"; do
57+
read -r repo version <<< "$rel"
58+
59+
# Parse minor version
60+
minorPattern="(^[[:digit:]]+\.[[:digit:]]+)\."
61+
[[ "$version" =~ $minorPattern ]]
62+
minor="${BASH_REMATCH[1]}"
63+
64+
echo "$repo" "$version" "$minor"
65+
66+
pushd "$repo/CHANGELOG"
67+
68+
git fetch upstream
69+
70+
# Create branch
71+
branch="changelog-release-$minor"
72+
git checkout master
73+
git branch -D "$branch" || true
74+
git checkout --track "upstream/release-$minor" -b "$branch"
75+
76+
# Generate release notes
77+
gen_patch_relnotes "$repo" "release-$minor"
78+
cat > tmp.md <<EOF
79+
# Release notes for v$version
80+
81+
[Documentation](https://kubernetes-csi.github.io)
82+
83+
EOF
84+
85+
cat out.md >> tmp.md
86+
echo >> tmp.md
87+
88+
file="CHANGELOG-$minor.md"
89+
cat "$file" >> tmp.md
90+
mv tmp.md "$file"
91+
92+
git add -u
93+
git commit -m "Add changelog for $version"
94+
git push -f origin "$branch"
95+
96+
# Create PR
97+
prbody=$(cat <<EOF
98+
\`\`\`release-note
99+
NONE
100+
\`\`\`
101+
EOF
102+
)
103+
gh pr create --title="Changelog for v$version" --body "$prbody" --head "$GITHUB_USER:$branch" --base "release-$minor" --repo="kubernetes-csi/$repo"
104+
105+
popd
106+
done

‎go-modules-update.sh‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
#!/bin/sh
2+
3+
# Copyright 2023 The Kubernetes Authors.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
17+
18+
# Usage: go-modules-update.sh
19+
#
20+
# Batch update dependencies for sidecars.
21+
#
22+
# Required environment variables
23+
# CSI_RELEASE_TOKEN: Github token needed for generating release notes
24+
# GITHUB_USER: Github username to create PRs with
25+
#
26+
# Instructions:
27+
# 1. Login with "gh auth login"
28+
# 2. Copy this script to the kubernetes-csi directory (one directory above the
29+
# repos)
30+
# 3. Update the repos and master branch so locally it has the latest upstream
31+
# change
32+
# 4. Set environment variables
33+
# 5. Run script from the kubernetes-csi directory
34+
#
35+
# Caveats:
36+
# - This script doesn't handle interface incompatibility of updates.
37+
# You need to resolve interface incompatibility case by case. The
38+
# most frequent case is to update the interface(new parameters,
39+
# name change of the method, etc.)in the sidecar repo and make sure
40+
# the build and test pass.
41+
42+
43+
set -e
44+
set -x
45+
46+
MAX_RETRY=10
47+
48+
# Get the options
49+
while getopts ":u:v:" option; do
50+
case $option in
51+
u) # Set username
52+
username=$OPTARG;;
53+
v) # Set version
54+
v=$OPTARG;;
55+
\?) # Invalid option
56+
echo "Error: Invalid option: $OPTARG"
57+
exit;;
58+
esac
59+
done
60+
61+
# Only need to do this once
62+
gh auth login
63+
64+
while read -r repo branches; do
65+
if [ "$repo" != "#" ]; then
66+
(
67+
cd "$repo"
68+
git fetch origin
69+
for i in $branches; do
70+
if [ "$(git rev-parse --verify "module-update-$i" 2>/dev/null)" ]; then
71+
git checkout master && git branch -d "module-update-$i"
72+
fi
73+
git checkout -B "module-update-$i" "origin/$i"
74+
rm -rf .git/MERGE*
75+
if ! git subtree pull --squash --prefix=release-tools https://github.com/kubernetes-csi/csi-release-tools.git master; then
76+
# Sometimes "--squash" leads to merge conflicts. Because we know that "release-tools"
77+
# is an unmodified copy of csi-release-tools, we can automatically resolve that
78+
# by replacing it completely.
79+
if [ -e .git/MERGE_MSG ] && [ -e .git/FETCH_HEAD ] && grep -q "^# Conflict" .git/MERGE_MSG; then
80+
rm -rf release-tools
81+
mkdir release-tools
82+
git archive FETCH_HEAD | tar -C release-tools -xf -
83+
git add release-tools
84+
git commit --file=.git/MERGE_MSG
85+
else
86+
exit 1
87+
fi
88+
fi
89+
RETRY=0
90+
while ! ./release-tools/go-get-kubernetes.sh -p "$v" && RETRY < $MAX_RETRY
91+
do
92+
RETRY=$((RETRY+1))
93+
go mod tidy && go mod vendor && go mod tidy
94+
done
95+
go mod tidy && go mod vendor && go mod tidy
96+
git add --all
97+
git commit -m "Update dependency go modules for k8s v$v"
98+
git remote set-url origin "https://github.com/$username/$repo.git"
99+
make test
100+
git push origin "module-update-$i" --force
101+
# Create PR
102+
prbody=$(cat <<EOF
103+
Ran kubernetes-csi/csi-release-tools go-get-kubernetes.sh -p ${v}.
104+
105+
106+
\`\`\`release-note
107+
Update kubernetes dependencies to v${v}
108+
\`\`\`
109+
EOF
110+
)
111+
gh pr create --title="Update dependency go modules for k8s v$v" --body "$prbody" --head "$username:module-update-master" --base "master" --repo="kubernetes-csi/$repo"
112+
done
113+
)
114+
fi
115+
done <<EOF
116+
csi-driver-host-path master
117+
csi-driver-iscsi master
118+
csi-driver-nfs master
119+
csi-lib-utils master
120+
csi-proxy master
121+
csi-test master
122+
external-attacher master
123+
external-health-monitor master
124+
external-provisioner master
125+
external-resizer master
126+
external-snapshotter master
127+
livenessprobe master
128+
node-driver-registrar master
129+
EOF

‎prow.sh‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ configvar CSI_PROW_BUILD_PLATFORMS "linux amd64 amd64; linux ppc64le ppc64le -pp
8686
# which is disabled with GOFLAGS=-mod=vendor).
8787
configvar GOFLAGS_VENDOR "$( [ -d vendor ] && echo '-mod=vendor' )" "Go flags for using the vendor directory"
8888

89-
configvar CSI_PROW_GO_VERSION_BUILD "1.20" "Go version for building the component" # depends on component's source code
89+
configvar CSI_PROW_GO_VERSION_BUILD "1.21.5" "Go version for building the component" # depends on component's source code
9090
configvar CSI_PROW_GO_VERSION_E2E "" "override Go version for building the Kubernetes E2E test suite" # normally doesn't need to be set, see install_e2e
9191
configvar CSI_PROW_GO_VERSION_SANITY "${CSI_PROW_GO_VERSION_BUILD}" "Go version for building the csi-sanity test suite" # depends on CSI_PROW_SANITY settings below
9292
configvar CSI_PROW_GO_VERSION_KIND "${CSI_PROW_GO_VERSION_BUILD}" "Go version for building 'kind'" # depends on CSI_PROW_KIND_VERSION below
@@ -199,7 +199,7 @@ kindest/node:v1.18.20@sha256:738cdc23ed4be6cc0b7ea277a2ebcc454c8373d7d8fb991a7fc
199199
# If the deployment script is called with CSI_PROW_TEST_DRIVER=<file name> as
200200
# environment variable, then it must write a suitable test driver configuration
201201
# into that file in addition to installing the driver.
202-
configvar CSI_PROW_DRIVER_VERSION "v1.11.0" "CSI driver version"
202+
configvar CSI_PROW_DRIVER_VERSION "v1.12.0" "CSI driver version"
203203
configvar CSI_PROW_DRIVER_REPO https://github.com/kubernetes-csi/csi-driver-host-path "CSI driver repo"
204204
configvar CSI_PROW_DEPLOYMENT "" "deployment"
205205
configvar CSI_PROW_DEPLOYMENT_SUFFIX "" "additional suffix in kubernetes-x.yy[suffix].yaml files"
@@ -240,7 +240,7 @@ configvar CSI_PROW_SIDECAR_E2E_IMPORT_PATH "none" "CSI Sidecar E2E package"
240240
# of the cluster. The alternative would have been to (cross-)compile csi-sanity
241241
# and install it inside the cluster, which is not necessarily easier.
242242
configvar CSI_PROW_SANITY_REPO https://github.com/kubernetes-csi/csi-test "csi-test repo"
243-
configvar CSI_PROW_SANITY_VERSION v5.0.0 "csi-test version"
243+
configvar CSI_PROW_SANITY_VERSION v5.2.0 "csi-test version"
244244
configvar CSI_PROW_SANITY_PACKAGE_PATH github.com/kubernetes-csi/csi-test "csi-test package"
245245
configvar CSI_PROW_SANITY_SERVICE "hostpath-service" "Kubernetes TCP service name that exposes csi.sock"
246246
configvar CSI_PROW_SANITY_POD "csi-hostpathplugin-0" "Kubernetes pod with CSI driver"
@@ -564,7 +564,15 @@ go_version_for_kubernetes () (
564564
local version="$2"
565565
local go_version
566566

567-
# We use the minimal Go version specified for each K8S release (= minimum_go_version in hack/lib/golang.sh).
567+
# Try to get the version for .go-version
568+
go_version="$( cat "$path/.go-version" )"
569+
if [ "$go_version" ]; then
570+
echo "$go_version"
571+
return
572+
fi
573+
574+
# Fall back to hack/lib/golang.sh parsing.
575+
# This is necessary in v1.26.0 and older Kubernetes releases that do not have .go-version.
568576
# More recent versions might also work, but we don't want to count on that.
569577
go_version="$(grep minimum_go_version= "$path/hack/lib/golang.sh" | sed -e 's/.*=go//')"
570578
if ! [ "$go_version" ]; then

‎pull-test.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ CSI_RELEASE_TOOLS_DIR="$(pwd)"
2525

2626
# Update the other repo.
2727
cd "$PULL_TEST_REPO_DIR"
28+
git reset --hard # Shouldn't be necessary, but somehow is to avoid "fatal: working tree has modifications. Cannot add." (https://stackoverflow.com/questions/3623351/git-subtree-pull-says-that-the-working-tree-has-modifications-but-git-status-sa)
2829
git subtree pull --squash --prefix=release-tools "$CSI_RELEASE_TOOLS_DIR" master
2930
git log -n2
3031

0 commit comments

Comments
 (0)