Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#29697] Add prism artifact building workflow. #31369

Merged
merged 41 commits into from
May 29, 2024
Merged

Conversation

lostluck
Copy link
Contributor

@lostluck lostluck commented May 21, 2024

Adds a build & stage to RC creation to release signed hashed and zipped prism binaries to a release.

Currently requires a release with the RC tag to exist, actually straightening out the logic is best handled in a separate PR, as this change is sufficiently complex as is without including the clobber protection logic if targetting an already published release.

Changes the workflow dispatch to us a JSON object instead of individual string fields. The dialog now looks like this:

image

Not an ideal experience, but we hit the 10 input limit otherwise.

The action syncs down the Dev Apache SVN repo, builds prism binaries for ARM64, AMD64 for Linux, Darwin (osx), and Windows, zipping up the binary, creating signature and sha512 hash files.

These are uploaded to a Github prerelease for the RC tag, like so: https://github.com/apache/beam/releases/tag/untagged-819e4037252d95b05550

Finally, the same artifacts are committed to the Dev Apache repository for final publishing.

Signature and hash are not included in the Zip so the archive doesn't need to be decompressed before being able to validate it.

TODO in a subsequent PR:

  • automatically create a draft prerelease release for the tag if one doesn't exist yet.

  • Update existing draft GH release for the release version with the new RC tag.

  • Fail early if a published release has already gone out for the version to avoid clobbering an existing non-RC GH release.

  • Updating the Release Manager guide.

Part of #29697


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

@github-actions github-actions bot added the build label May 21, 2024
@lostluck lostluck marked this pull request as ready for review May 24, 2024 19:44
@lostluck
Copy link
Contributor Author

R: @kennknowles
cc: @damccorm @jrmccluskey @Abacn

Copy link
Contributor

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control

@Abacn
Copy link
Contributor

Abacn commented May 27, 2024

thanks. The refactoring makes sense as there are more and more fields in the action which is inconvenient. Just wondering if there is a way to keep the descriptions of field concise. Currently the dropdown is very long. (It's fine to keep as is for now)

Copy link
Contributor

@Abacn Abacn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, had two optional comments.

Do we plan to deliver prism binaries for Beam 2.57.0 (release cut soon)

.github/workflows/build_release_candidate.yml Show resolved Hide resolved
@lostluck
Copy link
Contributor Author

lostluck commented May 27, 2024

thanks, had two optional comments.

Do we plan to deliver prism binaries for Beam 2.57.0 (release cut soon)

Yup. That's why I was working on this all last week, to figure out and solve problems before we ran into them in the release.

It's easier to get the SDK downloading step working with a proper release having the artifacts. That half is for 2.58.0 though, and not 2.57.0.

@lostluck
Copy link
Contributor Author

thanks. The refactoring makes sense as there are more and more fields in the action which is inconvenient. Just wondering if there is a way to keep the descriptions of field concise. Currently the dropdown is very long. (It's fine to keep as is for now)

Unfortunately I tried a few things for this and nothing made it better. We're ultimately going to need to keep the release guide documentation (which is in a later PR), paired with this and link to the documentation here to reduce the visual noise.

TBH, we'd probably move the initiation for this action to a cooy/pastable command line to do the workflow dispatch. See https://cli.github.com/manual/gh_workflow_run

Copy link
Contributor

@Abacn Abacn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If Want LGTM from the release manager then it's good to add milestone to this PR. Otherwise this LGTM and feel free to merge

@lostluck lostluck added this to the 2.57.0 Release milestone May 27, 2024
@lostluck
Copy link
Contributor Author

Since Kenn's going to affected by it, hopefully he'll review it first!

@lostluck
Copy link
Contributor Author

Merging in, but I'm still going to be working with @kennknowles to make sure this is iterated on for completeness.

@lostluck lostluck merged commit 7d28155 into master May 29, 2024
9 of 10 checks passed
@lostluck lostluck deleted the prismBuildRelease branch May 29, 2024 03:14
lostluck added a commit to lostluck/beam that referenced this pull request Jun 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants