Skip to content
This repository was archived by the owner on Apr 2, 2024. It is now read-only.

Comments

Build arm64 packages#575

Draft
JamesGuthrie wants to merge 2 commits intomasterfrom
jg/arm64
Draft

Build arm64 packages#575
JamesGuthrie wants to merge 2 commits intomasterfrom
jg/arm64

Conversation

@JamesGuthrie
Copy link
Member

@JamesGuthrie JamesGuthrie commented Dec 2, 2022

Description

This commit adds support to build arm64 packages. The steps taken are
somewhat convoluted.

As GitHub doesn't offer hosted arm64 runners, we have a self-hosted
runner. For security reasons, GitHub doesn't allow self-hosted runners
to be used with public repositories. For that purpose, we have created
the promscale_extension_private repository, which is a private clone
of the promscale_extension repository.

The release GitHub workflow is designed to be run in both the public
and private repository. In the public repository its behaviour is
unchanged. In the private repository, it uses our self-hosted arm64
runners and attempts to attach the packages to the draft release which
was created in the promscale_extension repository.

Merge requirements

Please take into account the following non-code changes that you may need to make with your PR:

  • CHANGELOG entry for user-facing changes
  • Updated the relevant documentation

@JamesGuthrie JamesGuthrie changed the title WIP arm64 Build arm64 packages Dec 5, 2022
@JamesGuthrie JamesGuthrie force-pushed the jg/arm64 branch 3 times, most recently from bdb3624 to f293624 Compare December 5, 2022 12:29
All matrix builds were using the same cache key, causing collisions.

pgx puts the postgres data directory in `target/pgx-test-data-<pg_ver>`,
which the `Swatinem/rust-cache` action cleans, by removing all its
content, and subsequently caches. This breaks pgx tests, which do not
expect the directory to be empty.

The combination of the cache collisions and incorrect caching caused a
number of build failures, which could only be fixed by removing the
cached artifacts from the GitHub cache.
This commit adds support to build arm64 packages. The steps taken are
somewhat convoluted.

As GitHub doesn't offer hosted arm64 runners, we have a self-hosted
runner. For security reasons, GitHub doesn't allow self-hosted runners
to be used with public repositories. For that purpose, we have created
the `promscale_extension_private` repository, which is a private clone
of the `promscale_extension` repository.

The `release` GitHub workflow is designed to be run in _both_ the public
and private repository. In the public repository its behaviour is
unchanged. In the private repository, it uses our self-hosted arm64
runners and attempts to attach the packages to the draft release which
was created in the `promscale_extension` repository.
@JamesGuthrie
Copy link
Member Author

JamesGuthrie commented Dec 5, 2022

I have the following TODOs before this is going to work:

  • The Attach arm64 artifacts to release step is not going to work because when the action runs on timescale/promscale_extension_private repo, it doesn't have permissions to modify/attach items. It would be possible to do this, but it requires a PAT, which I'm concerned about because of the scope of access that it would allow.
  • The Upload Artifacts to PackageCloud step will not work in the timescale/promscale_extension_private repo until it has access to the IO_PACKAGECLOUD_TOKEN secret, which is an organization-level secret.
  • The Notify slack on build fail step will not work in timescale/promscale_extension_private because the SLACK_BOT_TOKEN secret is not configured there.

@JamesGuthrie JamesGuthrie requested a review from sumerman December 5, 2022 15:48
@JamesGuthrie
Copy link
Member Author

I opened this change as a PR to the promscale_extension_private repo as well: https://github.com/timescale/promscale_extension_private/pull/1, which illustrates that the build works there with these changes. Note: The idea is not that we will open PRs against that repo for builds, but that by pushing a tag to that repository, we can get the package built for a tag. We could potentially automate this step in the future.

@JamesGuthrie
Copy link
Member Author

It would also be nice to automate the tag push from promscale_extension to the promscale_extension_private repo. There is even a github action which does this: https://github.com/cpina/github-action-push-to-another-repository.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant