-
Notifications
You must be signed in to change notification settings - Fork 2
Add dune pkg binary release workflow #7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| # The actions/cache github action allows builds of tags to use artifacts cached | ||
| # by builds of the default branch, but doesn't allow builds of tags to use | ||
| # artifacts cached by other tags. Thus to make the dependencies of the project | ||
| # available to the "release" workflow, they are built and cached by this | ||
| # workflow. | ||
|
|
||
| name: Build and Cache Dependencies | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - 'main' | ||
|
|
||
| jobs: | ||
| release-unix: | ||
| name: Build and cache deps for ${{ matrix.os }} | ||
| runs-on: ${{ matrix.os }} | ||
| strategy: | ||
| matrix: | ||
| include: | ||
| - os: macos-15-intel | ||
| - os: macos-15 | ||
| - os: ubuntu-latest | ||
| steps: | ||
|
|
||
| - uses: actions/checkout@v4 | ||
|
|
||
| - uses: ocaml-dune/setup-dune@v0 | ||
|
|
||
| - name: Cache build artifacts | ||
| id: artifact-cache | ||
| uses: actions/cache@v4 | ||
| with: | ||
| path: | | ||
| _build | ||
| ~/.cache/dune | ||
| key: ${{ matrix.os }}-artifacts-${{ hashFiles('dune.lock') }} | ||
|
|
||
| - name: Build the project | ||
| run: dune build @install --release --only-packages container-image |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| name: release | ||
|
|
||
| on: | ||
| push: | ||
| tags: | ||
| - '*' | ||
|
|
||
| jobs: | ||
| release-unix: | ||
| name: Release for ${{ matrix.name }} | ||
| runs-on: ${{ matrix.os }} | ||
| strategy: | ||
| matrix: | ||
| include: | ||
| - os: macos-15-intel | ||
| name: x86_64-macos | ||
| - os: macos-15 | ||
| name: aarch64-macos | ||
| - os: ubuntu-latest | ||
| name: x86_64-linux | ||
| steps: | ||
|
|
||
| - uses: actions/checkout@v6 | ||
|
|
||
| - uses: ocaml-dune/setup-dune@v0 | ||
|
|
||
| - name: Load dependencies from the cache | ||
| id: artifact-cache | ||
| uses: actions/cache/restore@v4 | ||
| with: | ||
| path: | | ||
| _build | ||
| ~/.cache/dune | ||
| key: ${{ matrix.os }}-artifacts-${{ hashFiles('dune.lock') }} | ||
|
|
||
| - name: Build the project | ||
| run: dune build @install --only-packages container-image | ||
|
|
||
| - run: echo OUT_NAME=container-image-${{ github.ref_name }}-${{ matrix.name }} >> $GITHUB_ENV | ||
|
|
||
| - name: Release a tarball of build outputs | ||
| run: | | ||
| mkdir -p "$OUT_NAME" | ||
| cp -rlf _build/install/default/* "$OUT_NAME" | ||
| tar czf "$OUT_NAME.tar.gz" "$OUT_NAME" | ||
|
|
||
| - name: Upload assets | ||
| uses: ncipollo/release-action@v1 | ||
| with: | ||
| allowUpdates: true | ||
| artifacts: "*.tar.gz" | ||
|
|
||
|
|
||
| release-x86_64-linux-musl-static: | ||
| name: Build for x86_64-linux-musl-static | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
|
|
||
| - uses: actions/checkout@v6 | ||
|
|
||
| - run: echo OUT_NAME=container-image-${{ github.ref_name }}-x86_64-linux-statically-linked >> $GITHUB_ENV | ||
|
|
||
| - run: mkdir -p $OUT_NAME | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
| - run: docker buildx build --output type=local,dest=$OUT_NAME -f scripts/build-static.dockerfile . | ||
|
|
||
| - name: Test we can execute the exe on the host as a basic sanity check | ||
| run: $OUT_NAME/bin/image --help | ||
|
|
||
| - name: Make a tarball of build outputs | ||
| run: tar czf "$OUT_NAME.tar.gz" "$OUT_NAME" | ||
|
|
||
| - name: Upload assets | ||
| uses: ncipollo/release-action@v1 | ||
| with: | ||
| allowUpdates: true | ||
| artifacts: "*.tar.gz" | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,3 +15,8 @@ | |
| fmt.cli | ||
| logs.fmt | ||
| fmt.tty)) | ||
|
|
||
| (env | ||
| (static | ||
| (link_flags | ||
| (:standard -cclib -static)))) | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this really working to build a fully statically linked binary? Isn't there a missing musl here, since glibc doesn't support static linking? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it's more that glibc is recommended for static linking because parts of it don't work properly - a scary halfway house that it works sometimes? I'm not sure how to test it, but if the switch contained
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. glibc unfortunately degrades very badly if you statically link it despite ldd showing no dependencies. This is because various pieces of it (nss, resolv, iconv) can call dlopen and expect to find .so files around. In my experience musl or equivalent is the only safe way to statically link since it's explicitly supported there. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I was hoping that if any of those things could possibly be done, then the program would fail to link statically… but perhaps my faith is misplaced 🙈 Regardless, completely agree that linking with musl is the way to go…
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fair enough! For personal projects that build with dune package management I usually make statically-linked binaries by building in an alpine docker container with musl installed since the turnaround times for debugging are short (I can test locally) and since there's no glibc on the system at all I know it won't sneak into my build unexpectedly. Would that be an acceptable solution here or would it be better to keep building the project directly in a github action but start linking it with muslc?
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. An Alpine version is fine, but the problem (as always) is that the binary needs to be tested on the host distro it's going to be run on, and there are some considerations around static-pie to take into account. This thread is a good start about how to do it in opam: https://discuss.ocaml.org/t/generating-static-and-portable-executables-with-ocaml/8405
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I've added a docker-based action that statically links |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| (version 1.9.1) | ||
|
|
||
| (build | ||
| (all_platforms | ||
| ((action | ||
| (progn | ||
| (when %{pkg-self:dev} (run dune subst)) | ||
| (run dune build -p %{pkg-self:name} -j %{jobs} @install)))))) | ||
|
|
||
| (depends | ||
| (all_platforms | ||
| (dune ocaml fmt astring cmdliner re stdlib-shims uutf ocaml-syntax-shims))) | ||
|
|
||
| (source | ||
| (fetch | ||
| (url | ||
| https://github.com/mirage/alcotest/releases/download/1.9.1/alcotest-1.9.1.tbz) | ||
| (checksum | ||
| sha256=1e29c3b41d4329062105b723dfda3aff86b8cef5e7c7500d0e491fc5fd78e482))) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| (version 0.16.1) | ||
|
|
||
| (build | ||
| (all_platforms | ||
| ((action | ||
| (progn | ||
| (when %{pkg-self:dev} (run dune subst)) | ||
| (run dune build -p %{pkg-self:name} -j %{jobs})))))) | ||
|
|
||
| (depends | ||
| (all_platforms | ||
| (ocaml dune bigstringaf ocaml-syntax-shims))) | ||
|
|
||
| (source | ||
| (fetch | ||
| (url https://github.com/inhabitedtype/angstrom/archive/0.16.1.tar.gz) | ||
| (checksum md5=a9e096b4b2b8e4e3bb17d472bbccaad0))) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| (version 0.3.2) | ||
|
|
||
| (build | ||
| (all_platforms | ||
| ((action | ||
| (progn | ||
| (when %{pkg-self:dev} (run dune subst)) | ||
| (run dune build -p %{pkg-self:name} -j %{jobs})))))) | ||
|
|
||
| (depends | ||
| (all_platforms | ||
| (ocaml dune ptime))) | ||
|
|
||
| (source | ||
| (fetch | ||
| (url | ||
| https://github.com/mirleft/ocaml-asn1-combinators/releases/download/v0.3.2/asn1-combinators-0.3.2.tbz) | ||
| (checksum | ||
| sha256=2b26985f6e2722073dcd9f84355bd6757e12643b5a48e30b3c07ff7cfb0d8a7f))) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| (version 0.8.5) | ||
|
|
||
| (build | ||
| (all_platforms | ||
| ((action (run ocaml pkg/pkg.ml build --pinned %{pkg-self:pinned}))))) | ||
|
|
||
| (depends | ||
| (all_platforms | ||
| (ocaml ocamlfind ocamlbuild topkg))) | ||
|
|
||
| (source | ||
| (fetch | ||
| (url https://erratique.ch/software/astring/releases/astring-0.8.5.tbz) | ||
| (checksum | ||
| sha256=865692630c07c3ab87c66cdfc2734c0fdfc9c34a57f8e89ffec7c7d15e7a70fa))) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| (version base) | ||
|
|
||
| (depends | ||
| (all_platforms (ocaml))) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| (version base) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| (version base) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| (version v0.17.3) | ||
|
|
||
| (build | ||
| (all_platforms ((action (run dune build -p %{pkg-self:name} -j %{jobs}))))) | ||
|
|
||
| (depends | ||
| (all_platforms | ||
| (ocaml ocaml_intrinsics_kernel sexplib0 dune dune-configurator))) | ||
|
|
||
| (source | ||
| (fetch | ||
| (url https://github.com/janestreet/base/archive/refs/tags/v0.17.3.tar.gz) | ||
| (checksum md5=2100b0ed13fecf43be86ed45c5b2cc4d))) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| (version 3.5.2) | ||
|
|
||
| (build | ||
| (all_platforms | ||
| ((action | ||
| (progn | ||
| (when %{pkg-self:dev} (run dune subst)) | ||
| (run dune build -p %{pkg-self:name} -j %{jobs})))))) | ||
|
|
||
| (depends | ||
| (all_platforms | ||
| (ocaml dune))) | ||
|
|
||
| (source | ||
| (fetch | ||
| (url | ||
| https://github.com/mirage/ocaml-base64/releases/download/v3.5.2/base64-3.5.2.tbz) | ||
| (checksum | ||
| sha256=b3f5ce301aa72c7032ef90be2332d72ff3962922c00ee2aec6bcade187a2f59b))) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| (version 0.10.0) | ||
|
|
||
| (build | ||
| (all_platforms | ||
| ((action | ||
| (progn | ||
| (when %{pkg-self:dev} (run dune subst)) | ||
| (run dune build -p %{pkg-self:name} -j %{jobs} @install)))))) | ||
|
|
||
| (depends | ||
| (all_platforms | ||
| (dune dune-configurator ocaml))) | ||
|
|
||
| (source | ||
| (fetch | ||
| (url https://github.com/inhabitedtype/bigstringaf/archive/0.10.0.tar.gz) | ||
| (checksum md5=be0a44416840852777651150757a0a3b))) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| (version 0.2.1) | ||
|
|
||
| (build | ||
| (all_platforms | ||
| ((action (run ocaml pkg/pkg.ml build --dev-pkg %{pkg-self:dev}))))) | ||
|
|
||
| (depends | ||
| (all_platforms | ||
| (ocaml ocamlfind ocamlbuild topkg base-unix rresult astring fpath fmt logs))) | ||
|
|
||
| (source | ||
| (fetch | ||
| (url https://erratique.ch/software/bos/releases/bos-0.2.1.tbz) | ||
| (checksum | ||
| sha512=8daeb8a4c2dd1f2460f6274ada19f4f1b6ebe875ff83a938c93418ce0e6bdb74b8afc5c9a7d410c1c9df2dad030e4fa276b6ed2da580639484e8b5bc92610b1d))) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| (version 0.5.2) | ||
|
|
||
| (build | ||
| (all_platforms ((action (run dune build -p %{pkg-self:name} -j %{jobs}))))) | ||
|
|
||
| (depends | ||
| (all_platforms | ||
| (ocaml dune dune-configurator optint))) | ||
|
|
||
| (source | ||
| (fetch | ||
| (url | ||
| https://github.com/mirage/checkseum/releases/download/v0.5.2/checkseum-0.5.2.tbz) | ||
| (checksum | ||
| sha256=9e5e4fd4405cb4a8b4df00877543251833e08a6499ef42ccb8dba582df0dafc8))) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| (version 2.0.0) | ||
|
|
||
| (install | ||
| (all_platforms | ||
| (progn | ||
| (run | ||
| %{make} | ||
| install | ||
| BINDIR=%{pkg-self:bin} | ||
| LIBDIR=%{pkg-self:lib} | ||
| DOCDIR=%{pkg-self:doc} | ||
| SHAREDIR=%{share} | ||
| MANDIR=%{man}) | ||
| (run | ||
| %{make} | ||
| install-doc | ||
| LIBDIR=%{pkg-self:lib} | ||
| DOCDIR=%{pkg-self:doc} | ||
| SHAREDIR=%{share} | ||
| MANDIR=%{man})))) | ||
|
|
||
| (build | ||
| (all_platforms ((action (run %{make} all PREFIX=%{prefix}))))) | ||
|
|
||
| (depends | ||
| (all_platforms (ocaml))) | ||
|
|
||
| (source | ||
| (fetch | ||
| (url https://erratique.ch/software/cmdliner/releases/cmdliner-2.0.0.tbz) | ||
| (checksum | ||
| sha512=a7bd4eeb0cef7c08bca73b0077a65f748c19a230544133b39fc3360feb2cf0af08416a8b84031c94a2f4a007d5920a4db1368d87b9eeca561671828e2dad2885))) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| (version 6.1.1) | ||
|
|
||
| (build | ||
| (all_platforms | ||
| ((action | ||
| (progn | ||
| (when %{pkg-self:dev} (run dune subst)) | ||
| (run dune build -p %{pkg-self:name} -j %{jobs} @install)))))) | ||
|
|
||
| (depends | ||
| (all_platforms | ||
| (dune base-domains cohttp eio logs uri fmt ptime http))) | ||
|
|
||
| (source | ||
| (fetch | ||
| (url | ||
| https://github.com/mirage/ocaml-cohttp/releases/download/v6.1.1/cohttp-6.1.1.tbz) | ||
| (checksum | ||
| sha256=6b420c56203b3a0b515964f036bcea0fb9a362876b5791cd7ff50e12366c489c))) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| (version 6.1.1) | ||
|
|
||
| (build | ||
| (all_platforms | ||
| ((action | ||
| (progn | ||
| (when %{pkg-self:dev} (run dune subst)) | ||
| (run dune build -p %{pkg-self:name} -j %{jobs} @install)))))) | ||
|
|
||
| (depends | ||
| (all_platforms | ||
| (dune | ||
| http | ||
| ocaml | ||
| re | ||
| uri | ||
| uri-sexp | ||
| logs | ||
| sexplib0 | ||
| ppx_sexp_conv | ||
| stringext | ||
| base64))) | ||
|
|
||
| (source | ||
| (fetch | ||
| (url | ||
| https://github.com/mirage/ocaml-cohttp/releases/download/v6.1.1/cohttp-6.1.1.tbz) | ||
| (checksum | ||
| sha256=6b420c56203b3a0b515964f036bcea0fb9a362876b5791cd7ff50e12366c489c))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this just use
actions/upload-artifact@v4instead? See https://github.com/actions/upload-artifact?tab=readme-ov-file#inputs insThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It depends on what sort of release you're doing. I'm trying to demonstrate releasing an archive with pre-built binaries, including making it available on the project's "Releases" page. But if you just want the archive to be uploaded as a workflow artifact and made available to other workflows, then
actions/upload-artifactcan be used instead. Let me know if you'd like me to change this workflow to the latter.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That makes sense; the upload-artifact is actually quite useful as it can run on PRs as well, and let branched-binaries-with-a-new-feature be quickly downloadable to try out. Not directly applicable here as this workflow on runs on push, but this could be generalised to work on pull_request too.