`astro deploy` results in segmentation fault

[jientile]

Describe the bug

Upon starting a new project and deploying it, the image building step fails with a segmentation fault.

~/tmp$ mkdir astro
~/tmp$ cd astro
~/tmp/astro$ astro dev init
Initialized empty Astro project in /Users/steve/tmp/astro
~/tmp/astro$ astro dev start
✔ Astro machine initialized
✔ Project image has been updated
✔ Project started
- Airflow UI: http://localhost:8080
- Postgres Database: postgresql://localhost:5432/postgres
- The default Postgres DB credentials are: postgres:postgres
~/tmp/astro$ astro deploy
Authenticated to Astro 

Select a Deployment
 #     DEPLOYMENT NAME     RELEASE NAME                  DEPLOYMENT ID                 DAG DEPLOY ENABLED     
 1     steve-dev            linear-electronics-3140       cmgz5kk7k15a501jxbmnzsh9s     true                   
 2     steve-prod           radiant-spectroscope-5100     cmgz5krwt15ar01jxvb2dyxpm     true                   

> 1
Building image...
STEP 1/1: FROM astrocrpublic.azurecr.io/runtime:3.1-2
Trying to pull astrocrpublic.azurecr.io/runtime:3.1-2...
Getting image source signatures
Copying blob sha256:b6d370e0dcfccc06110f33825b4628b749be3fcc6122e66e222db0a51f987aa2
Copying blob sha256:e5ce4ff4a9f3bc5a4037d8b9c316bc8df4b52f8928007c634429ce103e8d7615
Copying blob sha256:453345f9dcb4a5572979c63f2bd57a1428c6f94367126ad59241214229570adb
Copying blob sha256:4ed247da84f7bed630d055baea823059b0bbf42892b674af3a5d99f1b5c41dd1
Copying blob sha256:9dee5a08a32de4efbadb89a533e09e538a5e8d457245e5bc83b7146a99ea6746
Copying blob sha256:a2a1495cbbb8e340977493fc3401a3c70c39580aa5b5904be6bc299ffe61f2dc
Copying blob sha256:a73b43cbe5e1d64b2293bea231738060867fad0e7b98e021b08db7df42678b53
Copying blob sha256:a964371027b9706f7c815a2a2e95a1201d5a81517422f115c50dab9e3ecb3d94
Copying blob sha256:f843f077a7053575a2010f43e24eb033005b2b4aeeb9c381c80fd87b1dfe2e66
Copying blob sha256:6251e74fb9cebf161e8b501f1fb3667fa68ab52794dd6d362e1bcd520d42ad33
Copying blob sha256:d3382d9754d831435527f8f707e63f4bf76127ef94fc0fca8a195ee25959f95c
Copying blob sha256:cc6ba8c554c78e5553b480882b7d3d78b4c023c78b281e8324218bbacf97e92b
Copying blob sha256:e3e71226cf75be63ef75e019ae6b7554c1a6fdbd2abc303d8d333477fb29cc25
Copying blob sha256:a8a092033c5b0ada64b18771771e99f33c3de611045117f46bbf40e91b870eda
Copying blob sha256:0cd394466b01abab29089a235b1139aa28b8aea6c2e567c862f527b7d841a0e0
Copying blob sha256:4f3fa4d9ed43f1b74a1ca227535cf6327f6fc57d2fdb33484ed6dd446080b518
Copying blob sha256:93243767137a960fb88814e96fb10662cff575a03dbc52ef7766bba35b22a8f6
Copying blob sha256:0cd394466b01abab29089a235b1139aa28b8aea6c2e567c862f527b7d841a0e0
Copying blob sha256:9b88c60a7c44513a3a9089df5aa90036f08bd01ee35905881e21be00c4a65e2c
Copying blob sha256:3828afabb092782a18d3cea1d8ec117abc1cc574e0a4a6a4209fb1e6b8343dc0
Copying blob sha256:0ef725c1ceae66f75daf251abef840aa5814613e86cd8ea44e47ff2b64ef2688
Copying blob sha256:4d218f36d15f9a899ded22c5109487138cf0d5dec8027392c8438c18e38b04b2
Copying blob sha256:63df3c6f68c9124fa56e7fb14fd26a0c2daa28045b230ddd9b4aecec85942ff4
Copying blob sha256:9f42fd6b7c61a080e5069094e25fdc0b774e9e932ce9bad03ce7a4b42575dceb
Copying blob sha256:ad4110fc0e9006f4fc2dd42fb51af50aa4b646f3e09495c7bd5600c8b7f2a3bd
Copying blob sha256:abda7257129294c8bc163dbe5870dc8f95cba3a12266b94bc7b675e4684298ac
Copying blob sha256:f172da04e8e4bdc0977ac95c50ef9b0028234ac06ff82e23c7b7f2daf547023b
Copying blob sha256:4c12e2ed61ae62349faec3dafec695bf4e1df196b47d127f65bc7ca3dffbb5fb
Copying blob sha256:bd9ddc54bea929a22b334e73e026d4136e5b73f5cc29942896c72e4ece69b13d
Copying config sha256:364a71c60050eab40369c15721b7ff40eb29f02bf043305838fa3c2ed1335c97
Writing manifest to image destination
STEP 2/8: COPY packages.txt .
--> 16459ee5e8e6
STEP 3/8: USER root
--> 5884938e36b8
STEP 4/8: RUN /usr/local/bin/install-system-packages
+ /usr/local/bin/install-system-packages
--> b4bd09292790
STEP 5/8: COPY requirements.txt .
--> e252f9d0a786
STEP 6/8: RUN /usr/local/bin/install-python-dependencies
+ /usr/local/bin/install-python-dependencies
Installing python dependencies using uv
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
/usr/local/bin/install-python-dependencies: line 15:    11 Segmentation fault      (core dumped) uv pip install --system --no-cache-dir -r requirements.txt -c /etc/pip-constraints.txt
Error: building at STEP "RUN /usr/local/bin/install-python-dependencies": while running runtime: exit status 139

Error: an error was encountered while building the image, see the build logs for details

What CLI Version did you experience this bug?

~/tmp/astro$ astro version
Astro CLI Version: 1.37.0

This CLI bug is related to which Astronomer Platform?

• Astro
• Software
• None/Unknown

What Operating System is the above CLI installed on?

~/tmp/astro$ sw_vers 
ProductName:		macOS
ProductVersion:		15.7.1
BuildVersion:		24G231

🪜 Steps To Reproduce

Run in order:

1. astro dev init
2. astro dev start
3. astro deploy

[schnie]

Hey @jientile thanks for reporting. Does your requirements.txt contain anything? Would you mind sharing if so?

Also curious, which docker runtime are you using locally? Docker Desktop, Podman, Orbstack or something else?

[jientile]

Thank you for the follow-up, @schnie. I can confirm that the issue occurs even with a pristine requirements.txt, as generated by astro dev init. For the container runtime, I am using Podman version 5.6.2.

[jientile]

If I replace the first step in the “Steps to Reproduce” with astro dev init --airflow-version 2.10.5, the deployment works as expected.

[schnie]

@jientile thanks for the details. Curious, is your mac running intel or apple silicon? This is working for me on my m1 mac. I also noticed the qemu which made me wonder if you were running on an intel.

[jientile]

Curious, is your mac running intel or apple silicon?

I am using Apple Silicon, specifically an Apple M1 Pro. I am also baffled as to why qemu is being invoked.

[kissmygritts]

All of my team members are seeing the same issue. We all have Apple M3 Pro chips, running podman, and using airflow v3.

This happens during the deploy process when the amd64 images are attempting to build. As far as I can tell it only happens when building in the astro-machine podman machine. When using the default podman machine (after running podman machine init) and building with podman build --platform=linux/amd64 -t astro-dev . the image will build. Then I can deploy with astro deploy -i astro-dev <deployment id>. However, this is not a great solution.

I recently switched our image process a bit that caused us to hit this issue. Previously, I was building the python dependencies in a different image, then copying them into the astronomer runtime image.

[nickstenning]

Hmmm, this sounds to me like podman is falling back to qemu when it should presumably be using Rosetta to build on an M3 Pro.

Could you share the contents of your ~/.config/containers/containers.conf?

[nickstenning]

Ok, having reproduced this locally (on an M4 Pro) it looks like this is an upstream issue.

Podman 5.6 ships with Rosetta disabled by default due to the compatibility issues between Rosetta and Linux kernels 6.13 and above. This means users will be using QEMU for architecture translation which is not as performant as Rosetta in many situations.

You can verify that Rosetta support is missing from your astro-machine builder as follows

$ podman machine ssh astro-machine cat /proc/sys/fs/binfmt_misc/rosetta
cat: /proc/sys/fs/binfmt_misc/rosetta: No such file or directory

I think our only workaround for now will be to either downgrade podman or switch to using Docker for Mac/OrbStack. If needed, you can force astro to use a different container binary with astro config set -g container.binary docker.