Apps need to wait to be provisioned until all the roles, private endpoints, etc of their references are provisioned #14482
Description
Today when an application is deployed, it can be provisioned before the resource it references is fully ready for it to be connected to. For example, if the role assignments aren't provisioned yet, or the private endpoints aren't provisioned yet, the app could be deployed to ACA but not work because the resources needed in order for it to connect to Azure services are not there yet.
In the example below, I'm updating my app with an Azure Key Vault. You can see the keyvault is provisioned first, then the provision-server-containerapp is started before the provision-server-roles-keyvault starts. This means the new version of the app could be ready before the role assignments are there to give it access.
Private endpoints make this problem even worse because they take longer to provision.
18:02:28 (provision-keyvault) ✓ Successfully provisioned keyvault (43.0s)
18:02:28 (provision-keyvault) ✓ provision-keyvault completed successfully
18:02:28 (provision-server-containerapp) → Starting provision-server-containerapp...
18:02:28 (provision-server-containerapp) → Deploying server-containerapp
18:02:28 (provision-server-roles-keyvault) → Starting provision-server-roles-keyvault...
18:02:28 (provision-server-roles-keyvault) → Deploying server-roles-keyvault
18:02:35 (provision-server-roles-keyvault) ✓ Successfully provisioned server-roles-keyvault (7.4s)
18:02:35 (provision-server-roles-keyvault) ✓ provision-server-roles-keyvault completed successfully
18:03:06 (provision-server-containerapp) ✓ Successfully provisioned server-containerapp (38.6s)
18:03:06 (provision-server-containerapp) ✓ provision-server-containerapp completed successfully
18:03:06 (print-server-summary) → Starting print-server-summary...
18:03:06 (print-server-summary) i [INF] Successfully deployed server to https://server.yellowsmoke-60a1f25c.westus3.azurecontainerapps.io
18:03:06 (print-server-summary) ✓ print-server-summary completed successfully
18:03:29 (provision-privatelink-vaultcore-azure-net) ✓ Successfully provisioned privatelink-vaultcore-azure-net (104.6s)
18:03:29 (provision-privatelink-vaultcore-azure-net) ✓ provision-privatelink-vaultcore-azure-net completed successfully
18:03:29 (provision-pe-subnet-keyvault-pe) → Starting provision-pe-subnet-keyvault-pe...
18:03:29 (provision-pe-subnet-keyvault-pe) → Deploying pe-subnet-keyvault-pe
18:04:41 (provision-pe-subnet-keyvault-pe) ✓ Successfully provisioned pe-subnet-keyvault-pe (71.3s)
18:04:41 (provision-pe-subnet-keyvault-pe) ✓ provision-pe-subnet-keyvault-pe completed successfully
We need to order the pipeline steps in such a way that all the role assignments and private endpoints need to be provisioned before the app can be provisioned.