Allow for AutoscalingFargateCluster services with no open ports#256
Allow for AutoscalingFargateCluster services with no open ports#256
AutoscalingFargateCluster services with no open ports#256Conversation
ryanjjung
changed the title
AutoscalingFargateCluster services with no open ports
| :type load_balancers: dict, optional | ||
|
|
||
| :param registries: A dict where the keys are names of services and the values are lists of Elastic Container | ||
| Registry ARN patterns. If your service depends upon an image stored in ECR, you should list their ARNs here. |
There was a problem hiding this comment.
I ran into an issue where we couldn't pull an image from ECR. I forgot to integrate this set of permissions.
| Designing Autoscaling Fargate Clusters | ||
| -------------------------------------- | ||
|
|
||
| The :py:class:`tb_pulumi.fargate.AutoscalingFargateCluster` class provides a one-stop pattern for deploying Fargate |
There was a problem hiding this comment.
Just a thought, should we differentiate between ECS and EKS Fargate deployments?
There was a problem hiding this comment.
Being that I haven't worked with the EKS half of that and it's not immediately useful, I'm going to say it's probably not worth the effort to figure out what the difference would be. When we start building K8s-backed systems, we can worry about modifying the code or building a new class for it.
There was a problem hiding this comment.
Sure, I see that. Just thinking ahead because ECS clsuter is to ECS Fargate as EKS cluster is to EKS Fargate, meaning a fully managed cluster of that type.
There was a problem hiding this comment.
Was also mostly thinking about documentation here, we would surely have to redo a bunch of this in the code at the point of EKS adoption.
There was a problem hiding this comment.
It looks like the EKS stuff comes with a different set of resources. Again, I haven't really dug into this super deep, but there is an EKS Cluster resource and an EKS Fargate Profile that seem related. There is probably a lot of overlap in the other resources (security groups and such), but the cluster looks like a different kind of resource in the API.
There was a problem hiding this comment.
Exactly, ECS Fargate and EKS Fargate are two different resources, so overloading Fargate to only mean ECS might cause an issue in the future. No blocker here, just a conversation.
There was a problem hiding this comment.
I had a weird brain worm around midnight last night and it occurred to me that you might have been asking me to rename the class to something ECS-specific like AutoscalingEcsFargateCluster to differentiate from some potential future class like AutoscalingEksFargateCluster. If that's the case, then I would make two points:
- This is deployed in the observability environments under this name, and so we would have to rebuild those clusters if we changed the class name.
- The term "autoscaling" refers to the services here. In a K8s environment (unless there's something special about EKS), autoscaling is moved to the Kubernetes manifests, so not really a part.
I think we would probably want to approach the EKS stuff as though it is something different and create a differently named class that better summarizes whatever features we put into it then.
| raise IndexError('You must provide at least one subnet.') | ||
|
|
||
| if cluster_name is None: | ||
| cluster_name = name |
There was a problem hiding this comment.
This allows us to have a cluster whose display name differs from the name Pulumi uses under the hood. In the Pulumi output, it's often handy to see an indicator of type in the name (I've been using afc for this class). But in the AWS console, the notion that it's a cluster is inherent in the context of the webpage you're looking at. There, "afc" is a mysterious acronym.
| for rule in sg_config.get('rules', {}).get('ingress', []): | ||
| rule['source_security_group_id'] = lb_sgs[lb_name].resources['sg'].id | ||
| # If there's a load balancer associated with it... | ||
| depends_on = [subnets[0]] |
There was a problem hiding this comment.
I split this code out a bit because determining the proper set of dependencies is a little more complex than a straight list.
| task_defs[service_name], | ||
| ] | ||
|
|
||
| service_sgs = [] |
There was a problem hiding this comment.
Similar pattern here, with both the dependencies and the potential list of service security groups.
2 participants
Description of the Change
AutoscalingFargateClusterclass to support running services whose tasks run without network ingress, such as a queue worker.Benefits
We now support a mode where we can run services without network ingress, necessary for us to run the accounts-celery task.
Applicable Issues
thunderbird/thunderbird-accounts#609