Skip to content

Instantly share code, notes, and snippets.

@ranchodeluxe

ranchodeluxe/inter-application-comms-ecs-k8s.txt

Last active April 14, 2023 18:57
Show Gist options
  • Save ranchodeluxe/ccc5082de72f65616862a9d7e3566b8c to your computer and use it in GitHub Desktop.
Save ranchodeluxe/ccc5082de72f65616862a9d7e3566b8c to your computer and use it in GitHub Desktop.
Download ZIP
inter-application communication options in ECS versus K8s
Raw
inter-application-comms-ecs-k8s.txt
Inter-application communication/networking options in ECS versus K8s with an emphasis on avoiding setting up load balancers:
+--------------+----------------------------------------------+-----------------+-----------------+-------------+
| FEATURE | DESCRIPTION | AWS ECS | KUBERNETES | CONCLUSION |
+--------------+----------------------------------------------+-----------------+-----------------+-------------+
| | Multiple containers per TaskDef/Pod. | | | |
| | Supports inter-container communication | supports | supports | |
| **Sidecar | via loopback on `localhost:<port>`. This | inter-container | inter-container | |
| Networking** | also means port collisions have to be | comms | comms | |
| | avoided. At the TaskDef/Pod level this | | containers | using this |
| | is the "crudest" form auto-scaling b/c | per | per | now |
| | if your TaskDef/Pod scales then so do | TaskDefinition | Pod | and works |
| | all your containers. This isn't necessarily | via | | |
| | bad design but depends on what the "sidecar" | network mode | | |
| | is doing. Plenty of examples of this in the | "awsvpc" | | |
| | Golang and K8s community | | | |
+--------------+----------------------------------------------+-----------------+-----------------+-------------+
+---------------+-------------------------------------------------------+----------------------------------------------------------+---------------------------------+------------+
| FEATURE | DESCRIPTION | AWS ECS | KUBERNETES | CONCLUSION |
+---------------+-------------------------------------------------------+----------------------------------------------------------+---------------------------------+------------+
| | | | | |
| | | The network mode "awsvpc" docs make it "seem" | Every Pod gets assigned | |
| **TaskDef/Pod | TaskDef(s)/Pod(s) can communicate across the cluster. | like ECS supports this via ENI but | an internal IP address | have not |
| Level | Specifically this means communication across | it's not clear. It seems like TaskDef(s) get an internal | and any other Pod in the whole | tested |
| Networking** | "compute instances (nodes in K8s)" whether | DNS hostname. But the question is | cluster (across computer nodes) | this yet |
| | they part of the same service or not | if two ECS services are put on different | can talk to it this way | on ECS |
| | | compute instances (EC2s) can they communicate? | | |
| | | I think this is what Leo is doing in "SatSearch" | | |
+---------------+-------------------------------------------------------+----------------------------------------------------------+---------------------------------+------------+
+--------------+---------------------------------+--------------------------------+---------------------------------------------------+-------------------+
| FEATURE | DESCRIPTION | AWS ECS | KUBERNETES | CONCLUSION |
+--------------+---------------------------------+--------------------------------+---------------------------------------------------+-------------------+
| | | | | |
| | | | | |
| | | | K8s does a few interesting things | |
| | | This is not an option | here. K8 services are the "crudest" | |
| | | with network mode "awsvpc". | level of load balancing above "sidecar" | |
| **Service | Services across the cluster | A load balancer is needed. | b/c a K8 service can use a "selector" | don't think |
| Level | can talk directly to each other | In AWS there seems to be all | to group Pods beneath it and auto-scaling | i need to test |
| Networking** | without a load balancer | this other tooling trying | happens at the Pod level. | and I don't |
| | | to fill the "service | | want to play with |
| | | discovery" gap such as: | In terms of networking not only | all the AWS Cloud |
| | | AWS Cloud Map, Mesh stuff etc. | do services get internal | Map/Mesh whatever |
| | | | IP addresses but they also get | here |
| | | | internal DNS names | |
| | | | so any "service discovery" is | |
| | | | dumb and predictable (you might | |
| | | | argue you don't have "service discovery" | |
| | | | ). | |
| | | | | |
| | | | The pattern is: | |
| | | | `<service-name>.<namespace>.svc.cluster.local` | |
| | | | | |
+--------------+---------------------------------+--------------------------------+---------------------------------------------------+-------------------+
External Load-Balanced services are basically the same
References:
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-networking-awsvpc.html
https://www.redhat.com/sysadmin/cni-kubernetes
https://kubernetes.io/docs/tasks/job/job-with-pod-to-pod-communication/
https://aws.plainenglish.io/understanding-the-differences-in-aws-eks-vs-ecs-and-when-to-use-what-7ed4d9efa6dc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment