context

I'm deploying a set of services containerized using Docker into AWS. Regardless of the deployment solution chosen (like raw EC2/ECS/Elastic Beanstalk/Fargate), we're going to face the problem of "service discovery".

To name just a few service discovery options I've considered:

• AWS Route 53 Service Registry
• Governor
• Hach Corporation Consul
• Spring Cloud Netflix Eureka

Details of my stack

I am developing a Java Spring Boot application using Spring Cloud and the target deployment environment is AWS.

Given that my stack is Spring based, spring cloud eureka makes sense to me when developing locally. Setting up a single node is easy, integrates well with the stack and ecosystem of choice, and requires very little setup.

Locally, we use docker compose (not swarm) to deploy the service - one of the containers deployed is a single node Eureka service discovery server.

However, when we move beyond local development into staging or production, we are considering options like Kubernetes.

My assessment of pros/cons

AWS Route 53 Service Registry

Requires us to specifically couple our code to AWS services. Not a problem by itself, we're stuck with the rest of the stack (SNS/SQS) anyway.

Since it relies on Route 53, making it slightly harder to run the stack locally, I guess we could open up a certain hosted zone for local development.

AWS native, no managed service registry or other "moving parts".

Spring Cloud Eureka

The downside is that it requires us to deploy and manage a cluster of highly available service registries, and it requires more resources. Another "movement part" to manage.

The advantage is that it fits our stack very well (spring ecosystem, spring boot, spring cloud, feign and zuul work well with this). It can also be easily run locally.

I think we need to configure the network and registry areas to ensure that clients publish their host addresses and not the docker container internal IP addresses. For example, if service A is on host A and wants to communicate with service B on host B, service B needs to advertise its EC2 address, not some internal docker IP.

question

If we are using Kubernetes for orchestration, there is no downside to using something like Spring Cloud Eureka over the built-in service discovery options described here https://kubernetes.io/docs/concepts/services-networking/service/ #discovering-services

Given that Kube provides this, using eureka deployed with kube for discovery doesn't seem like the best option. I think kube could do some optimizations to affect the usability and stability possible using eureka. For example, kube will know when to deploy a new service - eureka will have to rely on heartbeats/health checks, and depending on how it's configured (eg frequency), this can lead to stale records, while I think kube is not good enough for planned services A shutdown/reboot might not be affected by this. I'm guessing it can still handle unplanned failures like host failures or network partitions.

Does anyone have any advice on this? Do people use services like Kubernetes, but use other mechanisms for service discovery than those provided by kube. Is there a good reason to do one or the other?

Possible challenges I anticipate

We could replace eureka, but relying on Kube to perform discovery would mean that we would need to run kube locally for deployment, and currently we only have a simple small docker-compose file. Also, I'll have to see how easy it is to make sure this feature makes the ribbon, zuul, and pretense go perfectly.

Currently, we have eureka client configured for ribbon so that service A can serve service B like "service-b" and make ribbon resolve normal host through eureka client. I guess we can configure Ribbon to not use eureka and use an external Kube service name that will be resolved by Kube DNS at runtime...

Final note

Thanks in advance for any contributions or suggestions. I know this might draw attention to opinions. However, I'm hoping someone can provide objective guidance on when one solution is better than the other.