Ep #56: How Microservices Talk — The Secrets of Discovery, Load Balancing & Consistent Hashing
Understanding Dynamic Service Location and Traffic Distribution for Modern Systems
👋 This post is part of my deep dive series on Mastering Microservices.
If you’re new here, don’t worry - each article stands on its own. But if you’d like the full journey, you can catch up on the earlier parts here:
Part 1: Traced the shift from monoliths → SOA → microservices, explained why organizations adopt them, and introduced fundamentals like DDD, service sizing, tech stack choices, and database-per-service.
Part 2: Explored service-to-service communication — REST, gRPC, and messaging systems like Kafka/RabbitMQ — with trade-offs, reliability, and resilience patterns such as retries and circuit breakers.
Part 3: Dive deep into Data Management-Strategies like Database per service, Polyglot Persistence and Distributed Data Consistency like Saga and Event Sourcing Patterns in Microservices.
Together, these first three parts give us clarity on why microservices exist, how they’re structured, and how they talk to each other - a perfect lead-in to Part 4, where we dive into how does microservices locate each other.
In a microservices architecture, services need to communicate with each other efficiently and reliably. Service Discovery and Load Balancing are critical for enabling this communication, ensuring that services can find each other and distribute workloads effectively.
1. Service Discovery in Microservices
Service Discovery is the process by which microservices locate each other in a dynamic environment where services can scale, fail, or move (e.g., due to container orchestration like Kubernetes). Since microservices are distributed and instances may have dynamic IP addresses or ports, a mechanism is needed to find and connect to the right service instance.
Why Service Discovery?
Dynamic Environments: In cloud-native systems, services are deployed in containers (e.g., Docker) and orchestrated by platforms like Kubernetes, where IP addresses change frequently.
Scalability: Services may scale up or down, adding or removing instances dynamically.
Resilience: If a service instance fails, clients need to discover healthy instances automatically.
Decoupling: Services should not hardcode endpoints (e.g., IP addresses) to avoid tight coupling.
Service Discovery Patterns
There are two main patterns for service discovery: Client-Side Discovery and Server-Side Discovery.
Client-Side Discovery
In Client-Side Discovery, the client (or calling microservice) is responsible for discovering the target service’s location. It queries a service registry, retrieves a list of available instances, and selects one (often with load balancing logic).
How It Works:
A service registry (e.g., Consul, Eureka, or ZooKeeper) maintains a list of active service instances.
Each service instance registers itself with the registry on startup (e.g., providing its IP, port, and health endpoint).
The client queries the registry to get a list of instances for the desired service.
The client applies load balancing logic (e.g., random selection, round-robin) to choose an instance and sends the request directly.
Client-Side Discovery Example: In an e-commerce system, the Order Service needs to call the Payment Service. The Order Service queries a registry like Consul to get the list of Payment Service instances (e.g., payment-service:8080, payment-service:8081), selects one, and sends a request to process a payment.
C# Example:
public class ServiceRegistry
{
private readonly Dictionary<string, List<string>> _services = new();
public void Register(string serviceName, string instance)
{
if (!_services.ContainsKey(serviceName))
_services[serviceName] = new List<string>();
_services[serviceName].Add(instance);
}
public string Discover(string serviceName)
{
var instances = _services.GetValueOrDefault(serviceName, new List<string>());
return instances.Any() ? instances[new Random().Next(instances.Count)] : null;
}
}Pros:
Simple to implement for small systems.
Clients have full control over load balancing and failover logic.
No additional network hop, as the client communicates directly with the service.
Cons:
Clients need discovery and load balancing logic, increasing complexity.
Tight coupling between client code and the registry.
Clients must handle instance health checks and retries.
Real-World Example: Netflix uses Eureka for client-side discovery. Their microservices (e.g., movie recommendation service) query Eureka to find instances of other services (e.g., user profile service) and use Netflix’s Ribbon library for client-side load balancing.
What I Realized While Writing The Architect’s Mindset
When I started writing The Architect’s Mindset, I thought it would be about systems — decisions, trade-offs, scalability.
But somewhere along the way, I realized:
Architecture is not just about designing systems — it’s about designing thinking.What makes an architect different isn’t how much they know — it’s how they see problems.
Where most people see features, an architect sees flows.
Where others see issues, an architect sees constraints to balance.
Where others ask “How can we fix this?”, an architect asks “What does this break if we fix it?”That shift in perspective is what changes everything.
While writing, I kept noticing a pattern:
Architects don’t rush toward solutions — they step back, zoom out, and observe.
They ask:What’s the real tension here?
What are we optimizing for?
What happens if this succeeds too well?
And only then do they begin to design.
The Architect’s Mindset is my attempt to capture that invisible layer — the psychology, the reasoning, the calm that lives behind every technical decision.
It’s not a book about diagrams.
It’s a book about how architects think when no one is watching.If you’ve ever wanted to understand the difference between knowing systems and thinking systemically, this book is for you.
📘 Read more here:
👉 The Complete Architects Collection (System Design Trade-Offs Vol 1–3 + The Architect’s Mindset + 30 Advanced System Design Interview Questions)
👉 The Architect’s Mindset – A Psychological Guide to Technical Leadership👉 The Architect’s Mindset - A Psychological Guide to Technical Leadership (Free Sample)
Already a Paid Member? Continue the article below: