Microservices have become one of the pillars of the technological boom of the last decade, powering the apps of such giants as Amazon, Netflix and Uber. But there is a difference between simply building one’s app as a set of distributed services and adopting – from the very start – design approaches that unlock the true power of this architecture. And in this article, we are going to look at those cornerstone design principles and explain how they make our systems resilient and ready for growth.
When describing microservices, Google lists “easy scaling” as the number one advantage of such software architecture, further elaborating that one can “scale only the busy parts instead of the whole app”. But scaling even a single part of an application is not as easy as it might seem, and a service’s scalability is highly dependent on its design and topology, especially when relying on a cloud platform with its own constraints and peculiarities such as Amazon Web Services (AWS).
Simply put, merely switching from the monolith architecture to the microservice one does not guarantee improved scalability unless the services in question are built and fine-tuned to minimize bottlenecks and optimize performance and efficiency.
Design constraints
AWS documentation and microservice-specific architecture research articles can point one in the right direction, but it took me years of practice – such as working at AWS myself or building truck fleet management software – to figure out the pillars of good design which I would like to share with the community.
In essence, scalability in AWS emerges from deliberate constraints on coupling, state mobility, and failure propagation, with elasticity treated as a controlled system property rather than a reactive add-on.
What does this mean exactly? AWS reliability guidance explicitly warns that long synchronous chains increase brittleness because each dependency must succeed for end-to-end success, amplifying the probability of outages and latency variance; asynchronous communication patterns and failure-aware interaction design reduce systemic fragility by decoupling availability requirements across components.
A long synchronous request chain – for example, Service A waiting on B, which waits on C, which waits on D – makes a user request dependent on every downstream component being healthy and fast. The design response is to shorten synchronous chains and use asynchronous messaging where the business operation does not require an immediate answer.
MicroProf, a dynamic program analysis tool for optimizing microservice applications, attributes performance overhead to unnecessary data transfer, reinforcing that “chatty” microservice interfaces and redundant payload exchange create scalable inefficiencies that cannot be solved by autoscaling alone because scaling amplifies communication volume and cost.
If a platform relies on microservices, scaling requires a discipline of interface design: payload minimization, careful endpoint granularity, and deliberate use of patterns that control request volume and concurrency under stress. A deliberate design constraint must apply not only to the number of dependencies, but also to the volume and shape of communication: fewer unnecessary calls, smaller payloads, and controlled request rates.
For example, with Fleet Insights and Small Parcel Shipping projects at Wayfair, we used the Kafka messaging queue to buffer and normalize incoming events before any processing happened – which in turn was done asynchronously with event-driven patterns on cloud-based PostgreSQL. That absorbed traffic spikes without dropping data, resulting in doubled deliveries per route and a 4% revenue boost.
On the opposite side, we designed a distributed fleet of servers with load-balanced endpoints and automatic failover, which allowed us to scale to high event volumes while keeping latency consistent. And most importantly, we made the system asynchronous by separating read and write paths.
The table below summarizes those key points.
States and cells
As for state mobility, I strongly advise using stateless service tiers where feasible, because stateless components can be replicated, replaced, and distributed across zones without complex coordination; stateful components then become explicit elasticity bottlenecks that require controlled scaling strategies (partitioned state, careful failover, or managed services that provide multi-AZ durability and failover mechanisms).
In other words, state should be placed intentionally, kept out of rapidly changing compute layers when possible, and partitioned or managed so it does not prevent safe scale-out or recovery.
As for failure propagation, the AWS cell-based architecture paper presents the principle that scalable resilience is improved when workloads are decomposed into self-contained units that serve a subset of tenants or traffic, limiting the maximum affected population during a fault.
This defines failure-propagation control through bounded blast radius. Rather than one large shared system serving all customers through the same dependency path, the workload is divided into cells. A cell owns only a subset of tenants or traffic, so a failure is structurally prevented from affecting everyone.
Personally, I like to compare this approach to the one used in the design of modern ships and submarines: independent watertight compartments that keep the vessel afloat even if a section is flooded.
This partitioning simultaneously supports scale-out (replicating cells as demand grows) and resilience (constraining correlated failure to a bounded cell scope). AWS architectures treat Regions and Availability Zones (AZ) as isolation boundaries: multi-AZ deployment is not simply redundancy, but an explicit strategy for correlated-failure avoidance and capacity continuity.
Three planes, one goal
Scalable behavior in AWS follows from jointly managing three planes: isolation boundaries (Region/AZ/cell), interaction contracts (API topology and messaging semantics), and operational levers (change safety, observability, recovery actions). A system designer should treat these planes as mutually reinforcing rather than independent, since reliability targets and operability constraints shape the feasible scaling envelope.
Empirical microservice findings further justify this coupling: the same nominal “scale-out” decision can yield opposite outcomes depending on communication overhead, gateway positioning, and traffic/tenant partitioning, so scalability must be treated as a property of the complete architecture, not of an individual service or technology choice.
As a result, the presented principles converge on a consistent practical rule set: limit synchronous dependency depth, keep the elastic surface stateless where possible, partition workload ownership to bound correlated impact, and anchor scaling decisions in operationally testable procedures (pre-scaling, traffic shifting, controlled failover) that remain viable under impairment.
A truly scalable microservices platform is built around optimized payloads, endpoint granularity, and careful stress-testing and management of request volume and concurrency. At the infrastructure layer, multi-AZ placement and pre-scaling create a stable baseline that prevents capacity collapse during zone impairment.
At the architectural level, cell-based partitioning provides an explicit lever for constraining the population affected by faults, thereby improving the feasibility of aggressive horizontal growth, as each unit remains testable, observable, and recoverable within a bounded scope.
Built to scale
In practical terms, this means AWS scalability is not simply adding instances through autoscaling. It means deliberately designing limits on:
– Coupling – prevent services from forming long, chatty, all-or-nothing synchronous dependency chains;
– State mobility – make rapidly scalable compute stateless; explicitly partition, replicate, and protect stateful systems;
– Failure propagation – isolate infrastructure, workloads, tenants, and request paths so that a local fault stays local.
For example, instead of one globally shared synchronous order-processing chain, a resilient design might route customers to independent cells, run stateless API services across multiple Availability Zones, process non-immediate downstream work through queues, rate-limit overload, and keep order data in a partitioned, multi-AZ state store. That design constrains how far a traffic spike, slow dependency, or AZ failure can spread.
Bear in mind that end users will judge your system based on its worst, slowest response time during peak load, not on its average performance.
The table below lays out those key principles on the levels where they need to be applied.
In summary, microservices need to be designed with scale in mind from the very foundation – topology and architecture.
First, scalable distributed systems in AWS require engineering of failure domains (Region/AZ) and pre-planned survivability targets that treat impairment as a normal operating condition, not an exceptional case.
Second, interaction topology determines both performance and systemic fragility: limiting synchronous chains, governing APIs through validated patterns, and removing redundant data transfer directly improve scaling efficiency and reduce overload cascades.
Third, cell-based partitioning provides a structural method for simultaneous scale-out and blast-radius control, complementing multi-AZ deployment by constraining correlated impact at the workload level.
Like the houses of the proverbial three little pigs, the stability of a microservice depends on its building blocks and the way they are laid on top of each other and held together. Only a careful choice of those and the thorough alignment in all planes will protect an application from being blown away by the Big Bad Wolf of inevitable failures, demand spikes and scaling beyond the tested capacity.
Featured image credit
