Applied architecture stage summary

A microservice implements a single business capability, runs in its own process, and owns its own data store, and all three are required at the same time. Two services sharing a database are not independently deployable, because a schema change in one breaks the other, and a service that handles both authentication and order fulfilment has already given up the single-capability property. Martin Fowler and James Lewis named the style in a 2014 article, and the property that separates it from earlier service-oriented approaches is exactly this insistence on independent deployability and data ownership rather than a shared bus.

The spectrum runs monolith, modular monolith, microservices. A monolith deploys as one unit from one codebase over one database, and Shopify runs a commerce platform of enormous scale on a Rails monolith by holding modular discipline inside the single codebase. A modular monolith is one process with real internal boundaries where modules talk through explicit interfaces rather than reaching across domain lines, so it keeps the single deployment while removing most of the coupling that makes monoliths painful to change. Microservices sit at the far end and pay the full price of distribution: network failures, distributed transactions, and observability that has to be built rather than assumed.

The choice is therefore about the organisation, not the code. Sam Newman's threshold, that you must be this tall to use microservices, means a working CI/CD pipeline, tracing across services and a team that can operate distributed systems, all in place before decomposition begins. Netflix took four years and built Hystrix, Eureka, Zuul and Chaos Monkey along the way. The practical consequence is that when a team cannot name the specific pain point it is buying out of, independent release cadence across several teams, dramatically different scaling per component, or genuinely mandatory technology diversity, the honest answer is a modular monolith and an extraction plan for later.

Database per service is what makes independent deployment real, and it is also the hardest part to operate. An order placement that was three rows in one ACID transaction becomes three writes in three separate stores: the Order service creates the order, the Inventory service reserves the items, the Payment service takes the money. If payment fails after inventory has been reserved, nothing rolls it back for you, and the system is sitting in a state no single database ever allowed.

The saga pattern is the answer taught here. A saga is a chain of local transactions, each one publishing an event that triggers the next, and each one carrying a compensating transaction that undoes it. Inventory publishes ItemsReserved; Payment fails and publishes PaymentFailed; Inventory listens for that and runs ReleaseReservation. The system is genuinely inconsistent for the duration of the chain, and that is the deliberate trade. Two-phase commit would preserve immediate consistency but reintroduces distributed locks and coordination coupling, which is precisely the independence the decomposition was bought for.

The same ownership rule decides how one service reads another service's data. A second service that needs customer information calls the owning service's API or subscribes to its change events and keeps its own local read model. It does not query the other service's database and it does not set up replication of the other service's tables, because both create an implicit shared schema that no deployment pipeline can protect. Around all of this sits the operational layer the module treats as required rather than optional: service discovery so instances can find each other as they scale and restart, and a service mesh carrying mutual TLS, routing, retry policy and tracing as a sidecar so that services in different languages get identical policy without each one implementing it.

In a request-driven call, service A knows service B exists, knows its API, and inherits its availability: if B is slow A waits, if B is down A fails. In an event-driven design A publishes a fact, that an order was placed, and does not know who reads it. B subscribes and reacts. If B is slow, A is unaffected; if B is down, the events wait. The test of whether a system is genuinely event-driven is simple and unforgiving: when a new consumer is added six months after launch, how many changes does the producer make? The correct answer is none.

Three message types have to be kept apart. Domain events are past-tense facts inside one bounded context, such as OrderPlaced or StockLevelChanged, owned by the service that produced them. Integration events translate a domain fact for consumption outside that context, so an OrderPlaced domain event becomes something like OrderConfirmedForShipping for logistics, which stops the logistics vocabulary bleeding back into the order model. Commands are imperative and addressed to a named receiver, such as ProcessPayment, and they fail if that receiver refuses or is absent.

The distinction is not naming hygiene. A system built from commands wearing past-tense names still carries command coupling: the producer has a list of who must act, and adding a consumer means changing the producer. Past-tense naming is the cheapest available signal that a message is a fact rather than an instruction, so if a message name reads as a verb in the imperative, treat the coupling as request-driven no matter which broker it travels on.

Kafka is log-based. Events are appended to a partitioned, replicated commit log, each consumer group tracks its own offset, retention is configured in time rather than in deliveries, and several independent consumer groups can read the same events at their own pace. That is what makes replay possible: a service deployed today can read history from the beginning. RabbitMQ is queue-based. Messages go to queues, are deleted on acknowledgement, and the broker holds the delivery state, which makes it the natural fit for work distribution where one item must be handled by exactly one worker. AWS EventBridge is a managed serverless bus for routing events to targets inside a cloud-native estate. Support tickets that must be handled once go to a queue; an activity stream that four teams each need in full goes to a log.

Two Kafka mechanics decide whether the design survives contact with retries. Ordering is guaranteed within a partition and nowhere else, and the partition is chosen by hashing the message key, so using the order id as the key is what keeps every event for that order in sequence. Delivery is at-least-once by default, so consumers must be idempotent: a consumer that inserts a payment row per delivery will eventually create a duplicate payment, while one that upserts on the event id will not. Exactly-once semantics exist, requiring producer idempotence and transactional consumers, and most systems decline the latency and operational cost and rely on idempotent consumers instead.

Schemas then have to change without breaking anyone. Backward compatible means the new schema can read old data, which is what adding a field with a default gives you. Forward compatible means an old schema can read new data, which holds when consumers ignore fields they do not recognise. Renaming a field, changing its type or removing a required field is breaking, and the registry rejects it rather than letting it reach a topic that holds months of history. The working habit is to treat a topic as a published interface with a longer life than any consumer on it.

Bertrand Meyer stated the principle in 1988: a method should either change state and return nothing, or return data and change nothing, never both. Greg Young raised it to an architectural rule in 2010 as Command Query Responsibility Segregation. The command side handles state changes such as PlaceOrder or SubmitVATReturn and enforces the business rules; the query side answers reads and is shaped for the questions actually being asked. In its mildest form CQRS is two code paths in one service over one database, and even that improves clarity. The stronger form adds a separate read store.

A read model, also called a projection, is a denormalised structure precomputed for one query pattern and updated from the events the write side emits. A dashboard that would otherwise join several tables and compute aggregates on every request becomes a single stored document per user, and latency drops from hundreds of milliseconds to single digits. That is the whole benefit, and it is real when the read and write loads are genuinely asymmetric.

The cost is eventual consistency, and it lands in the user interface rather than in the database. Between the event being published and the projection being updated there is a window in which the read model is stale, so the user who just submitted something and refreshed immediately sees the old figure. That is designed behaviour, not a defect, and the design owes them a response to it: show a processing state, poll until the projection catches up, or route that user's own immediate reads to the write side. If nobody decides which, the support queue decides for you.

Conventional persistence stores current state and overwrites the previous value, so history exists only if somebody built an audit log beside it. Event sourcing stores the sequence of events that produced the state as an immutable, append-only log, and the current state is derived by replaying that stream: OrderCreated, ItemAdded three times, PaymentProcessed, OrderShipped. The log is the source of truth and the state is a projection of it, which is why the phrase events instead of state misstates the model. State is always there; it is just derived on demand.

Three capabilities follow that state-based storage cannot give you. Audit history is intrinsic rather than bolted on. Temporal queries become possible, because the state at a past moment is the replay up to that point. And projections can be rebuilt from scratch when a read model is corrupted or a new query pattern appears. An event store organises events into one stream per aggregate and appends with an expected version, so two writers racing to append the same position get one success and one concurrency conflict. Snapshots exist purely as a performance measure once streams grow long, and the log remains authoritative behind them. EventStoreDB is purpose-built for this, and a PostgreSQL table keyed by stream and position with optimistic locking is a legitimate alternative when the team already runs PostgreSQL.

The reason this matters more as a filter than as a technique is that both patterns are over-applied. Greg Young's own guidance is that CQRS suits a small minority of bounded contexts, and event sourcing a smaller minority still. Each adds real work: schema management for events, a snapshot strategy, projection rebuilds, and a learning curve. The question to ask before adopting either is whether full audit history, temporal queries or projection rebuilding is a stated business or regulatory requirement. For a content management system with no audit obligation, the answer is no and a conventional database is the correct design.

Eric Evans published Domain-Driven Design in 2003 after watching the same failure repeatedly: teams sharing a codebase used the same word for different things. Sales meant one thing by customer, finance another, support a third, and the single shared class grew fields for all three until every change for one team broke another. The diagnosis was that the problem was linguistic and the code had no boundary marking where each meaning applied. A bounded context is that boundary: inside it, a term has exactly one meaning, and the ubiquitous language it defines is used in speech, tests and code without translation. Where a method named processPayment could mean three things, settleTrade with a settlement date can be read and confirmed by a domain expert.

Contexts should follow team ownership, which is Conway's Law used deliberately rather than suffered. If the billing team owns the billing context, it owns the customer model inside it and no other team changes that model without agreement. A bounded context is not the same thing as a microservice: the context is a conceptual boundary drawn from the domain, the service is a deployment unit, and one context may be several services or several small contexts may share one. Identify the contexts first, then decide the deployment shape, because starting from services and fitting contexts to them reliably produces the wrong boundaries.

Inside a context, the aggregate is the consistency boundary. It is a cluster of objects with one root through which all external access passes, so the invariants, such as no line item with quantity zero and no items added to a confirmed order, are enforced in one place instead of being scattered through application services. Changes inside an aggregate are atomic; changes across aggregates are eventually consistent through domain events. Small aggregates are strongly preferable, because a large one becomes a contention hotspot: an Order aggregate that also holds the customer profile, the payment method and the shipping history will be locked by checkout, payment, fulfilment and profile edits at the same time. Size the aggregate by asking what must be true together in one transaction, not by which tables are related.

A context map shows every bounded context in the system and the relationship between each pair, and its value is that it forces those relationships to be chosen before code makes them permanent. A shared kernel means two contexts co-own a small part of the model and neither team changes it alone. A customer-supplier relationship means an upstream context publishes an interface that a downstream context consumes, with changes negotiated between the two teams. An anti-corruption layer means the downstream context translates the upstream model into its own at the boundary.

For service-based systems the anti-corruption layer is the one to reach for most often. When the ordering context publishes OrderConfirmed, the fulfilment context should not adopt the ordering context's data model; it should translate the event into its own FulfilmentRequest. The payoff is contained change: when ordering alters its event schema, only the translation layer moves and the fulfilment model is untouched. Without that layer, the upstream model quietly becomes the downstream model, and the boundary that the context map drew exists on the diagram and nowhere else.

Peter Deutsch and colleagues at Sun Microsystems wrote down the eight fallacies of distributed computing from 1994: the network is reliable, latency is zero, bandwidth is infinite, the network is secure, topology does not change, there is one administrator, transport cost is zero, the network is homogeneous. All eight are false, and the module singles out three as the most consequential for design: that the network is reliable, when packets are lost and connections drop; that latency is zero, when any call can become arbitrarily slow under load; and that topology does not change, when instances restart and move address as they scale in and out. A system built on them behaves perfectly in staging, where the network is local, and fails unpredictably in production.

The failure mode to understand is mechanical. When a downstream service becomes slow rather than dead, every call to it holds a thread waiting for a timeout that may be thirty seconds away. Fifty threads, fifty blocked calls, and the calling service now cannot serve requests that have nothing to do with the slow dependency. A circuit breaker, named by Michael Nygard in Release It! and made famous by Netflix Hystrix, watches the failure rate and opens when it crosses a threshold. Closed means calls pass and failures are counted; open means calls fail immediately with a fallback and no network call is attempted; half-open lets a single probe through after a timeout and closes on success or reopens on failure. What it really does is convert a slow failure into a fast one, which is what gives the threads back.

A bulkhead solves the other half. Named after the compartments of a ship's hull, it gives each downstream dependency its own resource pool, so a dependency that fills its pool with blocked calls cannot take the pools used for everything else. Separate connection pools, separate thread pools, or separate worker pools in an event loop all express the same idea, and each pool should be sized to the downstream service's capacity under normal load rather than to the caller's maximum concurrency. The two patterns answer different questions and a serious incident usually needs both: the breaker stops wasting time on a failing dependency, and the bulkhead stops that dependency's exhaustion spreading.

A network call with no timeout blocks forever when the far end never answers, and in a service chain that block propagates upwards: checkout waits on inventory, which waits on pricing, which has no timeout on its database query. Timeouts belong at each layer, and the outer one has to be longer than the inner one so that a retry has room to happen. The module's working ranges are a user-facing API in the low seconds, an internal service call from several hundred milliseconds to a couple of seconds, and a database query in the low hundreds of milliseconds.

Retries are for transient failures only, the network glitch that clears in milliseconds, and they need exponential backoff with jitter so that every client does not return at the same instant and produce a thundering herd against a service that was just recovering. The harder constraint is correctness: a non-idempotent operation must not be retried without an idempotency key. If the order was created and only the response was lost, a retry without a key creates a second order and a second charge, and no amount of backoff prevents that. The server records the key, recognises the duplicate and returns the original response.

Graceful degradation is the part that gets skipped. It means deciding, in advance, what a reduced answer looks like for every external dependency: popular items from cache when the recommendation service is unavailable, rather than an error page. Combined with chaos testing, where instances are terminated on purpose to prove the system keeps serving, the stance the stage argues for is that resilience is demonstrated continuously rather than assumed. An untested circuit breaker with library default thresholds is decoration, and the time to find that out is not during the incident.

Vertical scaling adds capacity to one machine and needs no application change, which makes it the correct first move more often than its reputation suggests; its limits are the hardware ceiling and the single point of failure. Horizontal scaling adds instances behind a load balancer, has no theoretical ceiling and survives the loss of any one instance, but it has a hard prerequisite: the application must be stateless. If a session lives in one server's memory, the load balancer cannot route the next request anywhere else, and the user appears logged out. All state has to be externalised, sessions to a store such as Redis, files to object storage, data to the database, so that any instance can serve any request.

Caching is the highest-impact single change for read-heavy systems and the most common source of quiet bugs, because a stale cache returns wrong data with no error at all. Cache-aside is the usual shape: check the cache, on a miss read through to the database, store with a time to live, return. It applies at several layers, a CDN at the edge for static assets and pages, a gateway for public idempotent GETs, and an application cache for query results. The invalidation strategy has to be designed with the cache and not after the first stale-content complaint: time to live expiry is simplest, write-through keeps the cache current at the cost of write overhead, and event-driven invalidation deletes the specific key the moment the underlying fact changes.

For the database, read replicas and sharding solve different bottlenecks at very different prices. Replicas take read load off the primary and cost you replication lag, which is fine for a catalogue and not fine for a user reading back their own just-completed write, which must go to the primary. Sharding partitions the data across primaries and is the only answer to a write-throughput or data-volume ceiling, but cross-shard queries become application code, a badly chosen partition key means migrating data to rebalance, and transactions across shards need distributed coordination. Instagram is the worked example: tens of millions of users on a three-person backend team using stateless application servers behind a load balancer, PostgreSQL read replicas, Redis and a CDN, with nothing invented and every pattern aimed at the bottleneck that was actually binding at the time.

Queue-based load levelling treats acceptance and processing as two independent design concerns. During a spike the endpoint accepts at the incoming rate and returns immediately, while workers consume at whatever rate they can sustain, and the queue holds the difference. Without one, a spike that doubles arrivals forces you either to double capacity instantly, which is rarely possible, or to drop requests. With one, the spike costs latency for the work submitted during the peak instead of failures.

Queue depth is the operational signal that matters, because a growing queue is the direct statement that workers are behind. Auto-scaling workers on queue depth is the standard cloud arrangement, and the threshold should come from the latency objective for that work rather than from a round number. The pattern fits asynchronous work where the caller does not need the answer in the same breath: image processing, report generation, email, exports. It does not fit work the user is waiting on in the current session, which needs a synchronous response or an explicit polling design.

The stage closes on the same discipline in a different form. Design for partition tolerance rather than assuming a network that never fails, because the test environment never partitions and production eventually does, and a system built for the happy path produces inconsistent results that nobody can explain afterwards. Across every pattern in the stage the rule holds: identify the bottleneck or the failure mode by measurement, then apply the pattern that answers it. A pattern applied to an unnamed problem adds complexity and solves nothing.