Event-driven foundations: choose semantics before services

Reading notesDownload original Markdown ↗

Begin with a familiar request: a customer places an order, receives a response, and expects the order to progress. If later work happens asynchronously, that response and the final result can disagree. This chapter gives you the vocabulary to explain that gap before choosing AWS services. You need basic familiarity with an HTTP request, a database write, and a worker; the messaging distinctions are developed below. Follow the order example through meaning, delivery, ordering, schema change, and measurement.

Evidence notation: C identifies a claim in the claim register, A a dated AWS source, F a foundational source, and CS a finding in the repository case study. The source index supplies the full source details. These labels are lookup aids, not facts to memorize.

The purpose is to make the service name the last part of an architecture answer. A senior design starts with the business invariant, accepted latency and loss, work ownership, replay need, ordering scope, overload policy, and recovery evidence. Only then should it map those requirements to a concrete service mode.

The controlled definitions live in canonical terminology, the high-risk wording in the claim register, and repository observations in the evidence map. “Documented” below is a source-backed semantic. “Inference” is an architecture conclusion from named premises, never an AWS guarantee.

The decision sequence

Before saying “EventBridge,” “SQS,” “Kinesis,” or “MSK,” answer these questions in order:

1. What fact is authoritative, and what invariant is synchronous? An order acceptance, reservation, execution, settlement obligation, ledger posting, portfolio view, and client notification are different facts.
2. Is the message a command, event, notification, document, or reply? The transport envelope does not decide the payload's business meaning (C37).
3. Who owns each unit of work? One worker, every interested consumer, or independently positioned consumers that may replay?
4. What can repeat or disappear? Name the producer acknowledgement, transport delivery, consumer commit, external effect, and reconciliation boundary separately.
5. What order matters? Name the key, causal/version rule, serialization point, and cross-key invariant. “Ordered” alone is incomplete.
6. How stale can each result become? Give normal and recovery lag budgets, not merely a throughput target.
7. What happens under overload? State admission, shedding, isolation, fairness, and backlog-drain policies before invoking autoscaling.
8. How will an old producer, a new consumer, and retained historical data coexist? That is the contract and replay-compatibility question.

That sequence prevents a common interview failure: choosing a familiar managed service and then retrofitting the correctness story around its marketing label.

Why create an asynchronous boundary?

Asynchrony moves waiting and failure out of a caller's stack. It does not remove either. The exchange is usually immediate coupling for delayed state, additional identities, and an operational recovery loop.

Reason	What it buys	What it costs / when the benefit fails
Temporal decoupling	Producer and consumer do not need to be available at the same instant; retained work can bridge a short consumer outage.	Completion becomes delayed and ambiguous. The caller needs an acceptance state, status model, deadline, cancellation semantics, and recovery source. Retention is finite.
Load leveling	A buffer converts a burst into work a bounded consumer fleet can drain.	A queue can hide overload until its oldest item violates the SLO. If sustained arrival rate is at least service rate, backlog has no finite drain time. Storage, duplicate work, and stale outcomes grow.
Independent scaling	Producers, consumers, and different projections can use distinct capacity and deployment models.	Every scaling dimension can move the bottleneck downstream. More consumer concurrency can throttle a database, exhaust connections, or violate a serialization requirement.
Failure isolation	A failed optional consumer need not fail the producer or unrelated consumers.	Isolation is bounded by shared capacity, contracts, and authority. Failure often becomes lag, a DLQ item, or divergent state rather than an immediate exception. Operators now own detection and reconciliation.
Latency hiding	A user can receive ACCEPTED after durable intent while slower work continues.	This improves response latency only by changing the product contract. It cannot truthfully return EXECUTED, SETTLED, or “funds available” before the authoritative transition completes.
Organizational autonomy	Teams can deploy consumers and read models independently behind stable contracts.	Coupling moves into schemas, event meaning, retention, permissions, SLOs, and incident ownership. Hidden subscribers make change impact and event cascades harder to see (C58/F29,F30).

Repository boundary: useful decoupling, missing completion evidence

Observed: the order handler writes a PENDING order and calls PutEvents; later handlers write the simulated trade, portfolio, and notification. Those separate calls create temporal decoupling, but the first database write and publication are not atomic (CS01).

Documented: a DynamoDB transaction can cover supported DynamoDB items in one account and Region, not EventBridge (C22/A20, retrieved 2026-08-22). A transactional outbox can commit business state and publication intent locally, but relay and consumption remain retryable (C38/A20,A21/F04; A20-A21 retrieved 2026-08-22).

Inference: asynchrony has converted a thrown downstream error into a potentially stranded order. The correct response is durable intent, explicit ACCEPTED status, outbox age metrics, idempotent consumers, and reconciliation—not the claim that the systems are “decoupled.” The same boundary can be a good design after its delayed-state contract and recovery loop are made explicit.

When not to add the boundary

Keep work synchronous, or inside one authoritative transaction, when the caller must know the invariant before proceeding: available-funds reservation, legal order-state transition, or ledger balance are examples. Asynchrony may publish the result afterward, but a stale portfolio projection must not approve the invariant (C46).

Also avoid an asynchronous network boundary when a cohesive module and one transaction already meet the workload. A queue between every function adds latency, contracts, retry states, and on-call surfaces without buying a real independence or overload boundary. For a strict tail-latency matching loop, a multi-hop general event path is normally a poor fit; use a measured, capacity-controlled sequencing design and keep asynchronous services around its control plane (CS12).

Pattern vocabulary: four different meanings of “event-driven”

The earlier decision was whether waiting can leave the request path. Now decide what crosses that boundary. A request to execute an order and a fact that execution happened can use similar JSON, yet they create different obligations. Read these patterns along three separate axes: message meaning, interaction between services, and where authoritative history is stored.

First separate payload semantics from interaction style and persistence model. These patterns can coexist; none implies a particular broker (C58/F07,F29,F30).

Message, command, event, and notification

• A message is the transport envelope. Delivery, acknowledgement, visibility, partition, and DLQ are properties of its channel.
• A command requests an action and is accepted or rejected: PlaceOrder, ReserveFunds, CancelOrder. It has an intended handler or authority.
• An event records a past fact: OrderAccepted, FundsReserved, TradeExecuted. It cannot be rejected after publication as though the fact had not happened; a consumer may reject or quarantine its processing.
• A notification announces a fact to a recipient. It may be a pointer or carry rich context. It is not intrinsically lossy: durability, acknowledgement, replay, and acceptable loss belong to the business contract and transport.

Calling a required action OrderPlaced does not turn a command into a fact. If the sender expects exactly one executor and treats non-execution as failure, say who owns the ExecuteOrder command. Fowler calls the misleading alternative a “passive-aggressive command” (C58/F29).

Event notification

The producer announces a past fact and does not prescribe a recipient's reaction. A minimal PortfolioChanged {portfolioId, version} can tell a client to refresh; a richer notification can avoid that read. The producer remains coupled to the event's meaning but not to a list of response calls.

Fit: optional reactions, cache invalidation hints, client status updates, and extensibility where zero consumers is valid.

Costs: a pointer creates synchronous read-back load and an availability dependency; rich payloads increase coupling, privacy exposure, and versioning work. Hidden event cascades make the end-to-end flow difficult to inspect.

Poor fit: a reservation or settlement step that must have one accountable owner and a completion deadline. Use a command or visible process manager; publish events about accepted/completed facts.

Event-carried state transfer

The event carries enough producer-owned state for a consumer to maintain the local subset it needs without a synchronous call back (C58/F29). A portfolio projector might receive execution ID, account ID, instrument, side, exact quantity/price, currency/scale, and authoritative aggregate version.

Fit: read-heavy projections, independent availability, low-latency local queries, and several consumers that need the same stable facts.

Costs: deliberate data replication, larger messages, privacy/retention scope, write amplification, stale copies, version/gap handling, bootstrap, and rebuild procedures. A “current-state” payload also cannot fill a missed intermediate transition when that transition matters.

Poor fit: large or sensitive records where most consumers need one field; rapidly changing data where copying it costs more than a bounded query; or any case where a stale copy would authorize a hard financial invariant.

Event collaboration

Components announce facts and independently react; the overall behavior emerges from subscriptions and event cascades rather than a central caller directing every step (C58/F30). It can use notifications, event-carried state, or both.

Fit: adding projections, analytics, compliance search, or optional notifications without changing the producer; domains that can own local state and tolerate a declared freshness window.

Costs: the workflow graph moves from source code into runtime configuration and contracts. Cascades can be cyclic or surprising. Partial failure, compensation, timeout, and completion ownership become harder to see. For a customer-visible settlement workflow, a process manager may provide a clearer state machine even when steps still exchange events.

Poor fit: a small, fixed sequence whose audit and timeout owners must be obvious, or a cross-component invariant incorrectly assumed to be ACID.

Event sourcing

An aggregate's authoritative state changes are stored as an append-oriented event sequence; current state is derived by replay, often with snapshots (C36,C58/F07,F29). Event sourcing is a persistence decision. It neither requires asynchronous processing nor follows from publishing integration events after a CRUD write.

Fit: domains where historical decisions, temporal queries, correction semantics, and alternative/rebuilt views justify event-store governance—for example, a carefully modelled order lifecycle or accounting domain.

Costs: stable event identity and ordering, optimistic concurrency, upcasters/schema evolution, snapshot policy, replay-safe consumers, deletion and privacy handling, projection rebuilds, and operational expertise. Historic events preserve past meaning; changing code does not rewrite what happened.

Poor fit: adopting it only to get an audit log, broker replay, or CQRS. An append-only technical event archive is not automatically the authoritative financial ledger (C47).

The repository through this vocabulary

Repository path	Observed	Bounded classification / inference
OrderPlaced to trade-executor	One named target is expected to simulate execution.	Inference: despite its past-tense name, it behaves command-like because order progress depends on one action. Make command ownership and acceptance explicit; then publish facts.
TradeExecuted to portfolio-updater	The payload includes trade/order/user, instrument, side, quantity, price, timestamp, and correlation ID.	Inference: an integration event with event-carried state sufficient for the current projector to calculate without reading the trade. It is not event sourcing, and it lacks exact money plus an authoritative sequence (CS07, CS10).
TradeExecuted to notification-sender	A separate consumer turns the fact into an SNS customer message.	Event collaboration plus a client notification. Whether missed delivery is acceptable must be stated; “notification” does not make it lossy.
EventBridge archive and DLQs	Transport/recovery paths with bounded behavior; the declared DLQs lack the required send permission (C57/A81, retrieved 2026-08-22).	Neither event sourcing nor a financial ledger. A DLQ is a holding area, not reconciliation (CS08).
Mutable portfolio row	Updated from integration events.	A projection, not authoritative holdings. Its optimistic lock is not an inbox or source sequence (CS06).

Channel semantics: work queue, router, and replayable log

These are semantic centers of gravity, not mutually exclusive product boxes. Concrete products combine traits; state retention, acknowledgement, ordering, fan-out, and retry separately (C42/F01,F02).

Question	Work queue / competing consumers	Publish-subscribe router	Replayable log
Primary purpose	Assign each work item to one worker in a competing group.	Route or replicate a publication to zero, one, or many independent subscriptions/targets.	Retain an ordered sequence so independently positioned consumers can read/re-read within retention.
Consumption state	Visibility/lease then acknowledgement/removal or expiry.	Each target/subscription has its own delivery boundary; the router may not retain a consumer position.	Each consumer/group tracks a position/checkpoint; one consumer advancing does not delete the shared record.
Parallelism unit	Consumers and, where used, message groups/lanes.	Targets/subscriptions plus the downstream channel chosen for each.	Partitions/shards and consumer-group membership.
Ordering	Product/mode-specific; often best effort or per message group.	Do not infer order from fan-out. Add sequence/version checks or an ordered downstream path when required.	Normally per partition/shard, not across the whole stream. Replay preserves only the documented scope and retained data.
Replay	Usually redrive of retained/unacknowledged/held work, not arbitrary independent history.	Often needs an archive or downstream durable channel; routing alone is not replay.	Core property within retention, but rebuild time, schema compatibility, and external side effects still constrain it.
Natural fit	Document generation, risk calculation jobs, notification delivery work, settlement file ingestion.	Domain integration events, independent notification/projection/compliance reactions.	Execution journals, market-data streams, audit feeds, rebuildable projections.
Main failure shape	Poison item, visibility expiry, duplicate work, queue growth, unfair tenant use.	Silent no-match, target-specific delivery failure, divergent consumers, hidden event cascade.	Hot partition, lagging consumer, replay storm, retention overrun, skew, checkpoint error.

Bounded AWS examples, not one-to-one mappings

• Competing work: SQS Standard is an at-least-once queue and can deliver duplicates or out of send order (C02; A09, retrieved 2026-08-22). With Lambda, batch processing/acknowledgement failure can cause records to appear again (A02, retrieved 2026-08-22). SQS FIFO adds strict order within a MessageGroupId and bounded producer-side deduplication, not one-time database or API effects (C03-C04; A10, retrieved 2026-08-22).
• Routing/fan-out: EventBridge buses evaluate events against rules and send matches to targets; SNS topics deliver publications to subscribers (C60; A07,A83, retrieved 2026-08-22). A target may itself be a queue. Neither “router” nor “topic” establishes replay, global order, or business completion.
• Ordered retained consumption: Kinesis Data Streams contains ordered shard sequences and maps partition keys to shards (C10,C42; A14, retrieved 2026-08-22). Kafka on MSK supplies partitioned-log and consumer-group patterns; Kafka transactions remain bounded to supported Kafka operations and do not atomically include an arbitrary external database or API (C29/A30/F28; A30 and F28 retrieved 2026-08-22).

A sound design often composes these: a router gives each consumer its own work queue for isolation; a retained log feeds several independent projections; a log consumer creates work-queue items for expensive parallel tasks. This is why “event bus versus queue versus stream” is not answerable until work ownership, replay, order, and overload have been stated.

Guarantees: delivery, processing, and business effect

Queues, routers, and logs describe how work moves. To decide whether the design is correct, follow one message beyond the channel into the database and any external call. For each success signal in the next table, ask which later action could still be missing after a crash. That is the boundary your identity, transaction, or reconciliation control must close.

Use three planes. Never let a guarantee jump from one plane to the next without an explicit mechanism.

Plane	The question	Evidence required
Delivery	How often can this named channel attempt to hand this message to this named receiver, and until what deadline/retention?	Producer result, channel identity, delivery acknowledgement/visibility/checkpoint, retry age/attempts, terminal destination.
Processing	How many handler attempts and commits can occur for one logical identity across timeout, crash, retry, and rebalance?	Stable event/command ID, transaction boundary, checkpoint timing, inbox/deduplication behavior, ambiguous-timeout handling.
Business effect	Can one logical input create more or fewer externally meaningful outcomes than intended?	Authoritative state transition, idempotency record committed with it where possible, external provider key/receipt, reconciliation and correction evidence.

Delivery vocabulary is boundary-scoped

• At-most-once delivery: the named boundary does not redeliver after its attempt, so loss or non-processing is possible. It says nothing about code that independently repeats an effect.
• At-least-once delivery: the boundary retries/redelivers until its success or terminal condition, so duplicates are possible. It does not promise that a valid consumer will eventually succeed before retention expires.
• Exactly once: admissible only after naming operation, scope, identity key, time window, transaction, and external side effects (C01/F01). Product terms such as FIFO send deduplication or workflow execution must be repeated with their documented boundary, never promoted to “the system is exactly once.”

The practical default is: at-least-once delivery plus an idempotent named business effect. Business idempotency means the same logical identity produces one intended transition and a compatible response; a broker ID or time-limited cache alone is insufficient (C40/F02,F04,F11).

A crash-window worksheet

For each consumer, enumerate these states:

1. message received, no authoritative commit;
2. authoritative commit succeeded, acknowledgement/checkpoint not recorded;
3. acknowledgement recorded, optional external effect pending;
4. external effect timed out ambiguously;
5. terminal retry or poison quarantine reached.

Then answer: what repeats, what durable identity exists, who retries, how a duplicate becomes a compatible no-op, and which reconciliation proves the outcome. If the database mutation and inbox record can be in one local transaction, use that boundary. If an external venue, email provider, or other store cannot join it, use its idempotency contract where available and retain a recoverable state machine; reconcile independently.

Repository guarantee audit

Observed: the order handler stores a cached 201 before publication, and a same-key retry can return it without evidence of valid routing (CS02). Three PutEvents callers ignore per-entry results (CS03). Trade creation, order update, and publication are separate operations; the duplicate-trade guard can block repair after a late publication failure (CS04, CS05).

Documented: PutEvents can return HTTP 200 with failed entries. Even HTTP 200, zero failed entries, and an EventId are only producer-API acknowledgement; AWS documents a nonexistent-bus case in which those signals accompany a dropped event (C08/A06,A82, retrieved 2026-08-22).

Inference: no existing producer, channel, or conditional write establishes one end-to-end effect. A defendable design transacts state plus outbox, validates routing configuration independently, retries under stable IDs, transacts consumer inbox plus mutation, and reconciles orders, executions, ledger postings, and projections.

Ordering, concurrency, backpressure, overload, and fairness

Order is a business relation, not arrival time

Name at least four orders:

1. source order: the authority assigns aggregate version 41 after 40;
2. transport order: a channel returns records in a documented key/partition scope;
3. processing order: concurrency and retries determine which handler starts or completes first;
4. commit/effect order: database transactions and external systems decide which state becomes visible first.

Transport order alone does not prove causal business order. A consumer needs an expected version rule: lower is duplicate/stale, equal/next is applicable, and higher reveals a gap. Ordering does not remove duplicates, and deduplication does not establish causal/version order (C41).

Per-key serialization trades throughput for a smaller reasoning boundary. One partition or writer can create order for an account, order, or instrument; additional partitions increase parallelism but require coordination or reconciliation where invariants span keys (C34/F01,F09,F24). Global serialization is possible only by accepting a common bottleneck or equivalent coordination—not by declaring that global order is impossible.

Head-of-line blocking and skew

Strict order means a slow or poison item can stop later work in its lane. If all orders share one FIFO group or partition, one failure creates global head-of-line blocking. If groups are too fine, related transitions can race.

A good key preserves the narrowest necessary invariant:

• orderId for one order state machine;
• accountId where positions/cash within one account require sequence;
• instrument or order-book partition for matcher price-time sequencing;
• a workflow ID for settlement steps.

Measure the distribution, not only cardinality. A popular symbol, institutional account, or noisy tenant can create a hot partition even with millions of keys. Salting a key increases parallelism but destroys straightforward order, so it needs an explicit merge/coordination rule.

Backpressure and lag budgets

Let average arrival rate be λ, sustainable service rate be μ, and backlog be B. If μ <= λ, the backlog cannot drain. When μ > λ, the idealized minimum drain time is:

drain time = B / (μ - λ)

Real recovery is slower because retries, skew, downstream throttles, deployment, and poison records consume capacity. A lag budget therefore includes:

• maximum normal oldest-item age and per-consumer freshness;
• peak/outage duration to absorb;
• spare capacity and maximum recovery time;
• retention margin beyond the recovery time;
• priority/tenant-specific budgets where aggregate lag hides starvation;
• an admission or shedding action before the budget is exhausted.

Backpressure is a design decision: bound concurrency/backlog, defer or shed lower-value work, protect downstream capacity, and monitor age/lag. Autoscaling alone can amplify overload or move it to a hard downstream limit (C44/F03,F13,F14). Retries need a deadline, bounded attempts, exponential backoff, jitter, and one clear owner where possible; retries at every layer multiply load (C45/F03,F11,F12).

Fairness is separate from throughput and FIFO

A system can meet aggregate throughput while a quiet retail account waits behind one institutional tenant or while low-priority reconciliation starves forever. State the scheduling policy: FIFO, weighted fair share, per-tenant quota, reserved lanes, strict priority with aging, or isolated capacity. Measure oldest age and rejection rate per class, not just total depth (C61/F03,F13,F14).

Possible controls include:

• per-tenant or per-priority queues/partitions with bounded worker shares;
• reserved concurrency/capacity for critical order and reconciliation paths;
• producer quotas and admission control before shared storage is saturated;
• maximum work age, retry budgets, and poison quarantine;
• paced replay so recovery traffic cannot starve live traffic;
• load shedding for replaceable notifications or stale derived work when the business contract permits it—never silently for accepted orders or ledger postings.

Overload review questions

1. Which downstream resource saturates first: compute, database partitions, connections, external API quota, or operator capacity?
2. Is the buffer sized for a burst or being used to deny sustained overload?
3. What is the oldest age by key/tenant/priority, and when is new work rejected?
4. Can a poison item block a FIFO lane? How is it quarantined without violating order?
5. How long will a replay take, and what live-work capacity remains?
6. Which accepted work may never be shed, and which notification/projection can be regenerated under an explicit contract?

In the repository, the portfolio applies arrival order without a source sequence. Inference: retries and concurrency can produce nondeterministic positions; adding an ordered transport can simplify delivery but does not replace inbox idempotency, expected versions, gap repair, and reconciliation (CS06, CS07).

Event contracts and schema evolution

Stable identity and source order need a durable representation. The envelope below carries those decisions between independently deployed services and through replay. Treat a field's unit and business meaning as part of its contract: a payload can still parse successfully after a change that makes its financial interpretation wrong.

An event contract is not merely a TypeScript interface or JSON schema. It binds identity, business meaning, units, authority, compatibility, privacy, and operational behavior. Compatibility is a relationship between writer and reader versions, including retained records replayed into new code (C59/F01,F25,F26).

A useful envelope

Keep transport metadata distinct from domain payload. A typical envelope has:

Field	Purpose / rule
eventId	Stable identity for this fact across retries and republish. Do not regenerate it during redrive.
eventType	Stable past-tense semantic name such as TradeExecuted; routing name is not a command disguise.
source	Authority/producer namespace, not merely the current function name.
subject / aggregate ID	The business entity to which the fact applies.
schemaVersion	Schema contract version; it does not replace compatibility policy.
aggregateVersion / sequence	Source-assigned order within the named aggregate/key; not a timestamp substitute.
occurredAt	When the authoritative fact occurred. Separate from publish/receive/process timestamps.
correlationId	Groups a customer request or business journey across messages. It is not a deduplication key.
causationId	Identifies the command/event that directly caused this event; useful for cascade diagnosis.
traceContext	Technical distributed-trace propagation. Sampling must not erase business correlation.
data	Versioned domain payload with exact units, currency/scale, nullable/optional meaning, and privacy class.

CloudEvents standardizes common event metadata; AsyncAPI can describe channels, messages, bindings, and schemas. Neither standard chooses domain authority, guarantees semantic compatibility, or makes effects exactly once (C37,C59; F25,F26).

Compatibility rules

Evaluate both directions explicitly:

• Backward compatibility: a new reader can understand data written by the old writer, including retained history.
• Forward compatibility: an old reader can tolerate data from a new writer during rolling deployment or independent team release.

Practical rules:

1. Adding an optional field can support mixed versions only when absence has a defined meaning/default and old consumers tolerate unknown fields.
2. Removing or renaming a required field, changing type, or narrowing allowed values is usually breaking for a deployed reader.
3. Reusing a field with a new unit, currency, sign, rounding rule, identity, or business meaning is breaking even if the JSON type is unchanged.
4. Do not mutate a past fact into a new interpretation. Introduce a new event type/version, dual-publish or translate at a controlled boundary, observe old consumer use, then retire according to policy.
5. Validate envelopes and domain invariants at ingress. Quarantine poison data with the original bytes, identity, schema version, error, and ownership.
6. Test old-writer/new-reader, new-writer/old-reader, and replay fixtures from each retained version. A registry check alone cannot test business meaning.
7. Minimize sensitive event-carried state. Encryption does not remove deletion, access, purpose, retention, or non-production replay obligations.

Consumer-driven risk

Independent consumers are easy to add and easy to forget. Maintain an owner and compatibility window for each event type, discover actual subscriptions, and record which fields consumers rely on. Consumer-driven contract tests can catch known syntactic dependencies, but a producer still owns coherent domain meaning and cannot outsource it to every downstream implementation.

The repository envelope has a useful correlationId, and TradeExecuted contains a stable tradeId. Observed: it lacks causation ID, schema version, authoritative aggregate sequence, currency, and scale. Inference: these gaps prevent deterministic projection ordering and safe monetary evolution; adding fields still requires mixed-version and replay policy. The current TypeScript interface is evidence of code shape, not proof that deployed producers and consumers are compatible.

Fintech pattern decision table

The rows deliberately describe different business cases rather than assigning one service to one pattern.

Case	Authority and invariant	Needed interaction/channel semantics	Failure and recovery design	Poor fit / trap
Order acceptance	Command authority owns request identity, validation, reservation policy, and legal ACCEPTED transition.	Synchronous durable acceptance may enqueue/append publication intent. One command owner; later facts can fan out.	Atomically protect command identity, order, and outbox where local; expose status; reconcile accepted orders to executions/cancels.	Returning EXECUTED because a message was accepted; styling a required executor command as an unowned event.
Trade execution / fill	Matcher or venue adapter owns stable execution ID, per-book/order sequence, partial-fill and correction semantics.	Capacity-controlled single writer per ordering scope plus retained journal/integration facts; independent consumers may replay.	Idempotent execution ingestion, conditional order version, outbox, deterministic recovery and venue reconciliation.	General fan-out bus as price-time sequencer; global order claim; retry that generates a new trade ID or price.
Portfolio projection	Execution/ledger authority, not the portfolio row, decides ownership and cost inputs.	Event-carried state plus independent consumer position/replay; per-account/aggregate expected version.	Transactional inbox + projection mutation, stale/duplicate/gap paths, freshness SLO, rebuild and control totals.	Using stale projection for pre-trade availability; optimistic locking as duplicate defense.
Settlement	Settlement process and ledger authorities own obligations, deadlines, cash/securities postings, and irreversible external steps.	Commands with accountable owners plus events for facts; often a visible process manager/saga state rather than an invisible cascade.	Durable step state, bounded retries, compensation where meaningful, manual exception queue, statement/ledger reconciliation.	Treating compensation as rollback or assuming pub/sub fan-out proves every party completed.
Client notification	The underlying order/execution/settlement authority owns the fact; notification state owns its own recipient contract.	Event notification, then per-channel work queues/routing as needed. Loss/retry/deduplication follow channel and product contract.	Stable notification intent, preference/consent, provider receipt where available, bounded retries, suppression, audit and support status.	Making email/SMS proof of execution; assuming notification means best-effort; retrying without recipient-level deduplication.
Compliance/audit	Ledger/execution/order authorities own facts; audit evidence has governed retention and integrity.	Retained append-oriented records and independently replayable projections/search.	Completeness control totals, immutable/integrity-validated evidence, access/retention policy, replay drills and discrepancy resolution.	Treating an event-bus archive, DLQ, mutable balance, or incomplete daily scan as a financial ledger.

The repository demonstrates every trap in compressed form: non-atomic order and trade publication (CS01, CS04-CS05), response-cache idempotency (CS02), ignored producer results (CS03), unsequenced/non-idempotent projection (CS06-CS07), unusable DLQ paths (CS08), incomplete snapshot (CS09), floating-point money (CS10), no apparent ledger/reconciliation authority (CS11), and an unjustified Lambda matcher (CS12). The full evidence and remedies remain in the repository case study; later chapters should reference those IDs rather than restating observations as service guarantees.

Metrics that reveal the real failure mode

One dashboard should not collapse the planes.

Plane	Minimum signals	What a healthy aggregate can hide
Producer	accepted commands, rejected/limited commands, publish API failures, per-entry failures, outbox count/oldest age	An HTTP 200 with failed entries or invalid routing; accepted state with no publication intent.
Transport	ingress/egress rate, oldest message/record age, backlog/lag, retries, throttles, no-match/drop, DLQ writes and failed DLQ writes	One hot key, tenant starvation, retention expiry, target-specific failure.
Consumer	attempts, success/error/timeout, duration, concurrency, batch reprocessing, poison/schema failures, checkpoint progress	A handler success after a duplicate effect; one partition stopped while others advance.
Ordering/contract	duplicate IDs, stale versions, sequence gaps and gap age, unknown versions/types, compatibility/quarantine counts	Silent semantic misinterpretation of unchanged JSON fields.
Business correctness	accepted-without-execution age, duplicate fills, execution-without-posting, ledger imbalance, reservation mismatch, settlement breaks, projection discrepancy/freshness	Green availability and empty DLQs alongside incorrect money or holdings (C48).
Recovery	replay/redrive rate, estimated drain time, live-versus-recovery capacity, records classified, reconciliation completion and unresolved aging	An empty queue after destructive or incomplete replay.
Fairness	wait/oldest age, admission/rejection, throughput and concurrency by tenant/key/priority	Aggregate p50/p95 and total throughput while a quiet class starves (C61).

Alert on user/business risk, not merely resource use. Queue depth without oldest age is ambiguous; zero DLQ depth is unsafe if the channel lacks permission to write it. In this repository, EventBridge's InvocationsFailedToBeSentToDLQ must be alarmed separately because the declared queues lack required resource policies (C57/A81, retrieved 2026-08-22; CS08).

Interview follow-ups

1. What changes if the API must return accepted in 100 ms but reservation must be complete before acceptance?
2. Why does reversing “database then publish” to “publish then database” only move the dual-write gap?
3. Can an SQS FIFO message group make a portfolio update exactly once? Name the send, delivery, consumer transaction, and external-effect boundaries.
4. If two fills for one account arrive out of order, which source sequence lets the projector distinguish duplicate, stale, next, and gap?
5. When would event-carried state transfer be better than event notification, and what privacy/rebuild costs does it add?
6. Is a Kinesis or Kafka log an event store? Under what authority, retention, schema, and replay conditions could it become one?
7. How do you prevent a replay of historic executions from sending customer notifications twice?
8. A queue grows for 20 minutes and then input stops. What variables determine whether it drains before retention or the business deadline?
9. How can aggregate throughput be healthy while retail orders are unfairly delayed?
10. Which schema changes are syntactically compatible but financially breaking? Consider cents-to-micros, currency, sign, and partial-fill semantics.
11. When should a settlement flow use choreography, and when should it expose a process manager?
12. Which metric proves a trade is financially complete? Why are function success, an empty queue, and a sent notification insufficient?

Concise senior articulation

I start with authority and invariants, then choose the asynchronous boundary. A queue assigns work, a router fans out or filters, and a log retains ordered partitions for independent replay, but managed services can combine those traits. I name ordering by key and guarantee by boundary: producer acknowledgement, transport delivery, consumer commit, and business effect are separate. At-least-once delivery normally means stable identities, transactional inbox/outbox where possible, version checks, and reconciliation; FIFO or broker deduplication does not make external effects exactly once. I design overload explicitly with lag budgets, admission control, fairness, and replay capacity. Finally I version both syntax and meaning, test mixed producers/consumers plus retained-event replay, and keep authoritative reservations, executions, and ledger postings separate from stale portfolios, notifications, and analytics. Only after those decisions do I choose the AWS service and mode.

Vocabulary contract for later chapters

Later chapters use the following without redefining them:

• command = requested action; event = past fact; message = envelope;
• notification loss/durability = explicit transport and business contract, not an intrinsic property of the word;
• event-carried state transfer, event collaboration, and event sourcing = separate dimensions;
• queue = competing work, router = independent routing/fan-out, log = retained independently positioned replay, subject to concrete service semantics;
• delivery, processing, and business-effect guarantees = separate boundaries;
• order = named key/scope plus source version and failure behavior;
• backpressure = admission/capacity policy; lag budget = normal plus recovery freshness; fairness = per-class isolation/scheduling policy;
• replay = controlled reprocessing; reconciliation = independent proof and discrepancy resolution;
• authoritative state, ledger, and projection = explicitly scoped roles.

The canonical forms and prohibited shorthand are in terminology. Service chapters may add product-specific terms, but must not broaden these guarantees.

Sources used in this chapter

Only the sources below support this chapter. Claim wording and retrieval status are controlled in the claim register; full metadata is in the source ledger.

Conceptual and pattern sources

• F01 — Kleppmann and Riccomini, Designing Data-Intensive Applications, 2nd ed., “Encoding and Evolution” and “Stream Processing.”
• F02 — Hohpe and Woolf, Enterprise Integration Patterns, message channels, competing consumers, publish-subscribe, idempotent receiver, and dead letter.
• F03 — Nygard, Release It!, 2nd ed., stability antipatterns and patterns.
• F04 — Richardson, Transactional Outbox.
• F08-F09 — Fowler, Accounting Patterns and The LMAX Architecture, for scoped ledger/authority and single-writer reasoning.
• F07, F29, F30 — Fowler, Event Sourcing, What do you mean by “Event-Driven”?, and Event Collaboration.
• F11-F13 — Amazon Builders' Library, Making retries safe with idempotent APIs, Timeouts, retries, and backoff with jitter, and Avoiding insurmountable queue backlogs.
• F14 — Google, Site Reliability Engineering, for overload and SLO framing.
• F15-F16 — Google, The Site Reliability Workbook and Service Level Objectives.
• F17-F18 — Harris, Trading and Exchanges, and FIX Trading Community, FIXimate FIX.Latest top-level specification (F18, retrieved 2026-08-24), for trading-domain authority boundaries.
• F24 — Elhemali et al., Amazon DynamoDB: A Scalable, Predictably Performant, and Fully Managed NoSQL Database Service, for partitioning/concurrency trade-offs.
• F25-F26 — CloudEvents specification and AsyncAPI specification.
• F28 — Apache Kafka, message delivery semantics.

Current AWS examples (retrieved 2026-08-22)

• A02, A09-A10 — Lambda with SQS, SQS Standard queues, and SQS FIFO key terms.
• A12, A28 — SNS FIFO ordering and deduplication and S3 event-notification considerations, used only in the cross-source ordering-versus-deduplication boundary (C41).
• A06-A07, A81-A82 — EventBridge PutEvents API, event bus concepts, target DLQs, and sending with PutEvents.
• A14 — Kinesis Data Streams terminology and concepts.
• A20-A21 — DynamoDB transactions and Streams change data capture.
• A24 — DynamoDB global-table consistency modes, used only through the controlled authority/projection claim.
• A26 — Step Functions error handling, used only through the bounded retry claim.
• A30 — Amazon MSK Developer Guide.
• A32-A35 — OpenSearch Service, DAX, ElastiCache, and Aurora replication, used only to bound derived read stores from authoritative financial state.
• A41, A43 — CloudTrail log-file integrity validation and the AWS Well-Architected Reliability pillar.
• A83 — What is Amazon SNS?.

Repository facts come only from the exact local routes in the evidence map. No local note or repository comment is treated as authority for service behavior.

Takeaway and next step

You should now be able to distinguish a requested action, a recorded fact, a transport attempt, and a committed business effect. Carry one order example forward: identify its authority, ordering key, duplicate policy, acceptable lag, and recovery source. Next, use those requirements to select a composition in AWS messaging and compute decisions.