Architecture Event-Driven Integration Opinion

The printer analogy — why this isn't event-driven architecture.

Calling something event-driven doesn't make it event-driven. When the architecture forces a system to send one million codes as one hundred batches of ten thousand, and receive one hundred responses back — that's not eventing. That's a for loop with a distributed systems wrapper. Here's why, and what it should look like instead.

Consult Code Create · ~8 min read

Nobody sets out to build a bad architecture. The decisions that lead to it are usually made under time pressure, by people who know the platform well but haven't stress-tested the approach at the volume it will actually run at.

The scenario I want to describe is one I've seen in production. A system needs to process one million codes. The architecture uses events as its primary message distribution mechanism. The receiving system has a limit of ten thousand records per request. The pods running the workload are single-threaded. Parallelism is achieved by scaling the number of requests, not by designing the architecture to be inherently parallel.

The result is an architecture that looks event-driven on a diagram — and technically is firing events — but forces the calling system to manage one hundred outbound messages and read back one hundred responses. The caller owns the complexity of a problem that belongs inside the receiving system's boundary.

The printer analogy explains exactly why this is the wrong way to think about it.

The analogy

The printer you'd never buy.

Imagine you need to print a one million page document. You wouldn't expect to sit at your desk and manually click print one hundred times — requesting ten thousand pages at a time — and then wait for each batch to confirm before sending the next.

You'd select the document, click print, and walk away. The printer handles the complexity internally. It manages the feed rate, the buffer, the order. You fire and forget. The document arrives at the output tray when it's ready.

The architecture described in this post

The calling system breaks one million codes into one hundred events of ten thousand. It fires all one hundred outbound messages — no waiting for confirmation between them. But the calling system is still responsible for constructing and dispatching all one hundred events. The receiving system processes each batch and posts back a response message for each — one hundred messages out. The calling system then has to consume one hundred response messages to reconstruct the full result. That is 400 message operations in total: 100 sent by the caller, 100 received by the processor, 100 sent back as responses, 100 consumed by the caller. Each with its own network hop, its own log entry, its own infrastructure cost. For what is logically a single request.

The reality

What this architecture is actually doing.

The limit of ten thousand records per request is a constraint of the receiving system. That constraint is real and it has to be respected. The question is: where should the complexity of managing that constraint live?

In this architecture, it lives in the calling system. The caller is responsible for knowing the limit, slicing the data accordingly, managing the sequence, sending each batch, and reading back each response. The calling system is doing the work that the infrastructure should be doing.

Here's what one million codes costs in network operations under this architecture:

100 Outbound events sent

100 Received & processed

100 Response messages sent

100 Responses consumed

That is 400 message operations for one logical request — before a single line of logging, before network latency on each hop, before the infrastructure cost of processing each message through the event pipeline. Every one of those 400 operations generates a log entry. Every log entry is storage, query overhead, and noise in your observability tooling.

Current architecture — what actually happens

Calling system has 1,000,000 codes to process

Calling system slices into 100 batches of 10,000 — the receiving system's limit

Sends 100 event messages of 10,000 codes each — no waiting, but still 100 separate outbound operations the caller is responsible for dispatching

Receiving system processes each batch and posts back 100 response messages

Calling system reads 100 response messages and reassembles the result

400 message operations. The calling system owns the complexity. Tight coupling to the receiving system's internal limits. Not event-driven.

The hidden cost

400 messages. 400 log entries. 400 network hops.

The message count is the headline number but the operational overhead compounds well beyond it. Every message operation in an event pipeline generates a log entry. Every log entry is storage, indexing overhead, and noise in your observability tooling. At 400 operations per logical request, a moderately busy system generates logging volume that obscures the signals that actually matter.

The overhead breakdown — per logical request.

What 400 operations actually costs in practice

100 outbound events dispatched by the calling system — 100 network calls, 100 log entries on the sending side, 100 entries in the event pipeline's own audit log.
100 inbound messages received and processed by the receiving system — 100 network acknowledgements, 100 processing log entries, 100 entries tracking batch state.
100 response messages sent back by the receiving system — another 100 network calls, 100 log entries on dispatch, 100 pipeline entries.
100 response reads consumed by the calling system — 100 acknowledgements, 100 log entries, 100 state updates as the caller tracks which responses have arrived.
Every one of those 400 operations carries network latency, serialisation overhead, and infrastructure cost. Multiply by concurrent requests and the noise floor rises fast — legitimate failures become hard to find.

The observability problem

When something goes wrong — and it will — you are debugging a failure somewhere across 400 operations spanning four systems. Finding the one message that failed, the one batch that didn't process, the one response that never arrived means querying logs across four hops with up to 100 entries each. In a well-designed async job pattern, the failure is attached to a single job ID. One query. One answer.

The compounding problem

Why using events here makes it worse, not better.

Events are the right tool for broadcasting state changes to interested subscribers. Something happened — a record was created, a payment was received, a status changed. Multiple consumers can react independently. The producer doesn't know or care who is listening.

That is not what is happening here. The calling system is not broadcasting a state change. It is making a request and waiting for a response. That is a request/response pattern — and trying to implement it using events introduces all the complexity of asynchronous messaging with none of the benefits, because the calling system is blocking on the result anyway.

Single-threaded pods create a false ceiling.

When the processing runs on single-threaded pods, parallelism can only be achieved by adding more pods — not by making the architecture inherently concurrent. The ten thousand record limit per request then becomes a hard constraint the calling system has to manage manually, rather than an internal concern of the processing layer.

The calling system knows too much.

The caller should not need to know that the receiving system has a ten thousand record limit. That is an implementation detail of the receiving system. When the limit changes — and it will — every caller has to be updated. This is tight coupling dressed up as event architecture.

Ordering and reassembly become the caller's problem.

When one hundred response messages come back asynchronously, the calling system has to track which responses correspond to which requests, handle partial failures, and reassemble the full result in the right order. This logic is complex, error-prone, and entirely unnecessary if the architecture is designed correctly.

Failure modes multiply.

With one hundred outbound requests and one hundred inbound responses, there are two hundred opportunities for something to fail mid-process. Handling partial failure — what happens if batch 47 times out? Does the caller retry from 47? Does it start over? — requires complex state management that wouldn't exist in a properly designed async pattern.

The fix

One message. One job. One response.

The fix is conceptually simple. The calling system should not be responsible for managing the receiving system's internal limits. It should send one message describing what it needs, receive a job identifier back, and wait for the result to be delivered asynchronously when it is ready.

The receiving system handles the chunking internally. It knows its own limits. It manages the parallelism. It assembles the result and delivers it back — in chunks if necessary, but as a coherent response to a single logical request, not as one hundred separate messages the caller has to reassemble.

Correct async job pattern — what it should look like

Calling system sends one message — "process these 1,000,000 codes" — and receives a job ID back immediately

Calling system is free to continue — it does not wait, poll, or manage batches

Receiving system handles batching internally — chunks the work, manages its own limits, runs in parallel

Results are delivered back in chunks referenced by job ID — the caller consumes them as they arrive, no reassembly required

Failure is contained — a failed chunk can be retried independently without restarting the whole job

2 logical operations. The receiving system owns its own complexity. Loose coupling. Genuinely async. Scales without changing the caller.

1 Outbound request

1 Job ID returned

N Chunks delivered

0 Caller reassembly

The distinction

What actually makes an architecture event-driven.

Events are for broadcasting state changes.

Not for implementing request / response

An event says "something happened." It does not say "please do this and tell me when you're done." The moment a producer needs a response from a consumer, it is no longer using events correctly — it is implementing request/response over an event bus.
True event-driven architecture is fire and forget from the producer's perspective. The producer does not track whether consumers received the event, does not wait for a reply, and does not care how many consumers acted on it.
If your "event-driven" architecture has a calling system waiting for responses, managing batches, and reassembling results — it is not event-driven. It is synchronous request/response with extra steps.

Async job patterns are the right tool for long-running requests.

Fire and forget — with a way to collect the result

When you need to process a large volume of data and collect a result, the correct pattern is an async job: submit the work, receive a job ID, poll or subscribe for completion, consume the result when it arrives.
The receiving system owns the complexity of chunking, parallelism, and internal limits. The caller's concern is the result — not how the result was produced.
This is what Service Bus, Azure Durable Functions, and similar tools are designed for. A durable orchestration can fan out to a thousand parallel workers and fan back in to a single result — entirely internally, invisible to the caller.

The limit of the receiving system is its problem — not the caller's.

Encapsulation is not just an OOP principle

A ten thousand record limit per request is an implementation detail. It belongs inside the receiving system's boundary. When the limit changes, the receiving system changes its internal chunking logic. The caller does not change at all.
The moment the caller knows about the receiving system's internal limits and manages against them, you have tight coupling. It doesn't matter that it's communicating over a message bus — the coupling is in the knowledge, not the transport.
Design the contract between systems around logical operations, not implementation constraints. "Process these codes and return the result" is a logical operation. "Here are ten thousand of them, I'll send the rest shortly" is an implementation concern that has leaked across a boundary.

Applicability

Choosing the right pattern.

The right pattern depends on what the operation actually is — not what it's called in the design document.

→Use events when broadcasting a state change to multiple potential consumers who act independently

→Use async jobs when submitting large volumes of work and collecting a result at a later point

→Use Service Bus queues when you need guaranteed, ordered, durable delivery to a single consumer

→Use request/response directly when the operation is genuinely synchronous and low volume

→Never use events to implement request/response — you get the complexity of async with the constraints of sync

→Never let the caller manage the receiving system's internal limits — that is tight coupling regardless of the transport layer

The name on the diagram doesn't change what it does.

Calling an architecture event-driven does not make it event-driven. The test is whether the producer cares about the consumer's response. If it does — if it is waiting, polling, batching, or reassembling — it is not using events. It is using events as a transport layer for something that is fundamentally a request/response operation.

The printer doesn't ask you to send the document ten thousand pages at a time. It accepts the whole job, manages its own constraints internally, and delivers the output when it's ready. That is the correct mental model for any system that receives a large volume of work and returns a result.

One message. One job ID. One result. The receiving system handles everything in between. That is the architecture. Everything else is a for loop in a distributed systems costume.