What is Idempotency?
Think of your paycheck. You don’t want a communications error
between your employer and the payroll processor to prevent you from
getting paid. If an error happens in that process, you want the
communication to be re-sent, and when the re-send occurs it
shouldn’t cause you to get paid twice. Effectively, idempotency is
the ability to re-send messages when needed without causing
problems from duplicated processing. While you may not mind
receiving your paycheck twice, you would feel differently if you
were charged twice for a plane ticket.
We live in a world of interconnected systems. Our systems need
the ability to re-send messages in the event of a failure, and
systems receiving the communications should be capable of receiving
the same message more than once without duplicating transactions.
In the most general of terms, handling messages in an idempotent
manner comes down to two concepts; knowing when a message needs to
be re-sent, and knowing if an incoming message has already been
processed. First let’s look at some of the message flows that can
occur in a REST-based system.
Point-to-Point Message Flows via REST:
When things go well, the message is sent once, processed once,
and acknowledged once.
The code to handle the happy path is simply a single line of
code invoking an async Send method.
Unfortunately, given enough time and enough traffic, you can be
assured that something will eventually fail.
Ways it can go wrong:
The original message could fail during the send, because of a
communication error or because the Consumer is down.
The message might be sent but have a failure in processing.
The Consumer may successfully report back a failure, or there
may be a communication failure preventing the Producer from
receiving a notification of success or failure.
In a point-to-point communication pattern like this, the message
producer now has to be responsible for re-sending when anything
other than the happy path occurs.
Point-to-point communication gets even more challenging when
multiple consumers are interested in the same message.
Once the concept of multiple consumers is introduced, the logic
for sending and re-sending messages reliably gets exponentially
more difficult. What should the Producer do when a message fails
for a single consumer. Does it repeatedly re-send the message to
all the consumers until it’s acknowledged by the failing consumer?
Does it stop sending to the other consumers until the failing
consumer finally acknowledges the message? Or, does the Producer
keep track of which consumers were successful and which failed and
send different content to each?
The problem gets even worse if you introduce the concept of
ordered message delivery.
With this type of setup, you can quickly find yourself writing
more code dealing with message delivery than with providing actual
business features. Instead of doing that, we can solve the problem
by changing our communication paradigm by using a system designed
specifically for handling messaging and the related
A Better Approach:
If we go back to basics and consider the Separation of
Responsibilities, our business code should be focused on providing
business functionality. The capabilities around handling messaging
and delivery rules have nothing to do with our core business logic.
It is critical to the proper functioning of interconnected systems,
but it’s not part of our core competencies. Instead of writing
message plumbing, we should instead focus on our business
deliverables and offload message delivery concerns to a system
designed specifically for that purpose. An additional benefit of
using a messaging intermediary is we can increase system
reliability since Message Producers can continue functioning and
sending messages even when one or more Consumers are
Various forms of messaging systems can act as intermediaries
between message producers and consumers, and they range from Queues
and SNS Topics to Enterprise Service Buses. One such option is
Pulsar by Apache, which is a cloud-native message broker that
addresses the shortcomings of some of the solutions that came
According to their website (https://ift.tt/uWdmZA8), “Apache
Pulsar is a cloud-native, multi-tenant, high-performance solution
for server-to-server messaging and queuing built on the
publisher-subscribe (pub-sub) pattern.” Instead of having
point-to-point communication where each Producer knows about each
consumer, Pulsar allows Producers and Consumers to communicate only
with Pulsar, and Pulsar manages the reliable delivery of messages
With this type of model, Pulsar is responsible for deliveries to
the Consumers. If Consumer 3 doesn’t successfully acknowledge a
message, the Message Producer won’t even know there was an issue,
much less have to implement logic to try a re-delivery. All of that
work can be offloaded to Pulsar. It is worth highlighting that this
type of approach does not have point-to-point communication between
systems. In other words, it uses asynchronous communication
We’ve Been Working Without a Message Broker Forever. Why
Breaking up the Monolith
When working with monolithic software, code can communicate
directly with other code dependencies via in-process calls.
Communication within a monolith is easy, but the disadvantages of
monolithic software begin to outweigh the benefits as software
projects get larger and more complex. As we break functionality out
into different services, we need another way for the services to
communicate with each other in a reliable fashion. The moment we
extract the first micro-service out of the monolith, we introduce
inter-process communications and now the stability of the system as
a whole is dependent on multiple services being up and running at
the same time.
Interconnected Application Domains
Applications don’t exist in a vacuum and need to communicate
across different Application Domains, especially in large
enterprises. In the past, a single business unit might have been
able to operate in isolation, but as part of a larger organization,
it becomes necessary for that business unit to share messaging and
events with other parts of the organization. Not only do we need a
way for messages to be transmitted within an application domain, we
need to be able to communicate between domains. Message brokers
facilitate this type of communication and can eliminate the need
for message producers to change each time a new consumer needs to
be added. Martin Fowler pointed out that message-based integration
patterns allow us to reverse the direction of dependencies between
Producers and Consumers. Upstream message Consumers can depend on
messages generated by downstream Producers as opposed to Producers
needing to know about specific Consumers in order to send
I previously mentioned that Idempotency was about the ability of
messages to be re-sent when necessary AND for message consumers to
handle the receipt of duplicate messages without undesired side
effects from duplicate processing. Much of the discussion so far
was about the logical complexities of re-sending messages. A future
post will show a code example with Consumer code capable of
receiving the same message more than once while processing the
duplicate messages “effectively once”.
This is the first in a series of blog posts related to
idempotency and how to handle it effectively.
Posted 27 July 2022 by Doug Ferguson, Director, Software Engineer, WSO Software, S&P Global Market Intelligence
IHS Markit provides industry-leading data, software and technology platforms and managed services to tackle some of the most difficult challenges in financial markets. We help our customers better understand complicated markets, reduce risk, operate more efficiently and comply with financial regulation.
This article was published by S&P Global Market Intelligence and not by S&P Global Ratings, which is a separately managed division of S&P Global.