THEORY: Distributed Transactions and why you should avoid them (2 Phase Commit , Saga Pattern, TCC, Idempotency etc)

avoid-distributed-transactions.md

Distributed Transactions and why you should avoid them

1. Modern technologies won't support it (RabbitMQ, Kafka, etc.);
2. This is a form of using Inter-Process Communication in a synchronized way and this reduces availability;
3. All participants of the distributed transaction need to be avaiable for a distributed commit, again: reduces availability.

Implementing business transactions that span multiple services is not straightforward. Distributed transactions are best avoided because of the CAP theorem. Moreover, many modern (NoSQL) databases don’t support them. The best solution is to use the Saga Pattern.

[...]

One of the most well-known patterns for distributed transactions is called Saga. The first paper about it was published back in 1987 and has it been a popular solution since then.

There are a couple of different ways to implement a saga transaction, but the two most popular are:

• Events/Choreography: When there is no central coordination, each service produces and listen to other service’s events and decides if an action should be taken or not;
• Command/Orchestration: when a coordinator service is responsible for centralizing the saga’s decision making and sequencing business logic;

How to implement the Outbox pattern in Go and Postgres

É possível fazer Event-Driven Architecture (EDA) sem um broker?

• Minha opinião no twitter sobre o tema
• ChatGpt: Implementando Event-Driven Architecture (EDA) sem um broker no centro

⭐️ The Write Last, Read First Rule: Keeping systems in synch

⭐️ Building a Durable Execution Engine With SQLite - by Gunnar Morling

• GitHub repository

On Idempotency Key - by Gunnar Morling
A good write-up on Twitter about the Gunnar's article - by Abhishek Singh (@0xlelouch_)

A good description on the difference between Deduplication and Idempotency

🔗 Post on Linkedin by Henri Maxime DemoulinHenri Maxime Demoulin

Many engineers confuse these two: deduplication keys vs idempotency keys. They look similar but solve two completely different problems.

Deduplication keys are used to detect that a unit of work has already been executed and skip the execution entirely.

Their goal is to prevent the work from running twice.

Idempotency keys are used to ensure that even if a unit of work runs twice (or more), the final state is correct and equivalent.

One prevents execution, the other effects.

Deduplication keys have been around for a while, e.g., SQS uses them to ensure only once delivery within 5 minute windows.

Before you ship your next API or background worker, ask yourself: do you need idempotency, deduplication... or both?

Idempotence comes in different shapes - by Dominik Tornow

Idempotence comes in different shapes

Idempotence is the guarantee that repeating a request yields the same outcome (or, more formally, does not change the state of the system beyond the initial application)

In practice, idempotence comes in a few variants, most notably positive and negative idempotence

Positive

Positive idempotence denotes that the system has accepted the request in the past:

I have accepted this request in the past, I will accept the request again, I will apply this request again, nothing changes

-or-

I have accepted this request in the past, I will accept the request again, I will not apply this request again

Negative

Negative idempotence denotes that the system has rejected the request in the past

I have rejected this request in the past, I will reject this request again

Negative idempotence is often harder to guarantee:

When a system accepts a request, the system state changes, the new state is evidence of the past acceptance of the request

When a system rejects a request, the system state may not change, there is no evidence of the past rejection of the request

⭐️ Real idempotence is about resilience to time and change, not just repetition. - by Abhishek Singh