eliasbrange.dev
Implementing the Transactional Outbox Pattern for Serverless Domain Events

Implementing the Transactional Outbox Pattern for Serverless Domain Events

1. Introduction

Events are the key part of an Event-driven Architecture (EDA). They are the mechanism that allows services to communicate with each other in a decoupled way.

For example, an Order Service emits an OrderCreated event when a new order is created. The Shipping Service then consumes this event that starts the shipping process.

But how can you ensure that you can reliably publish events? Take a look at the following code:

1
exports.handler = async (event, context) => {
2
const order = domain.NewOrder(...);
3
4
await persistOrder(order);
5
await publishDomainEvent(new OrderCreated(order));
6
}

This looks straightforward, right? The order service receives a request (or a command) to create a new order. It validates the order, persists it, and publishes a domain event.

When building distributed systems, eight fallacies describe false assumptions that developers often make. One of which is the network is reliable.

Look at the above code again. What happens if the publishDomainEvent call fails? The order will be persisted, but no domain event will be published. If you don’t handle this case, you will end up with an inconsistent state. The shipping service will not initiate the shipping process.

There are many solutions of varying complexity to this problem. In this article, you will see how to implement the Transactional Outbox Pattern to solve it in a Serverless environment.

2. What is the Transactional Outbox Pattern?

In the transactional outbox pattern, you persist updates to business entities and any domain events related to those updates in the same transaction. A separate process reads the persisted domain events and publishes them to a message broker. The transactional nature ensures that the entity and related events are either all persisted or none of them are.

The transactional outbox pattern.
The transactional outbox pattern.

In the diagram above, when the Order Service receives a request, it:

  1. Persists any updates to business entities in the Order Table and any related domain events to the Outbox Table in a single transaction.
  2. The Message relay process gets the domain events from the Outbox Table.
  3. The Message relay process publishes the domain events to a message broker.

3. Using DynamoDB Streams for Change Data Capture

DynamoDB Streams is a powerful feature of DynamoDB that lets you capture all item-level changes made to a table. When you enable a stream on a table, you can configure a Lambda function to be invoked on every change to the table. Streams are fully managed by AWS and are a perfect fit to implement the Transactional Outbox Pattern.

Looking back at the code in the Introduction, we were worried about what happens if the publishDomainEvent call fails after the persistOrder call succeeds. DynamoDB streams can help us solve this.

Using streams, the Lambda function only needs to persist the order to the OrdersTable. This change will be sent through the stream to another Lambda function, which publishes the domain events. If there are any errors when publishing the events, the Lambda function will retry the operation until it succeeds or until the records expire.

4. Implementing the Pattern in Serverless Applications

4.1. Naive approach

Let’s start with the naive approach. We will use the following architecture:

A simple but naive transactional outbox pattern.
A simple but naive transactional outbox pattern.

In this pattern, the CreateOrder Lambda function persists the order to the OrdersTable. The table is configured with a DynamoDB stream that triggers the PublishEvents Lambda function on every change to the table.

The PublishEvents Lambda must parse the change and determine what kind of domain event to send. The code might look something like this:

1
exports.handler = async (event, context) => {
2
for (const record of event.Records) {
3
const domainEvent = parseDynamoRecord(record);
4
5
if (domainEvent) {
6
await publishDomainEvent(event);
7
}
8
}
9
};
10
11
const parseDynamoRecord = (record) => {
12
if (isOrderCreated(record)) {
13
return new OrderCreated(record.dynamodb.NewImage);
14
}
15
return null;
16
};
17
18
const isOrderCreated = (record) => {
19
// Check if the record is an INSERT event for an order.
20
return (
21
record.eventName === 'INSERT' &&
22
record.dynamodb.NewImage &&
23
record.dynamodb.NewImage.type === 'ORDER'
24
);
25
};

This might look like a good and simple solution at first. But it has some drawbacks:

The above drawbacks are not ideal. They force you to move core domain logic to the outer bounds of your application while simultaneously coupling the logic to your database schema.

There’s a better approach.

4.2. Use a Separate Outbox Table and DynamoDB Transactions

A better approach to the transactional outbox pattern.
A better approach to the transactional outbox pattern.

This pattern introduces another table, the OutboxTable. It relies on DynamoDB transactions to atomically persist both the entity and domain event in a transaction. This means that either both the entity and domain events are persisted or none of them are.

Now, the PublishEvents Lambda function only relays events from the OutboxTable to EventBridge. The code might look something like this:

1
exports.handler = async (event, context) => {
2
for (const record of event.Records) {
3
if (record.eventName !== 'INSERT') {
4
// We only want to publish events when they are inserted in the outbox table.
5
continue;
6
}
7
await publishDomainEvent(record.dynamodb.newImage.event);
8
}
9
};

The logic to determine what events to publish is kept within the core domain logic inside the CreateOrder function. The outer bounds of the application are responsible for persisting the order and any domain events in an atomic transaction.

1
exports.handler = async (event, context) => {
2
const { order, events } = domain.NewOrder(...);
3
4
await persistNewOrder(order, events);
5
}
6
7
const persistNewOrder = async (order, events) => {
8
await dynamoClient.send(
9
new TransactWriteItemsCommand({
10
TransactItems: [
11
{
12
Put: {
13
TableName: "OrdersTable",
14
Item: order,
15
},
16
},
17
...events.map((event) => ({
18
Put: {
19
TableName: "OutboxTable",
20
Item: {
21
event,
22
},
23
},
24
})),
25
],
26
}),
27
);
28
};

This approach has several benefits:

5. Handling failures

The Transactional Outbox Pattern is all about reliably publishing events. Even though most of the services used are fully managed by AWS, errors can still occur. Let’s look at what happens if the PublishEvents Lambda function fails to publish events.

DynamoDB streams send batches of records to the PublishEvents Lambda function. If an invocation of the PublishEvents function fails, the entire batch will be retried until it succeeds or until the records expire.

For example, imagine that a batch of ten records is sent to the PublishEvents Lambda function, and the function fails to publish the 10th event. The entire batch is retried, and this time it succeeds. Events 1-9 will now have been published twice.

There are a couple of different ways to handle this:

6. Limitations

Since this pattern relies on DynamoDB transactions, it inherits the same limitations. Transactions are limited to 100 items. This means that you cannot publish, say, 150 domain events in a single transaction.

I can’t really see a reason for publishing that many events from a single operation, so this shouldn’t be a problem in practice.

7. Conclusion

The Transactional Outbox Pattern is a great pattern to publish domain events reliably. DynamoDB streams let you easily implement the pattern in your Serverless applications. Combined with DynamoDB transactions, your domain logic can be kept nicely encapsulated in the domain layer, while the outer parts of the application can be kept as simple as possible.

8. Further reading


About the author

I'm Elias Brange, a Cloud Consultant and AWS Community Builder in the Serverless category. I'm on a mission to drive Serverless adoption and help others on their Serverless AWS journey.

Did you find this article helpful? Share it with your friends and colleagues using the buttons below. It could help them too!

Are you looking for more content like this? Follow me on LinkedIn & Twitter !