Improving microservices reliability – part 3: Outbox Pattern in action
And here we are for the third and last part of the Series! Last time we talked about the Outbox Pattern and it’s now time to see it in action.
The main idea is to persist in the same transaction the Entity data plus the Messages that will be eventually sent to the potential subscribers.
This will help lessening the pain of dealing with distributed transactions, guaranteeing that Domain Events can be broadcasted at least once to all the interested parties.
Enough talking, let’s dive into the code!
Here’s the link to the GitHub repository. Take your time and wander a bit through the sources, I’ll wait.
For this demo I’ve used .NET Core, EntityFramework and WebAPI.
In the previous articles I used the eCommerce scenario as example. However here I’ve decided to switch to a different Domain just to avoid repetitiveness.
In this system we have Courses and Students. A Student can enroll, withdraw and complete a Course, and each operation will enforce some specific rules (eg. you cannot withdraw from a course if you’re not enrolled).
Also, every operation will trigger a Domain Event that will eventually be broadcasted to the subscribers. Bear in mind that for the sake of simplicity we’re not making any distinction between Domain and Integration Events.
The Events won’t be dispatched immediately but instead added to a list on the BaseAggregateRoot class.
Then, right before persisting the Entities, the Events will be serialized and converted to Messages. These will be in turn stored in the db. The SchoolUnitOfWork class handles all the logic for this.
It’s worth mentioning that once serialized, Events will be removed from their containing Entities to prevent them from being converted to Messages more than once.
This basically covers the first half of the Outbox Pattern. The second part is instead handled outside the API flow, in a background task. Ideally we could have had a completely separate application for this, probably even running on a different machine. All in all, it doesn’t have to serve all the traffic the API has, so it wouldn’t need powerful hardware.
For this demo I’ve decided to make use of a .NET Core BackgroundService and have it sit next to our API. We talked already about them in another post, take a moment to refresh your memory if you need.
The MessagesProcessorTask will execute at regular intervals and retrieve an instance of the MessagesProcessor class. This in turn, will fetch the unprocessed Messages from the db and publish them.
I haven’t used any “real” message queue here, I think it’s outside the scope of the article, although I might be tempted to add it in the future.
And this completes the series about the Outbox Pattern. Don’t forget to read part 1 and part 2 if you haven’t and don’t hesitate to add a comment or ask a question!