It’s a challenge faced by many businesses seeking to harness the power of event driven architecture with event sourcing as a foundation: what to do when you're not in a position to rewrite existing applications that are not event sourced?
Or alternatively, you might be wondering how to gain some of the value of event driven architecture while you are incrementally rearchitecting existing systems.
If you find yourself in either scenario, here's a valuable alternative that can help you leverage the benefits of event-driven architecture without the need for a full rewrite: the Sidecar Source of Truth pattern with EventStoreDB.
Here’s how this strategy can help you unlock the advantages of event-driven architectures while preserving your existing systems.
When rewriting isn’t practical
At Event Store, we’ve observed a growing number of organizations adopting the Sidecar Source of Truth pattern. This involves running EventStoreDB alongside other operational systems, capturing and processing the low level changes through techniques like Change Data Capture (CDC).
By ingesting these granular changes directly into EventStoreDB, they can be transformed into meaningful events that can be consumed by downstream systems or placed onto an integration bus for further distribution.
But what exactly is a Sidecar?
A Sidecar refers to a companion service or component that runs alongside the main application. It provides additional functionalities, such as logging, monitoring, or data processing, without directly affecting the core functionality of the main application.
In the Sidecar Source of Truth pattern, EventStoreDB acts as the Sidecar, working in conjunction with your existing operational systems to capture and process events.
Real-time transformation and reliable record keeping
There are two primary reasons why the Sidecar Source of Truth pattern with EventStoreDB is becoming widely adopted:
1. Event Processing
With EventStoreDB, you can convert low-level events (for example CDC events) into more meaningful domain or integration events in real-time. Low-level, out of context events are of limited use for integration to other systems, or sending to downstream data technologies. But with EventStoreDB’s projection feature, you can listen to one or more of these low level streams and emit higher level events needed across your enterprise.
2. Keeping a historical source of truth for your (amnesiac) operational systems
EventStoreDB preserves a record of events, starting from the low-level ingested events, all the way to the integration events. This historical record is critical for when those inevitable questions come from downstream systems or analytics team a day, a week, or even months later.
Sending the data from place to another, “data in motion”, is often not enough; a record is needed and the operational system, if it is based on traditional “current state oriented” database technology forgets the history necessary to support this. In contrast, EventStoreDB stores events as an immutable sequence wrapped in the business context, so you can easily access this history when you need it.
A trending integration pattern
Traditional integration patterns, such as the bus, broker, and adapter patterns, primarily focus on the orchestration and routing of data between systems. They are centred around the concept of "data in motion," paying less attention to the lifecycle of the data itself. This approach can lead to a kind of "data amnesia," where the history and lineage of the data are lost.
In contrast, the Sidecar Source of Truth pattern takes a different approach. By running EventStoreDB alongside operational systems, it acts as a source of truth and a real-time event processor.
This key distinction allows the Sidecar Source of Truth pattern to address the "amnesia" issue. The ability to store events, track changes, and retain historical information provides a powerful tool for your business. When downstream systems encounter issues, developers can consult this source of truth to understand the history of events and the system's state at any given time.
Furthermore, the Sidecar Source of Truth pattern introduces greater flexibility in data handling. Data no longer simply moves from one point to another; it evolves, enriches, and leaves a trail of its journey. This enhanced traceability and the ability to replay events for testing or system restoration purposes greatly enhance debugging capabilities and operational resilience.
Getting started: unlock the potential of event-driven architectures
The Sidecar Source of Truth pattern leverages the strengths of EventStoreDB to enhance the capabilities of traditional integration patterns. It addresses the common challenges faced in complex system integration scenarios, providing a more complete, reliable, and valuable solution for modern businesses.
To get started with EventStoreDB, download the open source version of the database and check out the getting started guide.
