Hey folks! 👋
📢📽️ New one sent right into the tubes!
Another video on using change data capture (often referred to as CDC), to hook up into the database transaction log, forwarding incoming entries to the outbox table.
Previously, we used Debezium for this, but I was left wondering, what if I want to implement something similar with .NET? Turns out it's not super hard, with the help of Npgsql, which provides us with APIs to hook into PostgreSQL Write Ahead Log, so we can read the incoming outbox messages and forward them to Kafka.
https://www.youtube.com/watch?v=4rnSzEd9jPI
🔗 Video related links:
Sample implementation: https://github.com/joaofbantunes/PostgresChangeDataCaptureOutboxSample
Intro to the transactional outbox pattern: https://youtu.be/suKSJ5DvynA
Outbox meets change data capture (feat. .NET, PostgreSQL, Kafka and Debezium): https://youtu.be/WcmLvoxs9ps
PostgreSQL Write-Ahead Logging (WAL): https://www.postgresql.org/docs/current/wal-intro.html
Npgsql Logical and Physical Replication: https://www.npgsql.org/doc/replication.html
Thanks for stopping by, cyaz! 👋