You can use Debezium and SQL Server CDC to publish data changes in a reliable and scalable way. Once its a key part of your solution, you’ll need to stay on top of any issues that you encounter.

The connector logs and the Debezium FAQ will help you solve most issues. However some issues don’t show up in the logs.

Changes Not Being Picked up After Initial Load

Before fixing this issue, check if you have lost any changes when CDC was down.


DMS

AWS DMS (Database Migration Service) supports ongoing replication to keep your source and target databases synchronized. However there are some pitfalls you should be aware of before you use it in production.

This post lists some of the issues you may run into with an ongoing DMS task that keeps a target database in sync with a source RDS SQL Server database.

Corrupted Data

You might notice corrupted rows in your target database - where some rows have values from other rows. This happens when DMS tries to replicate nvarchar(max) columns.

An easy way to identify this is to set up a…


Pause alarms

AWS CloudWatch Alarms includes most of the features you would expect in a monitoring solution. However, the ability to set up blackout windows to pause alarms is sorely lacking.

Popular solutions includes relying on lambdas - to enable / disable the alarm, or to act as a gateway for the alarm. Here’s a solution which does not use lambdas.

Step 1 — Specify a Blackout Window


A Large AWS S3 Bill

AWS S3 costs are pretty low. But when you are getting a $500 bill for storage every month, it is worth spending a few hours to bring that down.

Plan to do these steps over a few days instead of at one shot. Since the AWS Cost & Usage Report is not updated in real time, it can take up to 24 hours before you see the impact of your changes.

Which Bucket Is That?


Complex test set up and brittle tests can make you hate unit tests. Especially when writing tests takes a lot more time than writing code.

Anything in your code that makes it hard to write unit tests is something that should be changed.

Terms

From Martin Fowler’s TestDouble post referencing Gerard Meszaros’s book.

  • Dummy — compilers require all parameters when calling a method. If these parameters are not used in your test, you can set them to dummy values — usually null.
Func<int, int, int> multiplyDummy => null;


Gone are the days when only obscure commands in little black windows could get DevOps stuff done. Chat bots are everywhere now!

There are quite a few tutorials on how to get your first Slackbot up and running. This is what comes after. You don’t want your bot relegated to a side channel or be the equivalent of command prompt in Slack.

After all, if @treebot responds in #forest which everyone has muted, has it really responded? On that note…

A Good Bot Tells Bad Jokes

Everyone knows it’s a bot, but there is a world of difference between a bot that responds with ”calling InitiateGlobalThermonuclearWar…


CloudWatch Rules are an excellent way to trigger scheduled jobs if your solution is hosted in AWS. However, the lack of support for local time is a deal breaker, especially if you observe daylight saving time.

Proposed workarounds include having a lambda update the schedules 2 times a year, delaying execution using code where you have access to the local time, running it on both schedules if the job is idempotent, and others.

Here is a solution that you can adopt manually and slowly refine to be more automated.

The Not So Secret Sauce


In a microservices architecture, each microservice manages only data related to its bounded context. The entire domain data is spread across multiple databases and sometimes across multiple storage technologies — relational databases and different NoSQL variants.

Handling data requirements that cross microservice boundaries is not an easy task. There are several approaches you can take — getting data from multiple sources together when required, making the data available where required, or moving the data.

Aggregate Data When Required

If the page or API design allows it, have the client or an aggregator service query all sources and combine the data. …

Don

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store