LSM tree is at the heart of most storage systems that provide high write throughput, be it a key-value storage like dynamodb/cassandra or a messaging system like pulsar which is backed by bookkeeper. The various components of a typical LSM backed system are shown below. The main reason why LSM…

This blog is an attempt to pay homage to the classic book “Zen and the art of motorcycle maintenance” by Robert M Pirsig. Let me ponder over the ideas in the book in the context of problem-solving & writing code. There are various layers and dimensions to the book, but…

Event sourcing is an architectural pattern; it’s not a silver bullet — Having successfully built multiple systems using event-sourcing patterns, I have also encountered cases where people have complicated their architecture in the name of event sourcing — an inevitable consequence of being dogmatic about patterns instead of being pragmatic. This article aims to help you understand that event sourcing is just…

There is a common misconception that partitioning a table in Postgres will improve performance, while it’s true in some cases, but sometimes partitioning may actually degrade your performance if done incorrectly. Let me try to give a very brief overview about postgres partitioning in general and some guideline to consider…

Ever since the micro-services movement began, many organisations ventured into adopting it without making sure everyone involved in the process has the same understanding about the clear distinction between a message and an event, as a result they end up building a distributed monolith. I am not going to cover…

With a Kotlin implementation example — Most programming languages like Go, Rust, Kotlin, etc provide very “light-weight” thread abstractions. It allows you to spawn thousands of such lightweight threads without creating any operating system context switch overhead! Although these languages use different semantics to provide such lightweight threads, the underlying building blocks and the implementation patterns…

Atomic swap is a simple technique to exchange/swap the ownership of assets present in 2 completely different blockchains. Let’s consider an example where Alice, who owns 1 BTC, wants to exchange it for 1 DOGE owned by Bob. If Alice and Bob trust each other, then Alice could just authorise…

DynamoDB is a high performance key value database, designed to be highly available and always writable. Most traditional databases favour consistency over availability (CAP theorem), some modern databases however provide eventual consistency in favour of availability. If your organisation is running hundreds of services, you would need strong consistency for…

Every company want’s a good work culture, they look for help outside, but very few people look inside. You cannot pick a culture from a catalog and decide to use it, a specific culture emerges out of a specific type of people. For example, spotify model emerged out of a…