PostgreSQL is becoming a catch-all solution for simple scenario, and this is a trend, not an accident. A new article full of evidence enforced the ideas I wrote down in the 2019 about Kafka vs PostgreSQL: lets dig into it.

Read More

We already discussed about Queue management solutions in the past, and I am always happy to write about it. Queue managers are not easy to implement, and there is a reson if IBM MQ Series is still a successful product. Some month ago, a big bank customer asked me to provide a small queue implementation to increase asynchronous internal processing of our payment solution.

The project had very strong contraints: I could not use existing queue system because they were not yet available, and I need to be able to provide microservice parallelism in a cloud-environent.

I have very little time to provide a solid solution, and re-inventing the wheel was not an option. Performance was important, but we plan to have a managable numbers of transactions per seconds, far behind modern cloud database capacity.

Challenge accepted.

Read More

Some co-workers started using Apache Kafka con a bunch of our Customers.

Apache Kafka is a community distributed event streaming platform capable of handling trillions of events a day. Initially conceived as a messaging queue, Kafka is based on an abstraction of a distributed commit log[*].

To get this goal, Apache Kafka needs a complex servers setup, even more complex if you want the certification for the producing company (Confluent).  Now, if you are planning to use Kafka like a simple JavaMessaeSystem (JMS) implementation, think twice before going on this route.

PostgreSQL 12 offers a fair (and open source) partition implementation, whereas if money are not a problem, Oracle 12c can happy scale on billions of record before running into troubles (and ExaData can scale even more).

PostgreSQL and Oracle offer optimizations for partitioned data, called “Partition Pruning” in PostreSQL teminology:

With partition pruning enabled, the planner will examine the definition of each partition and prove that the partition need not be scanned because it could not contain any rows meeting the query's WHERE clause. When the planner can prove this, it excludes (prunes) the partition from the query plan.

This feature is quite brand new (popped in PostreSQL 11) but it is essential to a successful partition strategy. Before these feature, partitioning was a black magic art. Now it is simpler to manage.
Read More

In my code ramblings during the developement of Code Zauker, I ended up studing a bit NoSql  database.

Code zauker started using Redis, because Redis is a very bold memory-based no-sql db. Redis also support complex data type like sorted set, lists and so on, which was very userful. Anyway I needed a very fast way of doing statistics on data collected by code zauker, for inspecting data and finding out new feature I’d like to add to it. Worst, my current pocket pose a hard limit on a redis instance in terms of available RAM. So I started thinking a lot about a redis drop-in replacement for Code Zauker.

Read More

At Gioorgi.com we was never a SQL fan. In 2000 we thinked SQL was boring, mostly because sql algebra could be a bit boring. Then we found this book written by one of the father of SQL. Years ago Google and then Facebook popped out with new incredible ideas, for improved and super fast scalability, which eventually turned to the “NoSQL” mantra.

But NoSQL is a mature technology, or it is only a path traced by the Social company out of there? Let’s explore together…

Read More