amazon rds is expensive

Hi All.

It's been a time since my previous post about PostgreSQL VS Oracle and I thought about keeping the promise of finally posting about mongodb VS PostgreSQL as the main JSON store.

So I present to you "amazon rds is expensive" (But you already got that from the title).

The PostgreSQL/mongodb debate can wait (They both are great with PostgreSQL being better)

So ...

amazon rds is really expensive, but nobody talks about it.

When talking with my clients about why they chose to move their databases to the amazon (or the Microsoft/Oracle) cloud I often face the same logic I find in the stories of the Sabbath Hummus travelers.

You know, Over a cup of coffee after the weekend  they brag about how many hours they spent the last Sabbath on the road  just to enjoy "the best Hummus you can think of".

Well, none of them will ever admit they made that effort just to be served (too often) a Hummus plate from the nearest supermarket.

This is in many cases what I get from new clients who  chose to move from the local on premise to the amazon cloud.

"We don't need a DBA" - sure they do, but now it is called Devops

"We can write whatever code we want" - No they can't.  Bad code means bad performance bringing higher bills once they ask for more machine resources.

"We don't need to index/analyze/vacuum/house-keeping our databases"  - sure they do. they just don't know it yet. performance is usually good the first year mostly because there is a small amount of data at the database.

"We can finally go big data" - well actually they can, but the cost is going to be not reasonable. Going big data means many nodes and read replicas leading to higher cost.

All these scenarios I encountered (and obviously many more I didn't) end up with higher amazon bills than  expected.

Moving to amazon is (sometimes) much like the long trip up north, just to enjoy a plate of Hummus. No one will ever admit it was no fun and too much effort.

Here is a test case:

A client of mine has moved his database to an amazon rds instance.

It is a small PostgreSQL instance with 1 database, about 20 tables. The overall size is under 2 GB data.

By all means this is a small database, But still they faced performance issues.

So they increased the instance size from a mini to medium.

That solved the problem - for a while.

After 3 months the performance issues came back.

They increased the instance size once again. This solved the problem for only a few weeks.

At this point they realized (DAH !) something is wrong.

It turned out the first time performance was bad was due to a missing index.

The problems the second time were auto-vacuum related. One of the tables was bombarded with many small inserts and deletes (The table acts as a replacement for a queue) and the default auto-vacuum setting didn't fit.

The missing index issue was solved, well, by building the index.

The auto-vacuum issue (changed the parameters to make it more aggressive).

Increasing the instance resource covered the real issues only for a month or two.

And still, after all the improvements we made, the monthly cost of a relatively small multi zoned instance ( + 2 read replicas to offload queries from the primary) is VERY expensive. I can’t write here how much, but it is “whats going on” expensive.

And there is the vendor lock in.

It is easy to migrate into amazon. amazon provides the AWS Database Migration Services which basically replicates your local database into the amazon cloud, moving most of the database while your system is online and enabling a small downtime when making the switch-over to the amazon cloud.

However they do not provide a tool for the other way, from amazon to on premise or another cloud.

This make leaving the amazon cloud near impossible as it often requires a huge downtime  during which data id offloaded from amazon and loaded into the new database.

The application downtime is the simplest way to ensure the data is not changed during the database migration.

amazon not providing a convenient tool for migrating out of amazon is understandable. amazon wants to provide an easy way in but not out, and amazon doesn't have to.

The goal of this post is by all means not to complain about amazon. Their service is great and prices are roughly comparable  with their direct competitors.

I myself have few postgreSQL instances at amazon, mostly for testing and educational reasons. But that still doesn't change the fact.

amazon rds is expensive

BTW - If you are trying to find out who the above mentioned client is by reading my linkedin profile, then forget it. I didn't mention them in my profile.

Please let me know your thoughts about this.