Do not manually shard your DB
While striving for maximum DB scalability, many people are tempted by the allures of DB sharding.
While DB sharding may help scale out your DB system, there are some real dis-advantages of this approach:
- Instead of managing a single DB, you are now managing many independent databases
- Someone, usually the application developers, needs to design and develop [or re-write] their applications to use a sharded DB
- If your DB is relational, instead of being able to use SQL to do complex joins, you are usually limited to a sub-set of SQL and often can only do operations on the local shard or need to provide a extra parameter like a shard key to a SQL where clause
- If your DB is non relational [eg a key value store], often you need to 'de-normalize' your data model to work around the fact that you cannot do joins. De-normalizing usually has the downside of requiring redundant data. The bigger your database, the more bloat that this redundant data causes. This redundant data costs you in terms of disk, memory and CPU. While these resources may be considered 'cheap', they are not free.
Is there a better way? What if you could achieve the scaling benefits of DB sharding without the limitations?
We designed Oracle TimesTen Velocity Scale In-Memory Database to provide the benefits of sharding without the disadvantages.
Check out Oracle TimesTen Velocity Scale In-Memory Database at OOW 2017.
Disclaimer: these are my personal thoughts and do not represent Oracle's official viewpoint in any way, shape, or form.