Zerodha uses PostgreSQL

Zerodha, India's leading stock broker, relies on PostgreSQL for its data storage requirements. PostgreSQL, an object-relational database management system (ORDMBS), combines the features of both relational databases and object-oriented design.

Zerodha has discovered that PostgreSQL provides a robust and reliable foundation for building scalable systems. Over time, they have realized that some conventional best practices, such as normalization, may not always be suitable at scale. Instead, they have found that employing denormalization and leveraging materialized views yield better results in their large-scale operations.

• Zerodha started with around 100 MB of data being imported daily, with Postgres 8.
• Zerodha currently deal with around 100s of GBs of imports daily.

• ?????????????? ????????????????: Zerodha uses PostgreSQL without replication or a master/slave architecture.
• ?????????????? ??????????: A 500 TB cache across two disks to handle data retrieval, ensuring fast access for the Kite platform.

• Denormalize. -Understand your query planner and direction of yoru queries.
• Zerodha serves 40,000–50,000 queries per second during trading hours
• Redis Cache: Utilized for caching frequently accessed data to reduce the load on the primary database and enhance performance.

• ?????????????? ??????????????: Nightly updates to the console through batch processing.
• ???????????? ????????????????: Nightly backups stored in Amazon S3 for data safety.
• ???????? ????????????????????: Periodic removal of unnecessary data to maintain performance.
• ???????????????? ??????????????????????: Regular vacuuming, analyzing, reorganizing, and rebuilding indexes to enhance performance.

• ?????????? ????????????????????????: Queries are designed to minimize reliance on index columns.
• ????????????????: Efficient management of large datasets through sharding.
• ???????????????????????? ??????????: Use of materialized views for better query performance.
• ??????????????????????????????: Simplification of queries by denormalizing data.

• ?????????? ?????? ??????????????????: Extensive End-of-Day processes for data reconciliation.
• ?????? ??????????????????????: Utilization of AWS services for backups and disaster recovery.

• ?????? (?????????????? ???????? ??????????????): Integration of FDB to manage and query remote data sources as if they were local.
• ?????? ?????? ????????????: Use of open-source SQL JOB RUNNER for background processing and task scheduling.

Will go write more about it later.

Reference:

https://www.youtube.com/watch?v=XB2lF_Z9cbs&t=335s

https://zerodha.com/z-connect/business-updates/zerodhas-infrastructure-upgrades