Database migration to AWS
Igor Royzis
CTO | Software Engineering Leader in Cloud, Data & AI | Scaling Organizations, Driving Innovation, Delivering Results
History of AWS relational databases – or how we got here
Early 2000’s, everyone is happily buying servers, installing and configuring on-premise relational databases, both commercial?(SQL Server, Oracle) and open source (Postgres, MySQL). Life is good. Need more storage? Buy more disk space. Need more compute power? Buy more memory. Cannot upgrade anymore? Buy another server.
And then, at the end of 2009, when things are seemingly good, AWS introduces RDS for MySQL. Then RDS for Oracle in 2011, RDS for MS SQL Server in 2012, RDS for PostgreSQL in 2013, Aurora MySQL in 2014, Aurora PostgreSQL in 2017 … What’s happening? We don’t have to buy database servers anymore? We can upgrade storage and memory with a click of a button? What? Multi Availability Zone Support with a click of a button?
This is amazing! But we’re not ready to jump to this new technology yet. Let’s wait and see.
2015, 2016, 2017 .. time goes by. On-premise databases grow in size and complexity. Some applications are being moved to the cloud. Some new databases are being created in the cloud. Is it time to migrate our 15, 20 year old on-premise databases to the cloud? We’d love to, but it’s so complex, so much data, what about all the existing stored procedures and applications, how are we going to do the cutoff?
Sounds familiar? Well, it can all be done. It can be time-consuming, but in most cases it’s more straight forward than it sounds. Sometimes it’s complex, but it can and it’s been done with proper approach, planning and execution.
Use Cases for Database Migration to AWS
Here are a few popular database migration use cases, from simple to more complex. These are high level, just to give an idea of the approach.
E-commerce MySQL database migration to AWS RDS for MySQL
Source: On-premise MySQL
Size: 500GB
Target: RDS for MySQL
Complexity:?Low
Approach in a nutshell:
– Use DMS (Database Migration Service) for initial full-migration and on-going replication
– Test/Switch applications to use new database and stop using on-premise MySQL
AWS technologies used:
– AWS DMS (Database Migration Service)
– RDS for MySQL
Multiple MS SQL Server databases migration to AWS RDS for SQL Server
Source: On-premise MS SQL Servers
Size: 30TB
Target: RDS for SQL Servers
Complexity:?Medium
Approach in a nutshell:
– Use Snowball Edge, local DMS Agent, for initial full-migrations and on-going replication to S3
– Use DMS for migration from S3 to RDS for SQL Servers
领英推荐
– Test each web and mobile application against target database and switch one database at a time
AWS technologies used:
– Snowball Edge
– Local DMS Agent
– AWS DMS (Database Migration Service)
– S3
– RDS for SQL Server
Oracle data warehouse migration to AWS Redshift
Source: On-premise Oracle
Size: 50TB
Target: Redshift
Complexity:?High
Approach in a nutshell:
– Rewrite ETL/ELT jobs to generate data for the new data warehouse
– Use SCT (Schema Conversion Tool) to generate target Redshift schema
– Use Snowball Edge, local DMS Agent, for initial full-migration and on-going replication to S3, followed by DMS for migration from S3 to Redshift
– Test/Switch BI tools to use Redshift and stop using Oracle
– Start running new ETL/ELT jobs to keep populating new data warehouse
AWS technologies used:
– AWS Glue
– SCT (Schema Conversion Tool)
– Snowball Edge
– Local DMS Agent
– AWS DMS (Database Migration Service)
– S3
– Redshift
Conclusion
Migrating databases to AWS is a matter of choosing the right approach depending on the types and sizes of your databases, as well as usage patterns, rate of data change and timing requirements.
Vice President of Business Development
3 年The not so distant past sounds incredibly grueling. I am glad to be spoiled by Amazon Web Services (AWS) as a partner and to have Igor Royzis bringing it all to life.