AWS Database Migration Service (DMS) – Introduction

AWS DMS is a database migration solution which can also be used to sync databases. Simple to use, with minimal downtime for migration, supports widely used database technologies, low cost, fast and easy to set up, and reliable.

In Use

No drivers or apps to install - light touch. No changes to the source DB in most cases, though a user account is required for access (which also facilitates control around the level of access). Just a few clicks to start a migration from the console, the source databases continues to run throughout, until cutover. . DMS manages the complexities of the database migration for you. Changes can be automatically replicated, and it can be used for continuous replication.

Supports Oracle, MS SQL, Azure SQL, PostgreSQL, MySQL, SAP ASE, MongoDB, Amazon S3 and IBM DB2 sources. Targets can be Oracle, SQL Server, PostgreSQL, MySQL, Radshift, SAP ASE, Amazon S3, Amazon DynamoDB, Amazon Kinesis and Amazon ElasticSearch. Simple to execute:

- Name the task

- Connect to source

- Connect to target

- Create a replication instance

- Create a task

- Choose Migration type (existing data / replicate changes / both)

- Target Prep (Do nothing, drop tables on target or Truncate)

- Include LOBs - (None, Full or Limited Mode)

- Run the task

Enable validation and logging.

Can be run in a VPC over private connections, but can also be made publicly accessible if necessary. Multi-AZ is possible for improved availability, given some migration tasks can take time to run depending on data volumes.

DMS is highly resilient and self-healing, with continual monitoring of source and target databases, network connectivity and replication. Can also be piped in to Cloudwatch.

Useful Non-DMS Components

Workload Qualification Framework (WQF) - Assesses and classifies OLTP workloads to determine the ease of migration and staff-hour consumption, and to inform on appropriate target AWS services.

- Designed for solution architects, partners and consultants

- Assess workloads by complexity, technology and size

- Recommends migration strategy and tools

- Gives migration engineers clear and actionable information

- Integrated fully with broader toolsets

Generates reports of migration effort based on schema analysis, with no need for data.

AWS SCT - Schema Conversion Tool

Creates assessment reports for migrations. Can convert DB schema and DW schema, as well as converting embedded code and highlights where manual intervention is required. Schemas can be optimised as part of the conversion process. SCT is downloaded and run from any connected client device.

DMS Components

Replication Instance - the instance that runs the tasks - dedicated EC2 instances sized appropriately which can be removed once migration is complete. Supports multiple tasks. Private or Public connectivity.

Tasks are the workhorse of the migration, which run on a replication instance with many running in parallel when required. Task settings JSON gives control over performance and debugging.

Migration Scenarios

- Use DMS alone - DMS create tables and primary keys using drop tables on target. Manual secondary indexes and foreign key objects can be added after initial load.

- Use SCT and DMS together - using the schema tool to create all objects, disable secondary indexes and foreign key objects.

Migration steps

Having set up source and target, the basics are:

- Start Full load - time taken depends on data volume and available bandwidth.

- Once load is complete, any changes can be applied to reach a steady state. Not transactionally in sync, but near real time (bandwidth dependent).

- Disconnect apps from source DB, and connect to target.

Load is table by table, and changes are transactional.

DMS can manage single source to many targets, many sources to a single target or many sources to many targets, as well as the 1:1. Filtering tables is possible, and DMS can operate homogenously or heterogeneously.

Tips

- Enable Cloudwatch logs (not enabled by default)

- choose LOB mode carefully

- Replication instance security group is default for VPC, change after creation

- Check for unsupported data types

- Read replicas or standby can be used as a migration source

- Use larger instances for maximum throughput

- Check Network throughput

- Split load with multiple tasks or instances

- Contention can be reduced by turning off logging or running in a single AZ

And that, my friends, is about it for a intro. You can check out my Amazon AWS Certified Solution Architect book on Amazon here - I probably mentioned it being a best-seller! You can read more stuff from me on my website, here.