Architectural migrations - HOW TO

Disclaimer: Highly opinionated

Speaking from my experience, orgs/teams migrating their production systems to more mature architectures or tooling means it rarely ends well. It doubles the work, and suddenly you are short of resources and things start to break, often, in nasty ways. It doesn't mean we stop improving things and it may not be a choice sometimes.

There are 2 key possible reasons why you would want to migrate to new systems - stability and scalability. One of them is NOT the right reason. If you are looking for stability in your production systems please consult your architect and often making the necessary fixes to your existing systems should be it.

If you need to scale your systems then here is a better way to do architectural/tooling migrations. Engineering teams and everyone else are better off understanding the following aspects to succeed.

1. Don't patch your existing production systems. Plan and rebuild from scratch as an independent project. Allow yourself a fallback plan.
2. Data migrations are different from architectural migrations. DON'T do them together. Isolate them by either doing data migrations first and then migrating your architecture or the other way. I can't stress this enough -- don't mix them up.
3. Build your new systems as tiny, loosely coupled services. Trust me, it is a lot simpler to merge services(for right granularity) at a later point without breaking the modern microservices principles.
4. Plan the migration in many quick, small iterations by plugging the new services. You should be able to unplug and revert to the old system. This is not patching. Test well.
5. Prioritize functional consistency over scalability. It is okay to have bottlenecks in scalability during migration but never break the functionality of your production systems.
6. Communicate and communicate. Continuously communicate with your internal teams and users about the migrations, new deployments and expected changes upfront. Nobody likes surprises in these things and its everyone's business to know.