How to Build an Empire with Plug-in Modules

There was a time several months ago when with the mobile app we began experiencing problems that come with large legacy codebases :

• As more engineers worked in the same repository, when something broke, it broke for everybody. More merge conflicts needed resolution. 
• More features meant there were additional regression tests run. Unfortunately, there were many more tests added with each new release. As a result, CI runs took longer to go green.
• Swift became the language for all new features. It helped us attract and hire the best talent and also made the app more stable with its type safety and compile-time checks. However, the cycle time on stories trended up. 
• Teams in different time zones found they could not pick up and run with a story without stepping on other squad members. The acceleration benefits of timezones were lost.

We took a step back, collected data on the code and CI runs, and analysed it to see whether the story it told us matched up with our perceptions of the root causes - 

• 80-20 rule: App architecture showed that 2-3 feature modules made up a large percentage of the code. They had been written in Objective-C a few years ago and been hardly modified since. While these features were essential business offerings, they rarely changed and were used by less than 20% customers. Yet, at every PR, they added to compile-time and test run time.
• Swift and Objective-C Interoperability: Spaghetti like dependencies working in consort with error-prone design patterns were dragging down other areas in active Swift development. When modifying shared code, it became necessary to guarantee Interoperability which made freely using appropriate Swift language constructs quite tricky.
• Failures in all features treated equally important: Flaky or failed tests for features having low customer traffic blocked engineers and testers who were actively working on core customer features. 
• Adapting to new worlds became arduous: Antiquated lines of code in some features were causing the whole app to hit speed bumps when adopting Apple's new, secure frameworks and capabilities. Implementing Dark Mode, Cryptokit, MetricKit, Combine, SwiftUI, December 2020 Mandate for WKWebviews and April 2020 Mandate for the Split screen, was becoming challenging.

How could we ship new features frequently and at quality? How could we maintain our velocity and increase code quality as the engineering team diversified to different locations?

Architecting to clear the Hurdles

We needed a new mechanism that would support the mobile app's growth for years to come. It dawned on us that if we could modularise and separate these passive, non-core modules, then we could modernise them separately. Not having to code on the same trunk as the app freed the team from having to set up refactor firebreaks frequently in code. Developing a plugin-based architecture, along with a partnership feature squad model, would allow us to meet this goal.

The Master Plan

Think of the main app as a container, and the features as standalone plugins, developed in isolation. There are well-defined interfaces between the app and the plugins. By conforming to several protocols, the plugins communicate with the app without knowing anything about its implementation details. It’s encapsulation at app scale.

The app team provides the plugin teams with a starter project with a host app that helps develop and test against mocks. This project compiles into a framework, so when the feature is ready for release, the plugin team publishes a new version of it.

Once the plugin teams have a semantically versioned framework that tests green on the host app, the app team then pulls the new versions of all plugins, performs automated integration testing, security scans, and then integrates these plugins into the main app.

A remote configuration turns the plugin on or off on the fly. We ship the feature in phased releases and collect metrics and feedback that serves as input to plugin teams for future iterations. 

The plugins are triggered using standard controls so that the user can seamlessly navigate from the app to the plugin and vice versa. 

Accelerated Feature Delivery @ Scale

So, how does this architecture support the growth of our organisation?

All squads used to work on a single trunk. The plugin architecture, on the other hand, allows teams to work on separate codebases. 

The plugin teams move in two-week sprints and at the end which, they publish a semantic version of their feature as a framework.

The plugin team performs automated analysis, security scans, testing, and review of their code with tools and quality metrics at parity with the main app. Once green, the feature is incorporated into the main app with our dependency manager.

Features are developed and tested in parallel, in a distributed fashion, and this helps scale mobile development, while simultaneously accelerating the delivery. As an added benefit, since all the features are sandboxed and have to interact with each other and the app through well-defined interfaces, this architecture prevents spaghetti dependencies and protects the separation of concerns.

Guide-rails

Let’s take a look at how we reduce risk and take care of the plumbing for the feature teams so they can focus their time and energy on creating value for our customers.

Design System

We have actively built out a design system based on collaborative sessions with UX and Legal on brand guidelines. With this, the squads are armed with principles that focus on consistency/parity and efficiency. The component framework from this system can be dropped inside any feature and is the single point for brand decision and Apple HIG changes.

Engineering Trifecta

First, we onboard a feature team and get their workspaces off the ground.

Second, during the build phase, we provide support by helping identify complexities around dependencies, backend integration, and encouraging software best practices, scalability, and security. Our seasoned DevOps pipeline automates linting, static analysis, security scanning, and code coverage metrics into the code review process, reducing the risk of human error.

In the third and last phase, when the feature is in production, we closely instrument and monitor the feature’s performance and health metrics. We also have systems in place to make sure the feature is powered by APIs that are available, accessible, and conform to contracts. We do this by using industry-leading tools for instrumentation and observability across the mobile and backend. We then analyze the trends and send automated alerts to stakeholders as necessary. 

The Landscape now 

Our company's core strategy is to make a simple digital experience. The plugin architecture unlocks parallel feature development and testing, regular releases, consistent branding and quick uptake of Apple released capabilities. It frees teams from legacy designs and motivates them to skill up to use contemporary technology.

• We can swap out whole modules along with their backends with ease.
• This architecture is scalable and stable and allows us to create valuable content and integrate both internal and external development teams into the software development life cycle without compromising on velocity or quality.
• All teams are empowered to experiment, assured in the guarantee that all plugins will integrate securely with the main app. Each feature starts as a hypothesis. Squads iterate on it till they have an MVP. Then it is integrated into the main app.
• The plugin squads experiment, mature and maintain their framework dependencies. However, when we find those useful for the main app, they are promoted up to it.

Make no mistake, this architecture is no silver bullet. Its merit truly shines for features with the below characteristics: 

• Are in large part standalone business products 
• Can be vertically sliced to separate the UX and backend systems
• Developed and maintained by a cross-functional, independent squad
• Is triggered from a few discrete points in the main app
• Has dependencies with the main app that countable on fingertips
• Need the freedom to choose their release cadence
• Is not always used by the customer, so need not be loaded when the main app loads

Even as this architecture is a big step in software delivery, we're prouder of the growth of our team that put on its thinking cap and demonstrated imagination to overcome business challenges.