Flock.ing @ bol.com - Refactoring the Mobile App API
This post is part 4 of a mini-series about Flock.’s involvement with the mobile app development at bol.com. Be sure to read part 1 to learn more about the mini-series itself.
Within the app family of bol.com, development was initially split into disciplines: separate teams were responsible for iOS, Android and for the API delivering the data. But as the family kept growing, this way of working reached its limits. Everyone within the family needed to know about every bit of functionality, which meant delivery of new features became tedious and slow. As a result more and more time was spent on aligning stories between teams. The situation called for a shift from discipline teams to feature teams, which, if all kinks were resolved, would give each team more autonomy to bring features to production themselves. Another advantage would be an increase in the velocity of the app family and sustainable organisational growth.
A direct consequence of such set-up is having different teams working in the same codebase, with all the implications. What seemed trivial in a single team responsible for a piece of software (be it a mobile app or a backend service) turned out to be nothing but in a distributed setup. How do you ensure the quality of your software? How do you keep your app running in production? Which team picks up the non-functional stories and epics? Who will hit the release button when there’s been a code change? And so on....
I took a lot of responsibility in ensuring continuity for the app’s API backend service (AAPI) in the new setup. Together with the group formed by backend engineers in the app family we set the following principles for the future:
- The codebase should allow maximising parallel feature development, i.e. multiple teams working simultaneously on different user stories.
- Backend engineers within the app family should safeguard the quality of the software delivered, for future development as well as operations.
Funnily, it wasn’t until I left the bol.com last February when I realised that I was aiming for the app family to obey Conway’s law:
Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
or, in other words:
any piece of software reflects the organisation that produced it.
Maximizing parallel feature development
AAPI is a typical aggregation service, combining data from multiple services into a single data model. It consists of about 80k lines of code and exposes data to the mobile app through REST APIs. Much of the API endpoints largely follow a straightforward pattern of controller - service - adapter, with services calling functionally related repositories as well as some others required to for composition or aggregation.
For example, AAPI’s search api offers a search results response based on a user’s query. AAPI calls the search API within the search domain to fetch results, composes them with sponsored content, and aggregates the outcome with the appropriate product information (images, reviews, promotion information etc.)
The codebase was set up as a multi-module maven application and architecturally packaged by layer, meaning the code was organised based upon its technical capabilities, e.g. all API controllers were placed in the same module. The service layer was the odd one out and was split in separate functional modules.
It became evident that changes to a layered architecture result in changes across all layers, which in turn leads to relatively big, scattered merge requests. That was an issue engineers had been experiencing for some time and now it only became more troublesome as multiple teams were handling the codebase. To make matters worse, it was easy to accidentally change common logic (i.e. generic logic, used by many different code paths) and hard to predict whether changes applied would have side-effects on other parts of the codebase.
During the project, this setup was transformed to a package by component architecture, such that code was organised from a functional perspective. Components were created for search queries, wishlists, baskets, etc.. Common logic was extracted to ‘starter’ modules to encourage similar code style and organisation throughout the codebase; features were isolated as much as possible. We considered component independent domain models (e.g. language and country), common error handling for API’s (e.g. how to deal with expired sessions or unforeseen errors), helper methods to easily create http clients to call upstream service, and some more. Any new changes to these starter modules stand out immediately (they are in unique modules) and should be executed with care. To ensure the new architecture remained intact we enforced architecture rules within the codebase using ArchUnit tests.
Overall, we found that the benefits of a functionally organised codebase include higher cohesion and lower coupling. In addition, related code was both easier to find and to map onto the rest of the organisation. For example, the search-component of the app is a largely isolated piece of code now and the only one able to communicate with, and understand, the search domain. Similarly, this goes for shopping baskets, wishlists, and others. Most of these components make use of more generic modules like the ones responsible for serving product reviews and images, but they do so through their respective interfaces.
Other benefits include:
- Better foundation for the future; bol.com, moving towards distributed mobile app development, is adequately supported from a code organisation perspective.
- Clearer separation between shared logic that is used in multiple places vs specific, functional logic.
- Optimisations for merge requests (better isolation, clearer scope, less conflicts).
- Multiple bug fixes and improvements thanks to uncovering numerous peculiarities in the code during the transition.
This concludes the fourth part of the mini-series on my role at bol.com for the past 2 years. Curious for more? Keep an eye out for part 5: Conclusion, learnings & next steps.