A practical case study on identifying important technical design decisions and the cost of inexperience

Senior engineers and managers alike are expected to make technical design decisions choosing between generalizing or specializing solutions (I'll give you two of my fresh examples from the IoT and AI industry). A lot of decisions, in very different domains, fall in this category. Making the wrong decision can turn out costly. Hiring cheap and inexperienced staff can be both useful and necessary, especially for early bootstrapping, but it can also lead to the death of your company or project. Finding yourself in a dead end after investing heavily in a sub-optimal or inadequate solution is not uncommon but the risk of doing so can be reduced by having experienced persons in your team and taking time to reflect every now and then.

These decisions can sometimes be hard to identify. For a practical reference we can take a look at two such decisions that I'm currently facing. The two decisions are at very different levels in very different projects but both can turn costly if they are not correctly dealt with.

Project summary:

Creating a platform for IoT analytics marketed as insights and alerts as a service to customers in facility maintenance and real estate.

Decision:

The specialized solution of identifying and using a cloud based IoT platform suitable for our current needs allows us to get started quickly and reduces a lot of maintenance associated overhead. The generalizable solution is to build our own IoT system in e.g. AWS using low level components such as IoT Core, IoT Analytics and QuickSight. This will hold us back from market for at least an extra month even for the simplest MVP but gives us the flexibility to grow and organically change the platform as the needs changes.

Potential cost:

Getting locked in to a third party system can make us a stale company not able to act on new opportunities or needs from our customers. Once our customers are onboarded into the third party system the cost and process complexity of making the change rises dramatically. Some customers, just having learned the system, might not be open to moving to a new platform. Running dual systems side by side will not only be expensive from an infrastructure point of view but also from a maintenance and support perspective. Furthermore, manual migration of a large number of onboarded IoT devices is costly, often even requiring an integrator to go out to each site to physically connect with the device.

Project summary:

Development of a video analysis pipeline using neural network inference on a multi-camera stream. The pipeline will be a part of a clients research platform.

Decision:

The generalizable solution is to write small code blocks and methods specializing on small problems. The complex problems are solved by combining multiple of these small components. This does however often mean iterating over large data buffers multiple times and recomputing some common sub-results more than once leading to a less performant solution. The specialized solution is to unroll the code blocks and interlace them into a function that does all the needed calculations with a single, or as few as possible, iteration over the data buffers. This can in some cases increase the performance an order of magnitude or more. The code does however get hard to read, maintain and reuse.

Potential cost:

Refactoring of code is expensive, it is a time demanding job resulting in no direct gains. Due to the large risk of introducing bugs during refactorization a well built test infrastructure with both system level tests and unit testing is required. The introduction time for new employees will be much longer with a lot of highly specialized code in comparison to modular code built with abstraction layers. Specialized and interlaced code is hard to isolate enough to write unit tests. Overall results might be a reduce in the development pace and code swamps leading to a stale product which is impossible to further develop.

Take note that the case studies above makes no recommendation of what decision is the correct one, such a decision requires granular understanding of the system and the use cases and should if possible be based on measurables like benchmarking or KPIs.

If you feel that these are interesting or important topics you are welcome to reach out to me to discuss further.