Of onions and trains: Kill the MVP!

At the turn of the previous year I published a rather technical post about the benefits and challenges of real-time streaming in a larger organization here on LinkedIn. I reasoned that we should no longer perceive data as cell values in spreadsheets, but as a constant stream of insightful signals flying through the company. To gain some street credibility, I even added a little prototype showing my ideas as working software (with the great support of Ivan Burda ).

Throughout 2022, I kept pushing for that paradigm shift in data processing: At Erste, we kicked off several projects in various setups - from tiny to medium -, all under the common denominator of data processing in real-time. Whereas some of these initiatives show first successes, most of them prove to be challenging in certain aspects, probably also because I depicted them too ambitiously in the first place.

Despite these challenges (and a few true set-backs), I remain extremely bullish about our portfolio: We introduced new tools, started to rely on SaaS/cloud offerings, reshuffled a number of processes and responsibilities, and - most of all - ignited the idea of real-time data in a larger audience throughout the organization.?

Innovation is hard

Last year, I had the opportunity to talk to Jay Kreps - creator of Apache Kafka at LinkedIn and now CEO of Confluent (Nasdaq:CFLT). Especially one of his statements stuck with me: We asked Jay how long it took his team to invent and use Kafka. His answer was insightful, because it showed the gap between initial tech innovation and true adoption:

"It took as a few months to come up with the first working version of Kafka. But of course, it took years to convince all the teams at LinkedIn to switch over."

Engineers in the old economy often picture tech companies as the Promised Land - places where THE one single best fitting solution will be immediately and fully adopted by the whole enterprise. From own experience I must disagree with this version of the grass is always greener on the other side: Yes, tech companies are amazing environments regarding their capability to ship increments fast, they usually establish meaningful feedback cycles between the actual users and the engineering teams, they build cultures around sharing ideas and solutions, they tend to make decisions based on a deeper understanding of technical reality, and thus they're probably less engaged in playing the corporate politics game. However, also tech companies are full of humans with their own goals, likings and fears - and even the hottest organization eventually has to fight Dunbar's Number. As organizations often get bigger and certainly older, the growing amount of mess they're piling up simply is inevitable.

It's wrong to perceive innovation as something one could isolate to a single technology decision. Because even the simplest innovation requires a bunch of activities to take stuff the whole way from idea across the finish line. (And even this line lies way further down the road than one might initially think: A successful deployment only means the starting point for taking measurements, validating impact, incorporating feedback.)

My "onion", a model for clustering innovation

Thinking about making change happen, I suggest to look at my onion below: Any digital product consists of various layers, from the user interface on the very outside down to the actual reason of existence, a human's motivation to use the product. The layers are of course interconnected. Yet, they show different manifestations, undergo different lifecycles, follow totally different mechanics when force is applied. Such forces can be mapped to the usual suspects in digital teams: For instance a designer predominantly touches the user interface, some business stakeholder thinks of the back-office process, etc. These roles usually come in pairs which makes communication amongst them easier, but communication with other groups potentially much harder. (And yes, it's a simplification! A good UX person will certainly look at the underlying process, but remember: We're talking about oil tankers, not speed boats.)

The overall complexity of making any change - complexity which may surface as higher cost, longer throughput time, a growing amount of unpleasant conversations, etc. - will be a function of the layers one needs to cross. In other words: Any idea which is nicely isolated within a single layer can be much easier turned into reality than the ones, where you would need to cross a lot of boundaries between those layers.

And since I started this post from the angle of data processing: After two years of being solely focused on data, it is a hard topic because it lies exactly between business and engineering, and it can almost never be decoupled from the adjacent layers, be it the business process, or technology and tooling.

Moreover there is an additional force (inertia, actually) in place: The layers get more dense towards the inside. I've seen impressing iterations on the UI done within a few weeks, it may take a decade to see changes in people's behaviors. However, I bet upgrading the inner layers gives you more competitive advantage over your competitors, and this for a much longer time.

Bottom line: I suggest when dealing with any type of change, try using my onion. Looking at change from this perspective might give you a rough idea about the complexity involved. When mapping layers to your organizational setup, you may even locate risks that usually prove hard to spot.

So, how shall I innovate the onion?

The lean startup literature is full of valuable advice: "Ship small increments", "If you're not embarrassed of the first version launched, you were too late", "Release often, release early!", etc. The problem with these tips is that they all assume you'd be in control of the whole onion, or at least of most layers thereof. Yet, this never seems to be the case in a large organization.

But don't worry, you have two remaining options: You can either wait for one of the next Agile Transformations your corporate will be doing. You might be lucky and get in control of more layers. Or, you can nagelmacker your way through the organization. (Don't look that verb up; I've just invented it.)

Georges Nagelmackers, my onion-cutting hero

Georges Nagelmackers established international train connections throughout Europe in the late 19th century. His vision was a luxury train service spanning the whole continent - North to South, East to West. His service later became famous as the Orient Express trains we all know from Agatha Christie's novel (or one of the movies). Yet, at his starting point there was no such thing as a cross-border connection servicing long-distance travelers. Trains had heterogenous technical equipment and tooling, fell under different legislations, locally trained personnel was required to operate them. It was sheerly impossible to move a single locomotive from Paris to Vienna and onwards to Istanbul.

Yet, he started by cutting the onion of the railway problem in a smart way: Instead of creating the whole train line in the first place, his company (CIWL) built luxury sleeper carriages, later also restaurant cars, and attached them to existing lines of local train operators. With growing success he could take over more and more of operations, and use his own trains. (Yet, the Orient Express between Paris and Constantinople still had been rather a myth than a reality for a very long time - ferryboats were used to cross the Danube between Romania and Bulgaria, ships to travel the last few hundred kilometers from Varna to Istanbul. Eventually the train line existed end-to-end, and CIWL further expanded to the hotel industry and became the world's largest travel agency after buying Thomas Cook.)

I find this story inspiring, because Georges' type of innovation is not about cutting scope down to the infamous MVP as usually preached in the startup gospel. Nagelmackers neither offered a luxury train service between Liège and Aachen (two extremely close cities) and would later extend, nor did he start with a luxury hike from Paris to Istanbul which he would later upgrade to horse-ride, carriage and finally train. His offering piggy-backed on existing infrastructure, smartly circumventing the constraints of technology and legislation.

The next time I revisit my portfolio of projects, I will look for ways to piggyback on established infrastructure that gets me across the finish line. And if it doesn't work out: It's okay if one has to cry from time to time when cutting onions. Would be interested in your thoughts!