The Organization Design Challenge in Managing IT: Are we Setting up IT to Fail?

If you have ever visited London, you will be aware of the “Mind the Gap” signs that can be seen right across the city’s underground transport system. For those who are not familiar with the London underground, these signs warn passengers of the gap between the platform and the trains. Anyone who have been involved in IT for any length of time will be aware of the existence of a somewhat similar gap in most organizations, the so-called ‘IT-business gap.’ In fact, in some companies the distance between the two camps is so great that a more evocative word than gap is probably required!

So how does this ‘gap’ manifest? Talk to anyone from the business side and you will likely hear a chorus of disappointment with the returns from IT spend. There will be talk about IT projects being late, over budget and not delivering expected business outcomes. Employees might point to complexity in operations, hardwired by technology applications, compromising agility and making even the seemingly smallest of changes difficult and expensive. They will probably make reference to a specific system that they deem as being difficult to use, not fit for purpose or just useless. To them, it seems IT makes everything so much more difficult than it ought to be. It is not known as the Department of No for nothing!

Talk to IT staff and you will hear comments like their colleagues in the business just don’t know what they want; that they say they want one thing and then, when delivered, it’s not exactly what they needed and proceed to place the blame for this situation firmly at the door of IT. They will point out that the business does not invest enough money in IT; that they pay for a ‘Mini’ but expect a ‘Ferrari’ service. They will very likely express the view that the business expects mountains to be moved and just don’t understand what’s involved in managing a complex IT estate. They are also likely to vent their frustration at the lack of any real direction from their counterparts in the business and that they either don’t get involved in IT or, if they do, it is reluctantly and without any real conviction.

Both sides would probably agree that despite their best efforts – and make no mistake, both have the best of intentions – technology just isn’t supporting the business it the way it should or could. The gap between both can reveal itself in relationship problems, although there is probably a catch-22 situation at play here: little or no meaningful engagement between business with IT staffs leads to the inevitable consequences, succinctly captured in the above paragraphs, which then becomes a self-reinforcing cycle.

To begin to understand the reasons why these problems exist, we need to explore the genesis of the gap, or at least why there are two camps in the first place. Its origin goes right back to organization design choices, themselves framed by a particular design brief. In this article I argue that this design brief is fundamentally flawed and that it leads to a focus on addressing the wrong problem. The consequential design choices merely set up IT to fail.

The IT box on the organization chart

There was a time when establishing an organization unit and assigning it responsibility for specifying, building and maintaining all the tech the company needed was the design response to manage computers. And it worked out OK. Even if there were occasional glitches, by-in-large, everyone involved was reasonably happy. Anyway, managing all those computers and data centers was seen as a back-office function, supporting what were essentially considered back-office or non-core activities. This was the lens through which technology was viewed at the time. Computers were new. Expectations were low; well actually nobody really knew what to expect. Knowledge of anything to do with computers outside of this unit was practically nonexistent. IT experts were the masters of their own universe; kings of their own castle! This was the priesthood of IT!

As we all know, over the years the role of IT has changed significantly. Today, few, if any, organization could function for very long without their IT systems. The current COVID-19 pandemic has demonstrated this dependence on tech. Many organizations are digitizing their offerings and introducing new business models shaped by data. As more and more business moves online, tech is increasingly being used to engage directly with customers and ecosystem partners. Information has been christened the new oil and is now (at last!) considered a strategic asset. Despite advice going back decades imploring companies to consider information and IT as a potential source of competitive advantage, few companies took this on board, content with having technology support their core operations. Today, this is no longer an option: IT is a competitive necessity.

Despite this seismic shift, most organizations continue to approach how they harness IT with a broadly similar organizational design approach. They establish an IT unit, staffed by employees with considerable technical know-how, and assign it responsibility for the management of IT. And in today’s digital world this just seems like an obvious thing to do. Just look at the organization chart of any company and you will invariably see a box labelled IT. But with this design choice IT is already on the path to problems.

As executives tire of IT not delivering on their expectations, which inevitably occurs with this design, the pendulum tends to oscillate between centralization and decentralization. If IT spend is judged to be out of control, perhaps due to a sprawling architecture, it is reined it with a more centralized approach. If IT is deemed not responsive to local market conditions, the reins are loosened with a more devolved attitude. To respond to what some called ‘digital’ a more nuanced ‘two speed’ or bi-modal approach to running the IT unit. Different variants of these basic models have appeared, such as pairing local relationship management structures with centralization to accommodate local interest in decisions made at corporate HQ. It is probably the middle ground of a federal model that seeks the best of both worlds. With a separate unit responsible for IT, the ultimate sanction for a problem IT unit is outsourcing to a vendor. After all, tech is usually not deemed core and the vendor will have more experience and track record looking after what is in “the box.”

IT as the problem child

Why can’t IT just get on with it is a frequent retort I hear from exasperated business executives! Notice that the problem is invariably expressed as one to do with IT and the IT unit; that it is IT that somehow needs to be fixed. But is there something different about IT that makes it so problematic? Surely we can have an organizational unit focus on the IT-technology bit, the rest of the organization then concentrates on using this tech. Logically, one side does all the “techie stuff” and delivers in the required tech capability. The other (i.e. the rest of the organization) specifies what it wants and then uses it. What could be simpler?

It can be argued with some confidence that other functional areas manage to operate successfully this way. Take for example, the Manufacturing unit, typically responsible for producing a company’s products. Once it is agreed what has to be manufactured, in what quantities and by what due date, the Manufacturing Director can essentially get on with the task of managing production. Everything she need to do this, the staff, the machines, and the know-how is under her direct authority. When produced, the product can then be shipped to customers by staff from the logistics and distribution area. “Value”, from a manufacturing perspective, is created by converting raw materials input into a finished product – of course we could argue that value is ultimately not created until paid for by the customer, but let’s not enter into this debate. The point is, it is very clear what the head of manufacturing is and is not responsible for.

Of course, the Manufacturing Director has no control over sales and would never be held accountable for product sitting in inventory or lying on the shelves in retail stores. That is the responsibility of other colleagues, possibly in the Sales and Marketing functions. If input materials fail to arrive at a factory on time, this is clearly an issue for procurement staff, assuming the purchase of raw materials is the responsibility of a separate procurement function. The Manufacturing Director is, however, accountable for any quality issues that may arise in the finished product, although this concern might possibly be traced back to the original design of the product. Maintaining the workings of the production line, minimizing downtime and maintenance, falls under her responsibilities. Improving production processes through continuous improvement initiatives also falls under her remit. For a global organization, distributed manufacturing sites add a layer of complexity and risk. However, this does not distract from what the Director of Manufacturing can and cannot be held accountable for. The role can be precisely specified, appropriate metrics assigned and performance assessed.

To get on with her job, the manufacturing director essentially manages around the boundary of her organizational unit, coordinating manufacturing activities with product design, sales, marketing, procurement, logistics, distribution, etc. The logic behind this dominant organization design is that each functional area can essentially get on with its assigned role, as long as there is a certain level of coordination between them. Achieving this means managing at the interface with other units. Mechanisms of coordination are well established. For example, if demand for products exceed production capacity, the Manufacturing Director will look to execs from other areas of the business to prioritize customer orders; it is likely that the head of sales will make this call. Factory capacity is finite, and this can be easily demonstrated. The time to bring new or additional facilities on stream can be calculated with some precision and priced accordingly.

Applying this logic to IT means that the CIO, as head of the tech unit, merely has to focus on building and maintaining systems. They can appoint relationship managers to liaise with colleagues across the business to understand their requirements and business needs, their priorities and plans as well as keep them informed of new technology innovations. But we know that this approach doesn’t work. If it did, IT outsourcing would be a piece of cake. Yet we don’t hear the same level of disappointment or frustration with contract manufacturing, for example, as we do with IT outsourcing. If this design logic doesn’t work for IT, is there something else at play here?

Setting up IT to fail

The genesis of the problem is that the organization design brief for IT is framed as how best to organize to manage technology. In responding to this brief, it is designated that all the necessary knowledge to achieve this is housed in a distinct organizational unit. This naturally leads to establishing an IT organizational unit. But making this design choice is a direct consequence of misunderstanding the real problem the organizations have with technology therefore misstating the design mandate. This brief should be framed as how best to organize to deliver value from technology not to manage technology. This might seem like a subtle difference but it is in fact highly significant and has profound implications for design choices.

In contrast to corralling all knowledge into a separate organization unit, the delivery of value from technology requires bringing together and integrating knowledge from different knowledge domains. This knowledge is distributed right across the organization in different areas of the business. Significantly, this know-how is not under the remit of any one individual, in particular the CIO. Therein lies the fundamental reason why organizations struggle to deliver value from technology.

Organizations are slowly waking up to this fact and beginning to adopt different approaches. For example, an increasing number are establishing so called “agile teams,” bringing together employees from diverse knowledge backgrounds to tackle problems and opportunities. These multi-disciplinary teams work in short sprints, eliciting frequent feedback from users to continuously deliver solutions to problems. This practice is in direct contrast to what previously occurred, where an attempt was made to first capture user requirements before beginning the process of building something to address these with a solution.

This latter practice is in response to the need to coordinate the activities of the IT unit (remember, it is separate organizational unit) with other functionally distinct areas. The example of an agile team suggests that, rather than creating a situation where coordination between an IT unit and other organizational units is necessary, the question becomes how do we create structures, mechanisms or environment that facilitates the fusion of diverse knowledge domains from across the organization.

Fortunately, there is already an established concept in the organizational and management literatures that accommodates this. It is called a capability. Thus, the argument being proposed is that to deliver value from technology, organization must build relevant capabilities. Many readers might say this is obvious, but perhaps have not considered the consequences. With employees (i.e. their knowledge) underpinning the manifestation of these capabilities distributed across an organization how can this be achieved? The task cannot be assigned to a CIO to build these capabilities in a way that a Manufacturing Director can build and organization’s production capabilities; remember that a manufacturing director will have authority over all the necessary resources. Nor can the answer be found in the convention of establishing coordinating mechanisms for inter-unit alignment.

Organizational capabilities to deliver value from IT

A capability represents an organization’s ability or capacity to deploy the knowledge and experience of employees to accomplish a given task or achieve an outcome. For example, designing a new digital banking product can require knowledge from banking professionals, product managers, UX professional, designers, compliance professionals, mobile app developers, etc. As such a capability is the ongoing process that is enacted through the everyday and ongoing work of employees that is embedded in modes of behavior, informal networks and personal relationships. Organizations can thus be viewed as a collection or portfolio of capabilities to get things done. 

While knowledge is “owned” by the individual, the integration of this knowledge to create knowing and understanding and get thing done takes place at a collective level: in groups and other cooperative forums, some of which are formal others informal. Although specialist knowledge is sometimes required to accomplish tasks in an organization, there are many situations in which individuals with specialized knowledge must represent their knowledge in a group or other collective to solve a problem, make sense of a situation, take a decision or merely help in getting something done. This is why conversations, discussions and debate are at the heart of a capability. Indeed, we know that the best indicator of alignment between IT and the business (excuse the language!) is when there are regular conversations taking place between senior business and IT executives (again, please excuse the language!).

Galvanizing and converting knowledge (residing in employees’ brains) into capabilities is an organizational wide endeavor. As emphasized throughout this article, the challenge in delivering value from IT is to bring together those with the relevant knowledge to create the necessary knowing and knowhow. This is much easier said than done and it points to the fundamental difficulty of delivering value from IT. The functional silos in organizations make this incredibly difficult to achieve.

This bringing together of employees can be contrived, as when we put together an ad hoc team. It can also happen naturally when staff reach out to colleagues to fill deficiencies in their own knowledge or understanding. Of course, a willingness to cooperate and an ability to work together are fundamental conditions that must be met. And perhaps an awareness that the person you are reaching out to actually has the knowledge that can be of assistance. So there are behavioral attributes outside of a person’s knowledge that are also at play here.

Identifying and building relevant capabilities

Pick up any article or research report in the broad area of digital transformation (I include analytics, business model innovation, ecosystems, etc.) and you are likely to find the conclusion peppered with recommendations listing out various capabilities to be developed. I have previously criticized this practice, not because I disagree with the prescription being proposed or the capabilities identified, rather my concern is with the imprecision with which capabilities have been defined and the lack of any real guidance as to how suggested capabilities can be built. The latter point is not surprising as identifying the specific knowledge underpinning any particular capability is incredibly difficult to do.

There are so many questions where we don’t have complete answers to. We don’t know the complete portfolio of capabilities needed to generate value from IT; I have attempted to do this in the past. We don’t know how they fit together. We don’t know how the knowledge that underpins them can be harnessed. We don’t know how practices contribute to the manifestation of a capability. We don’t know whether there is a sequence in which they should be built.

What we do know is that we need new concepts, blueprints and designs to help in achieving this. We are seeing new constructs emerging. For example, new platform-based organization designs or the creation of internal entrepreneurial entities. Management scholars are coming up with ideas about how to bust bureaucracy that have relevance for this discourse. The research on new ways of working are also signaling new approaches. Some companies are using the concepts of ‘principles’ and ‘mission’ to align and drive performance.

A small number of pioneering organizations are already throwing off the vestiges of the past. A global energy company is currently implementing a plan to eliminate its global IT unit. Despite being fundamentally dependent on technology, a mobile-only “challenger” bank has built an organization with no IT department. The CIO of an automotive component supplier is on record as stating that over the next 5 years everyone in the company will work in IT. An Asian bank is now organized internally around platforms rather than traditional banking departments. What all these organizations have in common is a realization that traditional approaches to how they organize to leverage technology and generate real business value are not working and new approaches are required.

  ******

Creating an IT unit is the result of working from an organization design brief that seeks to manage technology. In this article I have argued that this is to misunderstand and misstate the problem: the real challenge is to design an organization to deliver value from IT. With a separate IT unit, the leadership team attempts to bridge the gap with the rest of the organization with various coordinating mechanisms. While this approach works for other functional areas, it doesn’t work for IT. All it serves to do is set up IT to fail. Delivering value from IT is not a coordination problem but a knowledge fusion one.