How tech oriented should a Business Analyst be?

Setting the Context

In January 1970, Dr. Winston Royce presented the waterfall way of software development at IEEE WESCON. In the beginning, he explained two essential steps common to all computer program developments, regardless of size or complexity; there is first an analysis step, followed second by a coding step as depicted in the figure below:

Of course this figure only applies to small computer programs so he went ahead explaining the mechanics in the larger scheme of things and himself called out that although he believed the waterfall approach of software development is fundamentally sound, it is extremely risky and invites failure. The remainder of his presentation explained five additional features that must be added to this basic approach to eliminate most of the development risks; these features never made its way into our textbooks especially after the arrival of the Agile Manifesto in February 2001. However, the arrival of the manifesto also did not mean that the principles of waterfall were no longer valid; the approach was still sound and at least one principle was directly borrowed from Dr. Royce’s recommendations - Involve the Customer.

It’s important to understand the evolution that happened with software development between 1970 and 2001; so far, none had called out an explicit need for a Business Analyst. In fact, the International Institute of Business Analysis (IIBA) that now defines the profession for many of us alike only came into existence in 2003; and its first Business Analysis Book of Knowledge was published in 2005, the same year when Steve Blank published his Lean Startup movement. And in the simplistic world of software development, analysis still remains a step in the larger scheme of things.

Where are we going with this story?

Amid the Great Recession, I was among a few who were able to secure an internship with Deloitte and the first day was enough to make me realise that everything I learnt in college was outdated (except for the basic concepts). One of the days during my internship, my mentor handed over an Entity-Relationship Diagram for me to review; it was from a group of college graduates who were part of the new hires’ engagement program and they were to visit the office that afternoon for their presentation. That was the only day when I felt my college education was worth it, for what I had received was a diagram with two entities joined by one relationship and a bunch of attributes, all for a fully functional hospital management system. It was a satisfying experience for me to convince my mentor that instead of their presentation, we could take that time and co-create a decent Data Model with them and in the process share our knowledge. And that’s what we did. The next six months went by learning new skills and becoming a decent programmer in the process. Six months later, I secured a job as a Business Technology Analyst; I never really understood what this term meant, afterall all I was supposed to do was code.

In 2011, I was working for a client who provided short term annual loans to its customers. This product consisted of several modules including Loan Creation, Loan Disbursement, Scheduling, Tracking & Reminder, Repayment, Auto-Debit, and Closure which formed the complete life-cycle of my client’s business. A major portion of my job was defining the business logic for the PL/SQL programmers in specification that was understood by them. This involved creation of Logical Data Models, something that my fellow Business Analyst and I were well-versed with. We did well with our product and the account grew to a 40 people team. This required more Business Analysts on various modules and not everyone was fascinated with the idea of providing technical solutions for development. It took quite a bit of nudge to overcome this inertia and made a few analysts unhappy.

In 2013, I got engaged with a new client, a STM Publisher - the Scholarly Publishing Industry was going through a major shift during this time, the product, pretty straight forward, a website. A website where you can browse the latest journal articles, books, and chapters with a major shift towards better searchability of content. Until very recently, full-text scholarly articles were only available as PDFs but this era was moving towards HTML and the medium for content exchange was XML. Why XML? It’s very common for publishers to buy and sell journals based on their demand and profitability; so it’s not uncommon if your favourite Physics journal which was hosted with IOP Publishing until last year lands up in Springer Nature this year. XML based exchange seemed the simplest way of moving this huge dataset with thousands of articles. This required storage is XML (NoSQL) Databases, in our case - MarkLogic. This project was about storing & retrieving articles & books using their correct XML tags for content - full-text, front & back matter, citations, bibliography, and other taxonomy - based on a standard like NLM. Hence, the outcome of our analysis was the details of parsing this XML to HTML for an online version of articles and it wasn’t an attractive work for a few fellow analysts.

My stints with STM Publishers continued and with a new project entered Documentum - an Enterprise Content Manager. Developer’s expectations from a Business Analyst increased by a fraction, this time to ensure that the solution was carved within the boundaries of a tool - Documentum xCP. It is during my conversations with my analyst team members that I noted a pattern; many exclaimed that it was difficult for them to perform efficiently as a Business Analyst because the environment was very technical in nature. In order for us to navigate in this highly technical environment, we needed a Technical Business Analyst, someone who could translate the functional specifications into technical specifications for the Development Team. For e.g. this meant Business Analysts who understood working with APIs, building Data Models, performed intensive Data Analysis, well versed with UMLs if needed, etc. Hence, after careful consideration, we adopted the mechanics similar to SAP, having a group of Business Analysts, a group of Data (Business) Analysts, and a group of Technical (Business) Analysts.

Whats the point?

While you read this story of individuals who evolved from “technology oriented analysts” to “business oriented analysts”, I view it as individuals who evolved from “analysts with an affinity for technology” to “analysts with an aversion for technology”. This may sound unsettling since it goes against the beliefs of many, the simple matter of fact is that during this evolution, the only aspect that has stayed constant, is the Business of Software Development. The one thing that Agile Software Development has reinforced is that “Working software is the primary measure of progress” and there is first an analysis step, followed second by a coding step. At a bare minimum, the only individuals required for software development are programmers with analysis skills.?

One may argue that the business of software development itself has evolved and now we understand the benefits of individuals who bring in a business viewpoint that makes our software better suited for our customers; but this viewpoint (Involve the Customer) has existed since before 1970 including many other concepts that are appreciated even today. So what formulates a good reason to stay away from technology when one is in the business of producing technology?

And there’s definitely an argument that if Business Analysts are affiliated with specific technology then they may carry a confirmation bias; the tendency to seek out information that supports something one already believes in. This may result in our solutions getting restricted by the underlying technology for e.g. the way I was limited to find solutions within the boundaries of Documentum xCP. That is a possibility, however, if the user / customer is involved at every step of development who are oblivious to technology and focused only on solving their problem, this bias gets neutralised. Let’s also not forget that a Business Analyst’s job is to create a shared understanding; it’s essential to translate the business problem into a technical problem to create that shared understanding else the analysis isn’t complete.

As an analogy, let’s consider the evolution of Programmers; in our present context there are a few different types of programmers that exist: Front-end Developers, Back-end Developers, Full-Stack Developers, etc. If provided with an option, one inclines towards working with Full-Stack Developers however if we don’t have one, our non-Full-Stack Developers may work on parts of the technology they don’t specialise in, even if it doesn’t deliver the best solution. And since “Working software is the primary measure of progress”, a non-Full-Stack Developer who denies this responsibility in the absence of a Full-Stack Developer, is usually not considered for a team. The reason isn’t that it’s a brutal profession, it’s because of the underlying truth “it’s not about you, it’s about the software”.?

This rule applies to any individual on a software development team, our skills essentially fill the gaps required to deliver good software. At times not all skills are available and relevant individuals own up portions to fill the gaps in order to continue delivering software. This is where we diverge from the dogma; the common belief is that a Business Analyst performs the functional analysis of a system for example, given a User Story, a Business Analyst is responsible for identifying the Acceptance Criteria that fulfills the User Story. However in our bare minimum scheme of things, one cannot code unless analysis is complete which would include technical analysis as well. So who performs this type of analysis?

A typical answer would be a Technical (Business) Analyst however there’s another more appropriate answer, the Programmer. Since I’ve already mentioned that at a bare minimum, the only individuals required for software development are programmers with analysis skills, let's also face the reality, if there exists a customer who can explain software requirements to a Programmer, then we probably do not require a Business Analyst; this may be a controversial statement but this is the hard truth. So if we want the profession of Business Analysts to stay relevant in the realm of software development then it’s important that we take full ownership of that analysis step and this is where we circle back to Business Technology Analysts; if we’re not one, we’re probably not needed.

How tech oriented should a Business Analyst be?

This is still a choice of course, just like Programmers have a choice to be Front-end, Back-end, or Full-Stack Developers, Business Analysts also have a choice. However since working software is the only true measure of progress, as a Business Analyst our choices may be more constraining than advantageous. However there are ways to overcome these constraints:

• Have an intermediate knowledge of technology usage and software development; understand the value stream of your software development team and fill the gaps to improve the overall throughput. Get acquainted with the technology needed to deliver your solution, be it writing and executing database queries, understanding the functioning of APIs, the general architectural landscape, or just the possibilities based on the nuances of your underlying technology e.g. what can you do with Spark or Kafka.
• Work closely with your development team to understand how your technology is implemented; no one’s expecting Business Technology Analysts to be technology experts, however being close to technology can help you with your discussions with the customers as well. Utilising the knowledge of your team helps because the Business Technology Analyst not only flows knowledge from the customer to the development team but also the other way around and it only helps if this flow of information becomes faster and more efficient.
• Get used to learning technology based practices and having a basic knowledge set to support your teams. Although this may not sound like an industry wide norm, this is the very nature of a Business (Technology) Analyst. Even the latest version of BABOK published by IIBA mentions a bunch of practices that must be present in a Business Analyst’s repertoire, for e.g. Data Mining, Concept & Data Modelling, Data Flow Diagrams, various aspects of UML, Interface Analysis including APIs, etc.

This list doesn’t end here, technology is ever-evolving and this only means that the practice of Business Analysis is ever-evolving. The underlying principle remains “if you are in the business of creating technology, you cannot stay away from knowing technology”. Be relevant, be a Business (Technology) Analyst, it wouldn't be helpful if our results do not take us at least a step closer towards the outcome.

P.S.: Modern Analyst explains various types of analysts in an article here; although I don't align with the same idea it's something that's prevalent in the industry.