Solving the Architect/Engineer Paradox

Our industry (business/technology) is carrying around a whole set of paradoxes and unsolved problems. Not business problems, not technology problems but industry problems. By industry problems I mean that they are problems with the way technology, technologists, employers, clients and government and education behave, ie the Industry (big I*). Think of them as the infrastructure underlying how technology as an Industry behaves and delivers value. I call these paradoxes even though the are not impossible to solve (some of them are easy in fact) but why they are ignored or treated as unique IS a paradox.

This post is focused on our new focus on Software Architecture. We are launching our Software course in Oct. I am extremely excited about this course and believe it will quickly become THE course for software architects. This question is becoming one of our focus areas at Iasa for 2022.

Laying Out the Core Problem

I started Iasa because I am interested in this infrastructure. I have many posts about this topic so I wont belabor the point here. Just realize we are the ONLY Industry to deliver complex life altering things on the planet that does not work. Simple as that. The way we do things now, simply does not work anymore. We are also the only industry that seems to escape the consequences we cause. How many developers, SIs and vendors go to jail because the system they worked on kills people (yes Elon, I mean your self-driving deaths as well... how is programmer or AI error anything other than murder?)? For less exotic and extreme examples, consider the meltdown of the Irish Healthcare system. The ransomware attacks. Home Depot lost millions of credit card records... their stock price was only affected for a week!

Getting re-centered on a more desirable outcome focus (I hate people who burn and don't offer a solution), these industry problems include many fallacies that the general public, employers, education institutions, and clients accept for no apparent reason and are solvable mostly by stealing from other practices related to complex systems (homes, spaceships, skyscrapers, cars, transportation, energy, and my favorite, medicine). In all these fields the shape of the industry includes all of the primary groups mentioned before and in most of them there are clear distinctions between corporate and individual responsibility and liability as well as very clear building codes and enforceable ethics. If you want to become a building architect or doctor, there is a very clear, if not easy, career pathway. It includes straightforward bodies of knowledge, mentoring, education, experience, certifications, university accreditation etc.

So What About Architecture, Engineering and Development?

In our research over the years one of the most critical paradoxes is the continued confusion over the boundaries between architect, engineer and developer. I separate these into three even though there is no industry-wide adoption of particular boundaries. For the purpose of this argument think of developers as self-taught and focused on smaller problems (no I don't mean title I mean competency!), engineers as trained and mentored but focusing on systemic structural elements and architects as outcome driven, trained and mentored creative and innovation focused with strong overlaps in depth skills with their counterpoints at certain points but with a focus on breadth and rounded out competencies. Thus the scene is set.

So what is the solution? Short answer is anti-hype. It is not more rounded people everywhere, but more disciplined and competent specialists, engineers, and architects. It is rigorous training and mentoring. This opinion is not popular, nor is it realistic in the short term, there is simply too much work globally to go from zero to mature overnight. However, the answer does not change no matter which way you argue it as doctors, accountants, lawyers and 'real' architects have learned. Read the histories of those professions. While the may have been 'practiced' for thousands of years, they were not really good and stable until they figured this out. People died as often from going to see a doctor in the 1800s as just staying home unless it was extremely basic (think setting a broken arm - but if you look deep you will find poultices gave infection as often as staving it off).

What Are the Areas of Overlap?

This is what we at Iasa are dedicating 2022 to truly unearthing. We know the shared competencies between a software architect and a lead engineer/developer. But we do NOT have industry wide data that compares their roles, responsibilities and deliverables. It gets worse as you progress through other types of architects (specializations). DevOps is a moving target to get developers and operational personnel to work together, then make it DevSecOps, DevAISecOps, DevAISecDataBizOps... you get the idea. Pretty soon we are at the entire technology group in a 'team'.

Here are a number of the things we think we know from anecdotal evidence:

• Products are more valuable and delivered better when there is a competent lead architect and competent lead engineer working together. Again not measured by title, but by competencies (see the BTABoK competency model).
• Software/Solution Architects can grow to a point where they are too high and are no longer creating value but instead doing something akin to strategic management. They could no longer deliver a complex product as well as a lower level solution architect. This appears to be directly related to mapping a profession onto tradition HR and management promotion techniques. For example, a heart surgeon is not 'promoted' to hear surgeon or chief of surgery. They must earn it through years of experience applying and demonstrating success... not to their employer but according to the profession (research board certifications and advancement in medicine, it is fascinating). All the normal politics apply in a hospital, but they are tempered by the experiential and mentoring facet of the profession.
• Engineers/developers are focused on delivering working systems based on requirements. They focus on how or what. Architects are focused on WHY and by derivative how/what. There is a strong overlap with architecture/engineering on one area, quality attributes (also known as non-functional requirements - which is the worst term d'art ever, as QATTs are absolutely functional in nature).
• Each company randomly redefines engineering, architecture, and development for itself roughly every 3-5 years in a cycle. I have seen hundreds of companies in the last 20 years make dozens or hundreds of people 'architects' with a simple (not so simple) re-organization and re-title. This is possibly the most effective way to destroy quality delivery in the entire Industry. I am working on a post called the great lie. I have seen thousands of 'architects' created because it SOUNDED better to customers. It is like making all of your desk administrator at a hospital 'surgeons' and handing them a knife... Cut Away! they say. No one will call us on it and we are making more money than ever!!! BTW this is not just vendors.
• Engineering and development is not less than architecture. It functions with a different purpose, outcomes, measures and goals. Architects are often considered 'unicorns' because we keep trying to make engineers into architects. The aptitude and the attitude and most importantly the interest is not there in most cases.
• Architects are a scarce resource and we have managed our time very poorly. We tried to migrate to the top instead of shoring up our knowledge of what makes great architects at the 3-5 team level, then grow success from that science.
• It takes significantly longer to roll out proven methods than seems to be necessary due to the variability of competencies and the corporate-as-an-independent island of practice. Other professions have tried and tested methods to explore and experiment with new practices and then adopt them rather quickly when they are proven to work. Covid treatment is a great example of this, but so are new methods in accounting, law, building etc. Only in IT do we continuously bicker, debate and defend instead of creating Industry wide evaluations of methods, then applying them and feeding them back into a body of knowledge.

The last bullet point is what we are putting our effort into for the next couple of years; researching what is working and why and how it might be tied through example to the overall body of knowledge.