Gartner Software Engineering Research - September Findings

Hi Folks.

For the last 6 months or so, I've been sending out a monthly email to my team where I go through each content item we published that month. I write about what I find interesting about the research, and I sometimes mix in personal stories. I've been told that simply sharing this internally as an e-mail is a miss, because others might also find it interesting. I am not convinced, but I am willing to experiment. So, below are my September Research Findings, along with links to each item. And yes, I read every content item we publish, and no, I do not use ChatGPT to write my comments.

In the last 4 weeks or so, the Gartner SWE team has published 18 content items to help our clients make great decisions, including a mix of MQs, actionable guidance on both leadership and technical topics, and what I think is our final raft of "videos from the Vegas Dungeon." (IYKYK).

?Assessing Backstage for Your Internal Developer Portal ?(August 26, Gary Olliffe)

Internal Developer Portals are the latest trend in self-service, enabling platform engineering teams to improve developer experience and ultimately developer productivity. Backstage, originally developed by music streaming company Spotify, is the poster child for this trend as an extensible open-source framework enabling teams to implement IDPs (or to buy a commercial distribution of Backstage, available from a variety of providers including Spotify, Broadcom, Red Hat and others). Gary advises that teams consider using Backstage to learn about and pilot IDPs but before implementing at production-level be prepared to fund a development team skilled with Node.js, PostgreSQL, React and TypeScript. This is in part because this level of expertise is needed to take advantage of and maintain Backstage's flexibility, and part because as the most-active CNCF OSS project, Backstage is evolving with architecturally-significant changes on a regular basis.?

How to Measure the Value of User Experience Design ?(August 26, Brent Stewart and Will Grant)

We know UX is important because every day we interact with software for which it was an afterthought. Brent gives us insight into why - leaders don't effectively measure and talk about it in ways that win focus and funding. He recommends that instead leaders routinely baseline, benchmark and improve UX through provable, quantifiable data. Finally, he details a variety of metrics and methods that leaders can choose from in building their UX measurement strategy.?

DevSecOps Maturity Model for Secure Software Development ?(August 29, Aaron Lord, Manju Bhat & Aaron Harrison)

For at least the last decade, we've told clients that to improve software quality, they need to "shift left". Testing was the first piece of that, and security is the current one, brought about by the ubiquity of not only software but also of vulnerabilities.? Aaron provides a five-part framework and maturity model that will enable SWE leaders to create a defense-in-depth approach to software engineering security. He also recommends establishing a community of practice to cooperatively drive the implementation of security between departments and creating an enabling team to make it easier for development teams to adopt security practices.

AI Will Not Replace Software Engineers ?(August 30, Phil Walsh, Gunjan Gupta, Helen Poitevin, Keith Mann, Dave Micko & Manju Bhat)

Phil and an all-star list of authors worked together to develop this thought-leading piece of work which posits that future software engineers will focus on AI engineering that changes the role of the developer from a person who writes code to one who breaks down complex problems into a series of smaller tasks. Subsequently the developer will steer AI agents through natural-language prompts and retrieval methods to complete them using AI developer platforms. Further, Phil says that overuse of AI in the near-term will stunt the growth of junior development staff while outsized productivity advantages go to senior developers better equipped to make best use of AI. A thought-provoking idea, this note suggests to me that we may be on the cusp of an AI winter, as the immediate "big wins" may not be as forthcoming as industry players would like SWE leaders to believe.

Peer Lessons & Insights From Using GenAI in Software Engineering ?(September 2, Luke Parker)

We know that clients want to understand what others in their position are doing; a survey last year showed that among SWE leaders, about 70% first "phone a friend" when encountering a problem or situation they haven't dealt with before. In this note, Luke harvests and condenses the conversations of real-world SWE leaders, detailing the major use cases they are acting on today and how they're going about them. These include code generation, code quality, documentation and enhancing codebase understanding. How SWE leaders measure business outcomes toward these use cases varies, but there are tangible findings here that analysts and clients alike can use to better understand what is happening with SWE and GenAI within the Gartner audience.

Magic Quadrant for DevOps Platforms ?(September 2, Keith Mann, Tom Murphy, Bill Holz & George Spafford) &?Critical Capabilities for DevOps Platforms (September 3, Tom, Bill, Keith & George)

The SDLC tooling market has long titrated between centralized and decentralized; with the launch of the DevOps MQ last year (hat tip to Manju), we took a position that the complexity of modern software development is moving back in the direction of suite-based approaches to DevOps that combine an assortment of capabilities ranging from CI/CD, to developer portals, to AI-augmentation, to observability and more.?

In this year's edition of the MQ/CC, Keith, Tom and their coauthors show how the distance is growing between the true "best of brand" suite offerings and the challengers who are more appropriate for "best of breed" strategies. To be fair, a piecemeal approach may be the best-fit for a particular client organization, and while the usual suspects Gitlab, Microsoft and Atlassian keep moving "up and to the right" in the MQ, it is in the details where we can find the best alignment to a SWE leader's strategic priorities. ?

How to Measure Developer Experience ?(September 2, Alec Pallin)

Our successful Developer Experience Assessment has already been used by many organizations and we have thousands of data points to show for it. This wealth of facts is of course available to anyone in the team who would like to reference it in research. In this short note, Alec details the framework that we have deployed within the DX Assessment, so that SWE Leaders and their teams can understand and strategize how to improve DX in their organizations. This will also be a handy reference for any analyst taking inquiries on or preparing talk tracks for presentations that reference DX.

Measure & Monitor Technical Debt With 5 Types of Tools ?(September 8, Tigran Egiazarov, Tom Murphy)

Tigran says that the main problem with technical debt is "not detecting it in time and failing to put a remediation plan into action." I've often said that if technical debt goes on long enough, it becomes unpayable and you have to declare bankruptcy, usually by replacing a software system entirely. However you choose to describe it, unpaid technical debt leads to big problems, and usually SWE leaders don't even know how bad the situation is until something breaks. Tigran's note gives a good overview of the tools SWE leaders and their teams can use to expose, manage and remediate technical debt. By the end of the note I found myself wanting the answer to the question - where should I start first? In other words, which kind of technical debt that the tools can be help me to discover is the worst kind that I need to fix right away? I look forward to more research on the topic, as I'm sure do our clients.

How to Build AI Skills in Software Engineering Teams ?(September 9, Alec Pallin and Miriam Colman)

In this Data View, which is a content type that we use to explore in-depth results from a quantitative data set - in this case the 2024 SWE survey - Alec examines the demand for AI skills and gives some basic advice on how SWE teams can acquire them. Unsurprisingly, AI/ML engineers are highly in-demand, but approaches to upskilling vary across the industry. Formal training lags behind informal, self-directed learning and on-the-job approaches, and SWE Leaders also seem to frequently turn to hiring AI talent off the street to build up in-house talent rather than contracting or outsourcing.?

Video: The Best Approach to Balancing Microservices & Monoliths ?(September 10, Tigran Egiazarov)

Tigran lays out the strengths and weaknesses for MSA and Monoliths, detailing at a high level the tradeoffs between agility and complexity that they bring. He also points out correctly that the choice of architecture should be driven by business needs rather than technical desires. Finally he points to several other content items that will help the viewer make all of these choices.

Video: Leadership Vision for Software Engineering - Generative AI, Developer Productivity & DevEx ?(September 10, Mark O'Neill)

In his final video for us as our SWE Chief of Research - he's just been promoted to a bigger role where he can make even more of a difference - Mark lays out this year's Leadership Vision. These LV decks/notes require the CoR to take a position on which of the many important facets of the role are most important to focus on in a given year. That GenAI is on the list will surprise no one, SWE or not. DevEx and Dev Productivity are like two sides of a coin, and Mark tells leaders how to improve both by focusing on customer-facing changes and on change failure rates.

Video: Use Existing Assets to Embed AI & ML into Applications ?(September 11, Jim Scheibmeier)

Jim's enthusiasm about SWE comes through in this video about embedding AI into software through the many SDKs and APIs available to us already. A quick plug for the CAIDS MQ lays out the many features that developers can already use to enhance software with AI.

?Video: How Software Engineering Leaders Can Harness Open Source's True Value ?(September 11, Anne Thomas)

Open Source is critically important to SWE, as we build so much OSS into our software through underlying frameworks and libraries, to say nothing of complete OSS products used in the development process. We use OSS because it makes us more productive. Unfortunately, when we do so we expose ourselves to the underlying risks that may be inherent in the software that we're depending on. Anne gives some pragmatic advice on how to improve the safety of OSS consumption, in particular building an OSPO.

Tool: Assess Your Software Engineering Product Management Competency ?(September 11, Deacon Wan & Oleks Matvitskyy)

This tool provides a basic entry point for SWE Leaders to assess their affinity for and degree of maturity in five pillars of product management competency. Deacon's five pillars include instinct, requirements, CX, market and strategy. The document itself is a guide to using the downloadable Excel spreadsheet which offers several steps to take to improve maturity across each pillar. I would consider this a pretty basic "getting started with understanding yourself" for a SWE Leader who also must lead product management in his team.

Product-Centric Delivery Organizational Structure Library ?(September 11, Neha Agarwal, Akriti Kapoor, Sarah Baumunk)

"How do we go from projects to products?" is a bread-and-butter question we've answered for clients on thousands of inquiries (maybe tens of thousands?) over the years. Neha's research enables the SWE Leader to draw a destination postcard for what his future product-centric organization might look like. Expanding and refreshing the prior set of 7, this note details the organizational structures of 10 large enterprises spanning industries as diverse as automotive (Ford), healthcare (Neami National), government (Registers of Scotland) and financial services (Isbank and Vanguard). The downloadable library will be an excellent source of comparison for clients and a strong reference for analysts.

Develop Software Engineering Skills Using Online Learning Platforms ?(September 13, Peter Hyde)

We know from "How to Build AI Skills…" that self-directed learning is the #1 way that SWEs learn new skills. Peter's refresh of the GTP note on Online Learning Platforms dovetails nicely with this idea, since by definition these platforms require "self-motivation, [and] proficient time management." Besides using the platforms themselves in a way that best suits the learner, Peter recommends that they combine their efforts with on-the-job-type activities like experiential learning, which will help to overcome the forgetting curve. His advice that each learner take an hour out of each workday to focus on self-improvement is something we should all strive for. So much of our content is applicable not only for our clients, but also to us personally in our jobs, and this document is a case in point.?

Market Guide for Cloud Development Environments ?(September 18, Manju Bhat)

According to my back-of-napkin calculations, the global market for CDEs may already be over $1B, and Manju's MG is our initial foray into covering this fast-moving segment. CDEs provide a virtualized, easily-provisioned, instant-on place for SWEs to build products without needing to worry about dependencies, security, source control, etc. Further abstracting away the plumbing associated with the developer experience should enable greater focus on real work, and it decreases the likelihood that an individual developer will make a critical mistake with a dependency or a software supply chain security requirement. When my son William, 11 years old at the time, told me that he'd rather use Replit than work with me to get the right version of Python development tools on the Linux-based PC we'd built for him together, it made me a little sad that he cared so little for the inner workings of the machine. But, I can't deny the productivity enhancement an additional layer of abstraction brings; meanwhile, I will continue changing my own engine oil.