Software 2.0 takes shape.

Automating Software Development with AI

Introduction

In our previous post we described the components, tools, and companies for building AI applications. In this post, we closely examine companies who are building tools for (partially) automating software development. 

Why focus on software development? Demand for software developers is at an all-time high at a moment when software continues to grow in importance. According to a 2019 Stack Overflow survey, more than a third of developers identified a lack of resources (“not enough people for the workload”) as one of the main challenges to productivity. 

With demand for developers far exceeding supply, companies are looking for tools that can boost the productivity of their developers. Recent developments in AI and machine learning are beginning to make an impact in different areas of software development. A new generation of automation and machine learning tools are augmenting software developers across the software development lifecycle. About two years ago, we started tracking applications of machine learning to software development.  At the time there were already a few tools for automating various aspects of the software development lifecycle.  

In this post, we take a much more systematic approach to understanding how machine learning is impacting software development.  We will look at how AI and machine learning are being used to help developers through every stage of the software development lifecycle. We will also highlight several tools that software development teams can use right away, and point to promising research projects that can lead to tools in the near future. Finally, we will show that while AI is beginning to be used across all aspects of software development, some tasks are proving to be harder to automate than others.  

Machine learning requires large amounts of data, usually in the form of labeled examples. Is software development really a prime area for automation? We believe that it is for several reasons. First, data and metrics are already available and collected during the different stages of software development. Second, while there is already more than enough scale to justify the use of automation, software development continues to grow in importance and complexity.  Finally, most automation tools are actually human-machine hybrid solutions. We believe software developers will be receptive to technologies that automate routine tasks. Machine learning and AI are part of the broader software industry and thus developers are less resistant to AI and ML solutions. To a large extent, many developers already automate aspects of their work (developers are known to write scripts to automate mundane, repeatable tasks).  So it’s no surprise that software development is an area where automation and AI tools are beginning to appear.  

Figure 1:  AI disruption across the stages of software development

New solutions that rely on AI need to demonstrate sufficient business value to justify placing them alongside existing manual and rule-based systems. And since companies will need to purchase and implement new systems and workflows to accommodate any new solution, AI systems need to add value that exceeds any new costs and requirements (such as the need for labeled data, hiring new staff, etc.). AI solutions for software development will allow novice engineers to complete their projects more quickly while producing high-quality code.

With this in mind, Intel Capital has identified application monitoring and software testing as prime investment areas.  Companies already use a wide range of tools for rule-based application monitoring, thus we believe that AI solutions have the potential to disrupt this existing market. In the software testing and code analysis & compliance stages, existing tools mainly focus on process automation and lack of test generation. We are seeing promising AI-based products that automate a lot of manual work done by developers and QA engineers. 

Given that it has the highest business value, we’ve also peered into the future and examined AI in code development. We concluded that while current research and breakthroughs show a lot of promise, we aren’t quite ready to make major investments in AI tools for code development. With that said, we believe that over the next few years we will begin to see code development solutions that are target narrow problems (e.g., data processing, machine learning modeling, etc.).

Software development today

Automation in software is best viewed through the lens of continuous integration (CI) and continuous deployment (CD) - two widely used software engineering development practices. Large software projects involve multiple engineers working on copies of a shared codebase. CI is a practice that requires team members to integrate their work more frequently so errors can be detected and located more quickly. CD is a software engineering approach wherein code changes that pass automated tests are automatically deployed to production. Software development presents several opportunities for automation:

Code development and programming are simpler today compared to the languages and tools developers used 30 years ago. But it is still a manual process. We are beginning to see solutions that use AI to automate some aspects of code development, including solutions that automate the creation of code by program synthesis, smart code completion and static code analysis.  

Software testing is an area where AI and automation tools have made inroads. Testing is broad and includes unit/functional/integration testing, as well as regression and stress testing. The majority of AI-enabled testing startups are in functional, integration, and regression testing. 

Code Analysis & Compliance is another area where we see the rise of AI. AI can be used to ensure that code adheres to a company’s software development principles. AI is also being used for static code analysis and to produce alerts about critical vulnerabilities. 

Metrics and Observability tools are used to monitor the health of software. We are beginning to see AI systems that monitor and optimize applications in production.

Figure 2: Early Adoption of AI in SW Development Stack.

Automation in software development and applications

Which specific areas in software are prime candidates for automation and AI technologies? We believe that there are two key ingredients that are important in finding candidates for automation: (1) focus on heavily used systems or frequently occurring tasks, and (2) systems that have to be frequently adjusted or tuned.  Some examples include

●      Data management systems: There are early descriptions and examples of self-driving data management systems, as well as simpler systems that demonstrate the use of machine learning to learn joins and indexes, or to manage multiple storage engines.

●      Dashboards: Rather than go through the manual and tedious process of having to create them from data warehouses and reporting tools, we’ve long wanted dashboards that are automatically created! UX and presentation also matter: users are drowning in charts and alerts. Ideally, machine learning and AI can surface the right charts, trends, and correlations. There are some early, partial examples from startups like Anodot and Sisu, but we expect lots of progress in the years ahead.

●      Websites and mobile apps: We expect AI to lead to highly personalized sites and apps. A handful of technology companies are using real-time data for online learning and reinforcement learning to deliver highly personalized user interfaces. We expect these tools and technologies to become more accessible in the near future.

●      UX and Design tools: Simple automation and scripting are starting to appear in tools used for creating and editing digital media. Machine learning has the potential to take automation to another level. There are early examples of the use of deep learning and reinforcement learning (RL) to help game designers. Design is an expensive and important part of video games. Game designers can be augmented with new tools from computer vision (e.g., GANs). An even more complex example is the use of reinforcement learning to train non-player characters in games.  

Figure 3: Representative examples of tools, services, and companies in our SW Development stack.

We now describe the use of automation in each of the building blocks in the software development stack. We also look ahead and project future automation solutions.

Code development is the least automated process in the software lifecycle, developers today are more efficient thanks to tools created in the last half-century. Programming languages today are high-level compared to the ones used 30 years ago. While these languages have made developers more efficient, better tools are needed as demand for developers still exceeds supply. For example, Codata & Kite are “intelligent code completion” products that use AI to enhance code quality and enforce compliance. 

There will be a wave of tools designed to further augment and supercharge developers. A recent example from OpenAI shows a language model able to generate Python code, after being trained on GitHub code repositories using a Microsoft supercomputer. An earlier example (Autopandas) involving neural-backed generators, hints at tools that enable developers to become productive in libraries and frameworks without having to master intricate APIs. It involves a user specifying desired inputs and outputs and Autopandas generates an optimal program for generating the output. Similar tools can be built for data-intensive processes including ETL, data modeling, dashboards, and more. Another recent example from Facebook AI potentially paves the way for tools for migrating code to modern or more efficient programming languages. It demonstrates how unsupervised machine translation can be applied to source code to create a transcompiler.

We believe the move to microservices opens up possibilities for automation and will lead to more opportunities for AI-enabled software development tools. Breaking software into smaller pieces paves the way for highly targeted automation solutions. 

Software testing is beginning to leverage progress in deep learning in computer vision (CV) and natural language processing (NLP). Imagine a software application with a GUI, and a testing robot able to recognize and understand text boxes and other interface elements. This same robot can be enabled to automatically create test suites. This describes the situation today. One example is Mesmer, a robotics process automation company focused on automating customer experience testing. 

Looking ahead, we believe the next generation of automation tools will take these capabilities to another level. Imagine a set of AI agents that can download and test software from an app store and in the process learn better ways for designing and implementing software. These AI agents can offer tailored made application development services similar to what Wix offers for website development.

Code Analysis & Compliance is another area where we will see the rise of AI. Machine learning and AI will help organizations release better code that adheres to recommended development principles, provide alerts about critical vulnerabilities, and conduct comprehensive static code analysis. We are beginning to see a few companies with AI-enabled solutions in these areas (see Figure 2) and we expect more to follow in the future. 

Application Monitoring or AIOps addresses the need for operations support. AIOps combines data analytics and machine learning functionality to analyze data generated by IT systems. AIOps platforms enhance a broad range of operations including, anomaly detection, event correlation, prediction, and root cause analysis to improve monitoring and observability, and service management. This is an active area: existing “rule-based” application monitoring solutions are beginning to add machine learning, while new companies like Anodot offer anomaly detection and forecasting for application monitoring. 

Conclusion

The IT industry in general and software development in particular, have been early adopters of AI. In this post, we described some early applications of machine learning and AI to different aspects of software development. We also described areas where the use of AI for software development will accelerate in the years ahead.

Will automation mean that we will need fewer developers in the longer term? We think that the demand for software developers will remain steady. However, we believe that the software industry will need developers who possess different skill sets. In the future, developers will need to focus more on integrating building blocks and capabilities, and less on writing code from scratch. 

 ---------------------------------

Disclosure: Intel Capital is an investor in Anodot & Mesmer. Ben Lorica is an advisor to Anodot.

Ben Lorica is chair of the Spark+AI Summit, co-chair of the Ray Summit, and principal at Gradient Flow.

Assaf Araki is an investment manager at Intel Capital focused on AI and Data Analytics platforms and products.