Embracing AI Partnership in Software Engineering

“Over the course of the last ten years, fifteen years, almost anybody who sits on a stage like this would tell you that it is vital that your children learn computer science...?

And in fact, it's almost exactly the opposite. It is our job to create computing technology such that nobody has to program. And that the programming language is human…??

Everybody in the world is now a programmer. This is the miracle of artificial intelligence.”?

-Nvidia CEO Jensen Huang

The above quote from Nvidia’s CEO made waves in February, when on stage at the World Government Summit in Dubai, he recommended kids should study biology, education, manufacturing, or farming—instead of software engineering. Gaining expertise in another field, he asserted, would be more useful, since AI will be handling our programming.?

In March, newly launched startup Cognition Labs poured gas on this fire when they rolled out Devin, they described as “the world’s first fully autonomous AI software engineer” and a “a tireless, skilled teammate,” that can:

• Learn and use new technologies
• Entirely build and deploy applications itself
• Autonomously identify and resolve bugs in codebases
• Train and fine-tune AI models
• Complete engineering jobs on Upwork

Devin is departure from prior AI coding tools—built to solve entire solutions from start to finish, including searching the web for materials it needs. Its release is sending shockwaves through the industry as a symbol of the evolving role of AI in software development.

It also begs the question: are software developers developing themselves out of a career?

Today’s PTP Report takes a look at the impact of AI in software engineering: what current tools can do now, how they’re improving, and how developers can adjust to not only survive, but even benefit from AI-enhanced software development.

AI Coding: Strengths and Weaknesses

Prior to Devin’s release, most of the tools we’ve seen have been coding assistants, more like AI Agents built to help developers in their work. While these tools can write code, they rely on an engineer giving them sufficient inputs, reviewing their work, and making necessary corrections.

[For more on AI Agents and what they can become, check out the PTP Report What’s Next for AI: AI Agent as Platform.]

Note that OpenAI’s Codex was built off GPT 3 and its successor powers GitHub Copilot. Codex was initially closed beta that shut down, but after popular demand it remains open to researchers.

These are just examples, as there are more options seemingly every day, including: Replit GhostWriter, Amazon CodeWhisperer, Cody by Sourcegraph, Tabnine, MutableAI, AskCodi, Al2sql, Durable, Mintlify, CodeT5, Codiga, PolyCoder, Snyk Code, ClickUp, Safurai, Cursor, What the Diff, Code Snippets AI, COSINE, Kodezi, Blackbox AI, Bito, Debuild, Codium, and Cognition’s Devin.?

Current AI coding tools can:?

• Create code in a wide variety of languages
• Provide explanatory comments and assist with documentation
• Review existing code, identifying potential issues or bugs
• Write unit tests (successfully, too!)
• Refactor existing code
• Do some translation between languages (although this is an area that’s still developing)

They can also be leveraged as instructive tools for new coders, assist with code reviews, and digest and search through enormous codebases.?

To date, they’ve excelled most in environments where tasks are well-defined, repetitive, and may benefit from the analysis of patterns or solutions among large datasets.?

Still, significant issues remain. Researchers at Texas Tech put AI coding to a more systematic test, using GPT-4 and GPT-3.5 to solve well-known coding problems from LeetCode (a resource containing the kind of problems learned in school and used in job interviews, for example).

The results were evaluated on selecting data structures, using efficient algorithms, and manipulating strings:

• AI excelled at problems featured in a training dataset, outperforming humans
• But given complete test questions with examples not seen in the training, humans succeeded around 45% of the time, vs AI’s 26% for GPT-4 (and under 13% GPT-3).
• With incomplete information, GPT-4’s performance dropped to under 19% (GPT-3 below 13%)?
• Though GPT-4 results were notably superior, GPT-3 oddly outperformed GPT-4 on some problems?

They also found that the AI consistently failed at some basic tasks, such as defining variables.

AI solutions also can still struggle with writing secure code, and making judgments, and pale in comparison to human coders at understanding context and applying ethical considerations—in short, when dealing with complex, multifaceted problems common in the real world.

The AI solutions also can struggle with scale, and consistently perform better with older problems, perhaps owing to the wider availability of documentation and examples in their training data.?

This latter issue, at least, is already being attacked head-on.

AI Advancements in Coding

Many human developers start thinking through solutions in one of two places: Stack Overflow and GitHub.?

These resources are a core resource for developers, in many cases replacing textbooks as reference guides over the past decade plus.

Now both are also knowledge sources for AI training.?

Late last year, Google formed a partnership with Stack OverFlow, allowing its LLM (or LMM or MM-LLM—see Seeing Is Believing: The Multimodal AI Evolution for this distinction) Gemini to summarize the site’s questions and answers within Google Cloud. And Google can use this data—arguably the biggest coding knowledge base on the planet—without restriction, such as for training their models.

Rival Microsoft has a head start—owning GitHub since 2018.? It is already integrated GitHub with its own CoPilot AI.?

These platforms, rich in coding examples, solutions, and discussions, can significantly enhance AI's coding capabilities, both via real-time assistance to developers, and by increasing the own coding efficiency and quality of AI-generated outputs.

Evolve to Survive, and Ascend

Despite everything software development as a field is projected by the US Bureau of Labor Statistics to grow 25% by 2032, which is much faster than most occupations.?

But the AI partnership in software development is here, so how do developers stay ahead, and improve by using AI instead of dreading being replaced by it?

As discussed in a new report by McKinsey titled A generative AI reset: Rewiring to turn potential into value in 2024, developers need more than just coding skills to be successful in this new world. While coders need months of experience working with AI tools in building, testing, and verifying software, they also benefit from the following skills:

1. Design: To see where to focus solutions
2. Contextual Understanding: For the most relevant/highest quality answers
3. Collaboration: Working well with knowledge experts (such as for validation)
4. Forensics: Figuring out breakdowns (data, user intent, meta data, other)
5. Anticipation: Handling outcomes and coding in the right track

On the subject of education, the University of Pennsylvania recently became the first Ivy League school to offer an engineering degree in AI (starting this fall), and Carnegie Mellon created an undergraduate AI program all the way back in 2018.?

Associate Dean of Undergraduate Education at Penn Engineering Robert Ghrist explains that they are “training students for jobs that don’t yet exist, in fields that may be completely new or revolutionized by the time they graduate.”

But in contrast to Nvidia’s CEO, Ghrist still advises professionals to learn computer science, and math, because “without them, the rest is a black box of action without understanding.”

For the developer with these skills in an AI and software engineering collaboration, the fruits can be impressive, as the results from this GitHub-based study show:

But the rapid advancement of AI coding tools also shows just how much software engineering is about more than coding.

Full developers—as engineers—remain needed for: problem-solving, creativity, empathy, and ethical judgment, all skills that remain (for now) strongly human.?

[For assistance in finding developers, check out what PTP has to offer]?

For aspiring entrants to the field, a solid foundation in computer science principles such as data structures, algorithmic efficiency, and math are still key, coupled with proficiency in AI and machine learning.?

And as always, strong soft skills differentiate human engineers from AI ones.

Conclusion

As AI tools become more integrated into our development processes, the AI impact on programmers could be extreme.?

[For an example, check out our PTP Report to see how AI can be a game-changer for DevOps]?

Coders, or pure hackers who eschew broader development and engineering considerations, may, indeed, face the biggest threat from AI in the near term.

But by leveraging AI as a complement to human ingenuity, true software engineers can instead excel, offloading repetitive and tedious aspects of their work to focus more wholly on design, improvement, ensuring quality, and security.?

By adapting to these changes, embracing AI, and keying on areas of human strength, software engineers should flourish.

The future of software development lies not in choosing between AI and human expertise, but in combing them, to ensure relevance and value for years to come.

- Doug McCord

(Staff Writer)

Check out other articles from PTP on AI, career advice, and professional development.??

Get the latest updates on recruiting trends, job market, and IT, and expert advice on hiring and job seeking at the PTP Report?