AI-Powered Product Development (without software engineers)

The AI revolution is reshaping the tech industry, with Large Language Models (LLMs) at the forefront of this transformation. As these models continue to rapidly evolve, they're challenging long-held assumptions about the future of software engineering. Tech leaders are bullish on LLMs potential to revolutionize coding.?

• Jensen Huang, NVIDIA CEO, advises that people should not learn how to code anymore, as with the rise of LLMs, programming is no longer a critical skill.
• Matt Garman, AWS CEO, predicts that most developers will not be coding in 24 months

OurWorldinData.org aggregated results from recent AI benchmarks, to compare the progress of LLM performance against “human capabilities”. Their data shows that LLMs can outperform humans in complex tasks such as reading comprehension, image recognition, and nuanced language interpretation.LLMs currently fall short of human performance in tasks such as code generation, complex reasoning, and math problem-solving – key skills for software engineers. If their rate of improvement continues…. It will not be that long until they exceed human performance.

Since perfection is the enemy of progress, I think LLM’s will become “good enough” much sooner, and let's face it, with economic factors constantly driving businesses to reduce cost and increase productivity… an agentic AI engineer that is 85% as good as an average human software engineer, but can work 24/7 without salary and benefits…. Will be too tempting for many corporate balance sheets to resist.? As such, there will be some companies who opt to reduce their headcount in favor of using agentic AI to do the same amount of work with lower costs. However, the smart companies will opt to use agentic AI as a booster to their current productivity, and do significantly more work with only a moderate rise in costs (which will likely include headcount reduction to a lesser degree, such as not backfilling natural attrition).

However, the idea of agentic AI completely replacing software engineers faces several challenges. As Garman notes, software engineers will need to develop higher-level skills for problem solving, product innovation, and understanding user needs and motivations -? "How do I go build something that's interesting for my end users to use?" - so that they can get the best out of their agentic AI coworkers. Which if you ask me, sounds remarkably like some of the core skills of Product Management…

Hmm. So that poses an interesting thought experiment: What might it look like for a product manager to leverage AI to build a new marketplace platform, without relying on software engineers???

Take the following section with a pinch of salt, I am not a software engineer, I have not written code that went into production since my days at Netflix (2008-2013). The following is just a high level overview to get YOU thinking. There are plenty of holes to be poked, assumptions made, and nuanced details just glossed over. I will list out a few tools along the way, NONE of which I have actually used, just to highlight that the journey has already begun - Forewarned, is forearmed so to say

1. System Design and Architecture: AI as Your Virtual Architect

The journey would begin with system design and architecture, traditionally the domain of seasoned software architects. Product managers would now leverage AI to generate detailed architectures including microservices, API designs, cloud infrastructure, database schemas, etc DZone has a good overview in AI-Driven API and Microservice Architecture Design for Cloud, and Eraser.io already leverages an LLM to generate cloud architectures and other diagrams, based on prompting.

The product manager would create high-level requirements and use cases, and through iterative prompting, the LLM would generate a comprehensive design. This approach would speed up the initial design phase and allow for rapid exploration of different architectural approaches. Potentially uncovering an innovative solution based on the breadth of knowledge of the LLM, that a human architect might not have encountered yet in their career.

2. Code Generation Pipeline: From Concept to Codebase

With the architecture in place, the next challenge is translating a business systems design into actual code. This is where LLMs could really shine, potentially eliminating the need for a software engineering team.

• Brainboard.co can take infrastructure as diagram (IaD) and auto-generate infrastructure as code (IaC). Theoretically, if we can pipe an AI generated architecture diagram from Easer.io into brainboard via their API’s…
• Uizard.io and Framer.ai are 2 commonly mentioned with respect to AI automating aspects of the UX design process, allowing for faster generation of ideas and iterations of design.?
• GitHub Copilot, ClaudeDEV, AIDER are just 3 examples of how LLMs are pushing the boundaries of what's possible in AI-assisted coding, but they are not yet capable of automatically building a full product.?

The product manager's role here is to provide clear requirements, and work with the AI to iteratively refine the generated application. While not writing code directly, the product manager becomes the crucial link between business needs and implementation reality. However, “clear requirements” is a relative term, so we as product managers might need to look for some form of standardized notations such as Business Process Execution Language (BPEL) or Business Process Model and Notation (BPMN). So that we can provide enough detail and context to an LLM for the full scope of the product.??

3. Testing and Quality Assurance: AI as Your QA Team

Quality assurance, often a bottleneck in traditional development, can be significantly accelerated with AI through automatic generation of unit tests for all components, creation of integration and end-to-end test scripts, regulatory and controls testing, and performance testing scenarios

Tools like Functionize and Testim, use AI to create, execute, and maintain test suites. These tools can even adapt tests as the application evolves, reducing the maintenance burden. As part of the earlier business requirements process, critical test cases and acceptance criteria would need to be included ahead of the code generation. AI can then expand on them, and potentially uncover edge cases that humans might miss - How many bugs have been reported in YOUR products, despite having a “comprehensive testing and QA process”?

4. Documentation: AI as Your Technical Writer

In the AI-driven development process, comprehensive documentation is generated alongside the code:

• Automatic creation of API documentation
• Generation of user manuals and developer guides
• Real-time updates to documentation as the codebase evolves

AI tools can analyze the codebase and system architecture to produce detailed documentation. This ensures that documentation remains current, a challenge often faced in traditional development environments. However, in this hypothetical scenario of a product manager leveraging AI to build a new product, without software engineers, how much documentation and what level of documentation would actually be needed? There is an argument to be made, that detailed documentation would not be required as there is no expectation of any human software engineer involvement…

5. Security Implementation: AI as Your Security Expert

Security, a critical concern in any application, and subject to multiple different legal requirements across the globe (with serious financial repercussions if not complied with), can be addressed with AI-driven tools.

AI-powered security tools like Snyk can be integrated into the development process, providing real-time security insights and recommendations. The product manager's role here would be to define the security requirements and identify the relevant regulatory compliance needs, the AI tools would then ensure the appropriate permissions and controls are implemented/enforced consistently across the application.?

6. Deployment and CI/CD: AI as Your DevOps Engineer

A critical piece of this puzzle would be modern practices for continuous integration/continuous deployment (CI/CD). Using AI to build the CI/CD pipeline of the future could manage the entire deployment process, from code commits, automated code review, testing and building, through to deployment (and rollback if needed), with minimal human intervention.

Challenges and Considerations

Whilst an AI-driven development framework presents exciting possibilities, it is crucial to understand the current limitations and the challenges it poses. Some of which were touched on above:

1. Technological Readiness: Current AI tools, while powerful, are not yet capable of seamlessly handling all aspects of complex product development without significant human oversight. Personally, I think we are 5-10 years away from a truly autonomous AI development pipeline. But much closer than that to enhancing our software engineering teams with agentic AI coworkers (and product teams with agentic AI product owners).
2. Code Quality and Consistency: AI-generated code often requires substantial human review and refinement. Ensuring consistent coding standards and architectural coherence across an entire project remains a significant challenge. I also expect that large scale code generation by AI, risks similar bugs being introduced across that code base at scale, thanks to current LLMs tendency to hallucinate near the edges of its knowledge.
3. Complex Business Logic: AI struggles with nuanced, domain-specific business rules. Implementing intricate workflows or industry-specific regulations often requires human expertise to translate these complexities into accurate code. One approach to help address thai, would be to use a collection of AI agents tuned for specific roles, and have them operate as a team - ironically, mimicking exactly what we do as humans.
4. Security and Compliance: While AI can implement basic security practices, ensuring robust protection against sophisticated threats and compliance with evolving regulations (like GDPR or CCPA) still necessitates human expertise.
5. Integration Complexity: Seamlessly integrating various AI-generated components into a cohesive system is non-trivial. Product managers must be prepared for significant integration challenges, and a revolutionary change in their role
6. Ethical and Legal Considerations: As AI takes on a larger role in development, questions of accountability, intellectual property, and ethical use of AI-generated code become more pressing.

Product Management would need to rapidly Evolve

The role of product managers would undergo a significant transformation. With several new skills being added to an already multifaceted role:?

1. Business Systems Architect: We will need to develop a deep understanding of system architecture from a business perspective. This involves translating business requirements into system designs that AI can interpret and implement.
2. Regulatory Framework Architect: We must become adept at defining the regulatory frameworks in order to comply with various laws and standards. This requires a blend of legal knowledge and system design skills.
3. AI Orchestrator: We will need to become skilled at orchestrating multiple AI tools, understanding their interactions, and optimizing their collective output.
4. Ethical AI Steward: We must ensure ethical considerations are built into the product from the ground up. This includes issues of bias, fairness, and societal impact.
5. Cross-Functional Synthesis: The ability to synthesize insights from various domains – UX, data science, business strategy, and technology – becomes even more crucial in an AI-driven environment.
6. Prompt Engineering Master: Advanced prompt engineering, or meta-prompting skills will be essential, as the quality of AI output is highly dependent on the quality of input prompts.

However, just as this article poses the question of a product manager developing a new product without human software engineers… There is a follow-on question of how a business person could use AI tools to do the same but without the involvement of a human product manager… Could we get to a point where the capstone project of a Bachelors of Business Administration degree, is to launch your own cloud based product? ? (I dread to imagine the sheer volume of new business ads on social media IF this happens)

Bridging the Gap: Current Best Practices

In the meantime though, product managers can start preparing by:

1. Incorporating AI tools into their current workflows to augment, rather than replace, traditional development processes.
2. Collaborating closely with engineering teams to understand where AI can be most effectively applied in the development lifecycle.
3. Investing in AI literacy and staying updated on the latest advancements in AI-assisted development tools.
4. Experimenting with small, low-risk projects to gain hands-on experience with AI-driven development approaches.

The Future of Product Development

While this future is exciting, it's important to remember that we are not there yet. The most successful product managers (software engineers, UX designers, cloud architects etc.. fill in the blank with any knowledge worker role) will be those who can effectively collaborate with AI, leveraging its strengths while mitigating its limitations to create innovative, robust, and user-centric products. The future lies not in AI replacing human roles, but in the synergy between human creativity, human intuition, human empathy, and AI capabilities.