AI Vibe Coding - The Good, The Bad, The Tragically Ugly

There's good reasons why vibe coding is on a tear right now. In some ways, there's potentially legitimate value that will come out of this. At the same time, I think it's likely we'll see some seriously tragic outcomes. Just in case you're not up on the meme, vibe coding is simply a term for using AI to write computer code. (See the link for the origin of the meme.) Maybe you could argue that if we're going to label things, you could call it vibe composing or vibe whatever-ing, given that there's generative tools for a variety of object types from art to music and so on. But let's focus on computer code for now.

So. Besides just having programmers possibly be more efficient, what are the benefits? Perhaps one of the loudest benefits right now is being shouted by a cohort of entrepreneurs and product managers. They're thrilled! "I can just get software now!"

"I am free!"

What they mean is free from the stranglehold technology costs have had on the ability to get digital product done. The problem with the Temple of Tech has always been time, money, and perhaps worst, resource contention. And those pesky coders had something else the Kingpins of history have always had, "They who control the means of production control the world." But now? Now it's democratized! No more desperate postings by non-technical co-founders seeking technical help for zero dollars, but a little bit of stock. There's perhaps some irony here if you look at historical Bourgeoisie Control the Means of Production. But we can skip the philosophy for now. If you're old enough, you can remember that if you had a development request, you'd maybe literally head to some special room behind secure doors in a basement where florescent lights battled with the glow of the green or orange text emanating from terminals. There'd be a faint smell of ozone as even the loud ventilation system couldn't quite keep up with all the buzzing electrons. Here, you would make your tech request. Ironically, it's not that different now in many places. At least, from a relationship perspective, even if the scenery has changed.

So can entrepreneurs and product managers just ditch their tech colleagues now? Are the headlines true? Coding skills are no longer a hot ticket? Let's take yet another deep breath. Come on with me now... breeethe... in through the nose, out through the mouth... one... two... three... Good. Yes. Things are changing again. And fast. But maybe don't just lay off everyone just yet.

Business & Technology

Truly enlightened product companies have product teams where everyone, (including tech leads), are tightly coupled and making things happen together. (C'mon now... who hasn't read Cagan's Transformed: Moving to the Product Operating Model yet?) But in a lot of places, there's still a gating process for technical resources where the business leads aren't really fully in control due to tech resource constraints and ownership over the Keys to the Code.

Everyone already knows at this point that AIs can help with code to varying degrees. Among the many tests we use in the AI Battle Bots Marketing Battle of the Checkboxes are those that test for code quality. Similarly, we have tests for how LLMs do on science, (GPQA), math (AIME), and a plethora of others.

How might this play out? Let's look at some scenarios.

Before getting into the weeds, let's explore another irony. There's been a trend recently to perhaps dump some digital product managers from some more engineering-focused organizations. The thinking seems to be along the lines that tech can handle product issues with AI, we need to be engineering-led, we can save a lot of money as good Product folks are expensive, and so on. The only problem is such companies can quickly find things fall apart due to lack of strategic prioritization, poor decision-making and lack of customer focus. This isn't well-studied as yet, but you can sometimes see companies hiring back for the role when they realize things are falling to crap. They might be thinking "but wait, we have project managers, wasn't that the same thing?" Turns out, no. No it's not. (By the way, see what Dan Olsen has to say about this in Product Management is NOT Dead. And also, Gabriel Steinhardt in The Demise Of Product Management Is Fake News.)

Now I promised you some irony. Here it is: There's now some product managers and entrepreneurs who think, "Hah! The foot is on the other shoe now! I can dump tech staff and still produce product! And I don't need design either!"

Let me try to sum up the collective foolishness.

It's often easy to look at someone else's job and think, "that doesn't look too hard."

Mostly, this is wrong. It's maybe an especially wrong perception when you see truly skilled practitioners who are so good they make things look easy. And even where it might be slightly correct for those who have solid cross-over skillsets, it's rare that anyone can handle all of the roles alone. At least, not very well or to do anything very complicated.

OK. Let's get back to actual AI generated code issues and ask whether products and projects using it can be wildly successful.

VC Math Applies

Venture Capital (VCs) Math applies here. It's long postulated that VCs both suffer from and benefit by a win / loss ratio of something like 8:2 following the usual Pareto 80/20 rule. (And actually, it's probably more like 1:10 or much worse; depending on who's numbers you read.) Winning often means getting up to bat more than being able to consistently hit Grand Slams. (Because no one can really do that consistently.) The same has always been true of startups. Or for that matter, not just startups, but new product initiatives regardless of whether they're at a startup company or part of a larger firm.

Here's the problem. Or at least one problem anyway. We know about many of the success stories. Most of them actually. Because they're held up as case studies, featured in the media, and used as examples of why AI-driven development is the future. We really don't know failure rates though. Not really. There are some very public failures and some deadpool sites. Maybe some studies. But overall, the true scope of failures is opaque. In terms of startups, CB Insights puts failure rates at 90%, Forbes sees this as also about 90% for IT in particular. That's startups. For corporate, it may be hard to say the product failures, but on a project level, TeamStage says 70%+. Forbes also wonders if GPTs can help, as they cite general transformation failures.

Whether it's products, projects or otherwise, I think it's fair to assume that the more at-bats people take, the more hits we'll see. And these tools can probably get us collectively up to bat more. This is arguably a good thing. At the same time, there could be a crowding-out effect in some cases, where the flood of AI-generated code saturates markets, making it harder for high-quality projects to stand out. As we move forward and get more failures, though, perhaps a flight back to quality will emerge, where seasoned developers and well-structured teams regain their prominence over purely AI-generated solutions.

The Good

Basic Coding Assistance

AI-powered tools like GitHub Copilot, Tabnine, and ChatGPT and more can assist developers by fully generating or auto-completing code, suggesting optimizations, and reducing repetitive tasks. This improves efficiency and allows programmers to focus on higher-level problem-solving.

In cases where professionals are using these tools to increase their speed and overall deployment cadence, these tools might be efficiency accelerators like we've never seen before. For these situations though, coders can quickly review the output themselves. They'll have the skill to double-check to make sure things are happening as they see fit. If something is off somehow, they'll also have the skill to manually fix it.

As well, these tools can look at error messages and suggest improvements to code or configurations.

Side note: While early in my career I worked on some production code, I'm a product person, not a coder. Anyone sensible would not want me committing any code to production. And yet, I effectively use ChatGPT to create python code allowing me to build and debug my own small LLMs based on custom datasets. This has been primarily for personal use though. Also, I've used tools like n8n.io and others to build AI Agents and workflows that go across multiple AI models and other tools. In the space of just a few hours, I've been able to tie together multiple data sources, convert some of them to vector databases, and use them as Retrieval Augmented Generation (RAG) input to LLMs for doing my own prompts or transforming data. All this with no especially great skills in coding. None of this means I should be building production level software, but it does help me work better with my developer colleagues. By the way, I highly recommend this course: Learn to Build AI Agents & Multi-Agents. It's a great 'hands on keyboard' experience in building AI agents. I think it's useful for both individual contributors and senior product leaders to get a good sense of how these technologies are developing.

Prototypes by Entrepreneurs

Another benefit of these more capable tools is non-technical founders can quickly create prototypes to validate business ideas. (Maybe. There's a lot to do as a founder and even better tools might not be enough if they still take time from other strategic tasks.) Still, they can iterate rapidly, reducing initial development costs and making early-stage funding and testing more accessible. It used to be you'd come up with some requirements, find a technical co-founder or spend money with a software development firm. Now, you can try Lovable, Bolt, and others to spin up something fast. Or if you know a little Figma, have those designs go beyond prototype to actual application. This is possibly just fine and great to kick something out for a super high quality and high resolution truly working Minimum Viable Product (MVP). And in cases like Lovable, if you're smart you'll make sure you sync your codebase out to GitHub, and in all cases figure out how you can update things. You should probably also ask "how do you host?" "One click deployment" is great I suppose. But just where are you actually hosting? Did you just get trapped into a vendor's vertical solution that won't really scale? Uh oh. And is this anything you should really be selling? At all?

Some of my skepticism and concerns notwithstanding, these tools are - my opinion - MAGICAL! And yet, here's the kinds of risks we'll face. This example below is not a big deal. But consider a neophyte entrepreneur getting a little further. Maybe hooking up some billing. Perhaps reaching just a few hundred customers and then having a catastrophic failure. We'll have to wait for the lawsuits. Check out this message board posting below though. Look at the simple poor grammar by the poster, as well as the tone. There's a thrill to be free of the "professionals" and yet, you have to wonder about the quality here. It is possible this is just an english as a second language issue, but if not... even if it's just excited typing on a forum, is this the kind of tech lead who's product you'd want to depend on?

The answer is... maybe, but probably not. At least, not for anything serious. They might not have a clue even as to how to fix the problem expressed. They're wholly dependent on the tool to know how to help them fix it. Maybe that will work this time. Next time?

So for prototyping, great! For deploying reliable production ready product? Maybe. Depends on the risk level. If you've got some ad-supported content site, maybe that's ok. If you're taking money? Or doing something involving safety or regulatory issues? Again... we'll have to wait to see what failures and lawsuits we get that end up based on these types of deployments. I wonder if the lawyers will sue the AI code generators? (Who am I kidding. Of course they will!)

Prototypes by Product Managers and Teams

Product managers can experiment with functional proof-of-concepts without relying entirely on engineering teams. However, it's probably still best to have a development tech lead in on these efforts. You can still go faster than ever, but maybe avoid just going faster right into a brick wall.

Using these tools can accelerate innovation cycles and help teams focus on user feedback and market fit. Even if no technical team member can be assigned, these tools can allow non-technical team members to quickly iterate on prototype and ideas. If a product team is still working in a siloed corporate environment, this may be the fastest way to get some working ideas in front of customers for validation. However, not having technical team members and design involved early remains an unfortunate pattern. If at all possible, a tech lead will ideally be on such a team as this person can help guide solutions towards the more viable and feasible.

The Bad

Junior Coder Educational Atrophy

Even with great AI coding tools, we'll still likely need senior level coding skills and system architects. It might be harder to build these skills now.

Over-reliance on AI coding tools by individuals, organizations and society as a whole could lead to skill degradation among junior developers. If entry-level engineers depend too much on AI, (or are simply needed less), they may miss foundational coding principles, leading to long-term skill gaps. There may already be a challenge in tech with mentorship and spooling up junior personnel due to remote working. As someone who has generally worked remotely since long before pandemic times, I believe the benefits of remote - for many industries - far outweigh being in an office. But not always. There are times being together makes sense. And even when generally remote, there are at least two areas that are at risk. First is cultural assimilation for any new hire. But also mentorship and growth opportunities for those more junior in their careers. Now with these code tools for those in development roles, often the more basic tasks can be accomplished with minimal personnel effort or need for interaction with other team members. Unfortunately though, one of the ways coding proficiency and expertise gets built is by powering through basic challenges and spooling up to more complex tasks; often with the help of peers.

Will skill growth suffer because the hard knocks of getting through the basics no longer apply? Will we be able to get senior level architects from a cohort of developers that might lack some foundational skills?

Minor Risks

AI-generated code often lacks context-awareness. It might introduce inefficiencies, suboptimal logic, or unnecessary complexity that human developers have to clean up later, adding unexpected technical debt.

These issues are not necessarily tragic. And yet, sloppiness can lead to additional risks and performance issues. Essentially, this creates technical debt that may be wholly unknown and effectively undetectable by low-skill operators. Of course, a series of minor risks can also compound over time.

The Tragically Ugly

Safety and Mission Critical Failure Risk

AI-generated code may not meet rigorous safety and reliability standards required for applications in healthcare, aerospace, and critical infrastructure. A single unnoticed AI-generated error could have catastrophic consequences. Will individuals or teams rushing to market have the background and sense to do all the proper regression testing needed?

Perhaps customers of such services will have enough due diligence in place to mitigate gross failures, thus pushing vendors to take great care in their use of such tools. But in the pressure-filled race to get to market, is there enough incentive towards safety? Governance, Risk and Compliance (GRC) teams may struggle to keep up here. AI is certainly being used and planned for in safety and mission critical applications. From wildfire detection to Homeland Security, and so on. In these cases though, the tech is being used as a tool,

Financial Risk

Businesses relying too heavily on AI-generated code without oversight may find themselves exposed to major security vulnerabilities, compliance failures, or lawsuits, ultimately damaging their bottom line.

There's multiple failure modes here that have - what I believe - are clear and obvious liabilities. For one thing, whatever failure mode occurs might incur liability. And compounding that might be a very legitimate claim about gross negligence. It's one thing to put out a product that might experience a failure or two based on flaws after you and your team have done what most in industry would consider sensible diligence. It's quite another to knowingly throw something out to market where you really don't know how it works. If customers or others end up suffering somehow by virtue of a barely understood product you put out, they may seek to recover from you financially.

Attack Vector Risk

AI-generated code can inadvertently introduce security vulnerabilities. Without proper security audits, attackers may exploit weaknesses that AI-generated code fails to recognize.

Any such generated codebase is wholly dependent on the tool for industry best practices. Remember that there is code itself, but also the interaction of code bases with one another as well as the hosting environment. Is the output of an application fully covering all of these issues to the degree needed based on the nature of the project? That is, if you built out a content management system for some specialty publishing need, that will have a very different risk profile than if you put out a new crypto wallet of some sort. Will you even really know what your attack surface area looks like?

Recovery from Failure Risk

AI-driven projects that fail may leave businesses in a difficult position if they don't have the human expertise to debug, maintain, or pivot their codebases effectively.

If a failure does happen, teams overly reliant on AI might not be able to even trace critical errors. Though certainly, AI can be used to do so as well, and to good effect, a novice practitioner here may not be able to do so. Plenty of companies are using ML and AI tools to detect failures of various types and also do Root Cause Analysis. In the case of these types of deployments, if there is a failure, the only recourse is basically to prompt the tool, "Um... can you please help me fix this? I'm possibly in large amounts of trouble now and if you can't fix this code I don't know what I'm going to do."

What's Coming Next

What's coming next is a shift in how we integrate AI coding tools into workflows. Expect improved AI code reviewers, tighter integrations with DevOps pipelines, and a more structured approach to AI-generated code governance.

The AI coding tools are in their 1.0 or maybe 2.0 stages. Some of them deploy whole apps, but have limited ability to reach deep into the code bases themselves. Over time these will get better in general with having prompts fix more issues.

I also expect the following in the very near term:

• Deploy not just apps, but have better control over code. (Via their own custom Integrated Development Environments (IDEs), or plugging into popular options like Visual Studio, JetBrains and so on. (Yes, they already have AI tools perhaps, but I'm talking about directly embedding with - and better understanding - some of the more holistic app generation and deployment systems. Or perhaps being part of an agent based production flow.)
• Code backup to version control systems like GitHub, branching capacities and similar.
• An entire cottage industry of freelance developers and development shops that specialize in using these tools and providing "rescue services" for those struggling with broken projects or needing to take things to the next level.

What I've just outlined is for the professional crowd. For everyone else, what's coming is likely a temporary mess. If you thought the occasional bad HTML site or weak WordPress blog template was bad... what we may see soon could be a whole new level of bad. This is just pure opinion on my part. And yet another place where I actually hope I'm wrong. But we're putting really powerful tools in the hands of lightly skilled producers. Some will create amazing things. Others? Well, we'll see.

Some Successes

The success stories will showcase a new crop of self-made geniuses reaching significant MRR milestones faster than ever before. We'll see a surge in solo entrepreneurs and micro-startups achieving multi-million dollar valuations with minimal staff. The hype cycle will peak with breathless YouTube videos promising billion-dollar companies built by just a handful of people. (We're already seeing this.)

This will be amazing. Some entrepreneurs who would have struggled to bootstrap or get any kind of Angel, (much less VC), funding, might find their way to market whereas they might otherwise not have. And this is great. At the same time, we'll likely also have a trail of wreckage. Think of it like this, the NCAA's report on Probability of Competing Beyond High School shows us a fraction of a percentage of kids playing high school sports ever make it to the pros. (For football alone, it's something like 7.3% play in college, of which 1.6% get to the NFL. We can just round this down to 0.01% chance or so.)

I'm just guessing we'll see a similar power law distribution here.

Flight to Quality

As AI-generated code floods the market, there will be a reactionary shift toward more curated, well-structured, and human-reviewed codebases. Companies will start emphasizing AI-human collaboration rather than full automation. Perhaps even stamps of approval saying "Human Reviewed" will show up alongside "Oragnic" and "No GMOs." Maybe at the corporate level we'll even see product teams with Product Managers, Designers and Developers all working merrily together again!

We'll certainly still see entrepreneurs using new tools to build. As the tools get even better, we may also see a lower technical failure rate from poor execution. And yet, possibly in some ways a higher business failure rate as it will be easier to put bad ideas out as well. Overall though, through natural selection we could collectively end up with a much better universe of digital products.

A New Balance

We're still at the beginning here. When I first started woodworking, I created a lot of extra sawdust when I cut apart some failed early attempts so the parts would fit in the trash. But over time, with a lot of YouTube, I got to where there's some nice fine furniture in our house. There's still some flaws, but usually small and solvable. Still, I can take raw lumber and turn it into something useful. This is where I feel we are with some of these tools. They're good. They can take novices up a level. But it'll take us some time to level up. (Here's our dining room table starting with just a bunch of boards. And the full build progression.)

As we seek a balance, this is the phase where businesses and developers learn how to balance AI coding assistance with human expertise. Teams will develop best practices for when to rely on AI and when to lean on traditional software engineering principles. Right now, it's a footrace - as it's ever been - to see who can get what out faster.

The Timeline

Is any of what I just said true? Will things actually evolve this way?

Five years.

I need to come back and re-read this post maybe 5 years from now and see how I did.