Why hasn't coding improved with GPTs? How to?

We were so excited with announcement of GPT2 when it started spitting code that we literally had initial shock and a later awe. I suppose it was more like a thunderous applause for an act by a magician which later turns out to be a common card trick. You must know at least one magic trick which awed you once, but now you yourself feel embarrassed to be awed back then. We all do. AI is a science but when mixed with experience tricks, it seems nothing less than magic.

When it comes to the real word usage of GPT to generate meaningful code in software engineering, there are a lot of gray areas. GPTs would provide as smarter answer as your prompt. If your prompt is vague the answer would be equally vague and the generated code would be useless. Prompt engineering has evolved as a new stream of computer technocracy gaining prominence in past few months.

The amount of time that it takes to build prompts for generating a function in a software language outweighs the actual time save or quality gains. A legacy developer would write a basic code and may validate the logic with some standard testing. They may review some logic with StackOverflow or google search for any edge case exceptions. In a similar pattern, a developer using Copilot for coding would spend considerable time to prompt each function and refining those to achieve the desired goal. After developing 10 projects using copilot for 5 months, we learnt that the actual benefits are not clear at all. The developers end up writing a lot of prompts and all the prompt engineering looks like "over-engineering".

No real efficiency gains

The real benefit of copilot is nett zero due to heavy post processing and micro nature of prompts. You end up writing 3X to 4x of prompts for X number of functions. It means that developers who are using prompts to generate code are actually costing more due to added cost of LLM as well as the risk of copyrights.

This is one of the major trends in software engineering circles due to disappointments and anxiety. Are we missing something?

This article explores the reasons of the failure to reap efficiency benefits and possible ways to actually succeed with Copilots.

Micromanaging Fault Of Prompt Engineering

Asking better questions is always the way to find deep answers. But a question can only become better by asking and iterating, or by being more knowledgeable. In the current scenario, developers are iterating for better prompt by seeing the outcomes of AI. If they don't improve their knowledge, the situation will never improve. It has been called insanity by the famous Einstein. We need to change the approach to get any better.

Macro Approach: Bulk Prompt Generation

The creation and perfection of every single prompt is the faulty micromanagement of AI led development process. To improve this process, we firstly created the knowledge base of developmental goal, i.e. we wrote a product story. And then we designed an AI with NLP and NER to abstract Action, Actor & Intention from the whole of product story. We extracted all the bulk prompts from the story. And then we built a master prompt for the computer language of choice for code generation.

Results

This method saves 20% to 40% of raw development time based on 5 months of data since our launch in October 2023. There is a lot of scope of optimizing it further. The process starting from product story to build journey mapping, bulk prompts and master prompt creation is very amateur and we invested our own capital of USD 1M earned by bootstrapping our startup for last 3 years. Just imagine the possibilities if this becomes mainstream! We can truly achieve our vision of democratized technology. Anybody with an idea, irrespective what one knows about computers, can achieve their dreams. That will be very empowering.

Our learnings

A lot of early prospects whom we approached rejected our idea because their dev team was busy in trying the usual prompt based coding. And we didn't know if we can tell them any better. We ourselves didn't know. Since that time, and until now, we have enough data to show them clearly that straightforward prompting would yield no clear benefit, rather it is costly.

Path Forward

Before we can tell you the path forward, you should know the problem which we are solving.

Technology implementation requires intensive planning and organizing. Be it a new software development, or a software integration, planning requires a lot of manual work and ecosystem knowledge. Such product companies bear key man dependency due to this factor where only a few know the details of ecosystem to make planning and strategies. Technology development is also a very unstructured process where code generation by different types of engineers are often not reviewed due to lack of managerial coverage or competing priorities which creates future problems. For example, usage of loosely bounded while loops or unsafe class patterns are hard to check.?Similarly the coding structure and code architecture is a loosely followed principle with a lack of lack of standards and vast expanse of possibilities. The code and software quality suffers in the longer run due to such factors which are discovered only the longer runs. Any standard enterprise software has trillions of lines of code and it is not just impossible but also certain death case for every product, but a lack of better option has led people to the same direction.?Also the field of software engineering has the problem of long service life whereas people who build or maintain it has much shorter life in that team. Even the technology framework like languages which are used to develop such products have shorter life due to which the products require constant upgrades leading to total destruction of value. All these problems of planning, coding, debt, architecture, key man dependency and code quality are connected to one another. The surmounting tech debt of products and companies is not going to find any solution if something is not done.

We are solving this problem. It is not just important, but a must for humanity to progress further. Our world is so much mired with very basic IT problems with fancy bandaids and walls that we don't see them very clearly or have assumed that the high wall is a sky. Food, water, sanitation, healthcare, finance, banking, governance; everything is in shambles everywhere. And I am not talking about some shanty of Mumbai. It is as bad in New York City as in Shanghai. We are living in a makeshift IT world where we are made to feel comfortable while the whole world is burning in bad technology.

There is a way to make it better. Better softwares can heal our world which hugely depends on technology to stay connected, tick and prosper. Every person needs that empowerment and not just fancy engineers who know coding to some extent.

The situation is similar to driverless cars. We humans can drive very well. And some of us can drive so well in F1 circuits. But humanity needs driverless cars and trucks so that the coming times of logistics and transportation is reliable and not dependent on some sleepy or drunk driver. The stakes are higher in coming times. That's why driverless cars and drones are so much important. Humans can have the super human capability, but can also super-powered machines and technologies. It is always the right way to reduce entropy using technologies which reduces chaos and brings more order. Driverless cars and drones are going to reduce the entropy even after increasing the number of elements in the system exponentially.

Bulk code generation is going to be that superpower which the world needs, and their is no other option.

Note: I am founder of Cyberium and developer of the platform along with my super team. We do bulk code generation of our own.

Build product story at FastBuilder.Cyberium.Info