I Let AI Tools Write My Code: Here's What Happened!!!

A lot of people think that AI tools can replace programmers, and some believe that AI tools are a great help for programmers and can help them expedite their programming. There is a vast majority who have still not started utilizing these tools to boost their productivity.

Well, my article can be a great read for all of these 3 categories of software programmers.

Today, I thought of writing a program and then delegating the same program to various AI tools. I used Microsoft Co-Pilot, Google Gemini & OpenAPI ChatGPT3.5.

Prompt

The first thing we need to understand is that the output from any AI tool will be as good as the prompt that we are giving to it.

For this, I've given the following identical prompts to all these tools.

Forget all previou instructions. You are a .Net programmer. You need to write a method using .Net8 and C#12.

The method calculates and return income tax based on a given income amount. It operates on a progressive tax system, meaning the tax rate increases as income rises. The method args and return types should be decimal.

Tax Brackets:
- Income up to 150,000: No tax
- Income from 150,001 to 500,000: 10% tax
- Income from 500,001 to 1,000,000: 20% tax
- Income above 1,000,000: 30% tax

How it Works:
Input: The program takes an income amount as input.
Tax Calculation: It divides the income into different slabs based on the tax brackets.
For each slab, it calculates the tax by multiplying the taxable income in that slab by the corresponding tax rate.
The taxes from all slabs are summed up to get the total tax.
Output: The program returns the calculated total tax.

Example:
If the input income is 1,100,000:

The first 150,000 is tax-free.
The next 350,000 (from 150,001 to 500,000) is taxed at 10%, resulting in a tax of 35,000.
The next 500,000 (from 500,001 to 1,000,000) is taxed at 20%, resulting in a tax of 100,000.
The remaining 100,000 is taxed at 30%, resulting in a tax of 30,000.
The total tax would be 35,000 + 100,000 + 30,000 = 165,000.        

For the sake of keeping this article focused on outcomes rather than the technical details, I am not putting the output from tools here but we would talk about results.

Outcomes & Observations

First, all these tools generated the method as decimal CalculateIncomeTax(decimal) which is a good start. The signature and return type is no wonder because it was part of my prompt, but the uniform method name was a pleasant surprise.

Me

• Outcome Time - Slowest 25 minutes. The first spent 5 minutes identifying variables and flow.
• Initial Output Accuracy - 5/10 - Took several attempts to rectify the flawed logic.
• Readability - 6/10. - While I took care of nomenclature, my initial code did not contain the best practices and comments.
• Though I am slowest, not perfect at least a human is only the one who can validate the outcome of AI tools output.

CoPilot

• Response Time - Very Fast (2 seconds)
• Initial Output Accuracy - 10/10
• Readability - 9/10. The program was well commented and variable names were more understandable.
• Other observations - Though I had asked for a method to write, it wrote a whole program class for me including the main method and namespace.

ChatGPT

• Response Time - Instant < 1 second
• Initial Output Accuracy - 10/10
• Readability - 9/10. The program was well commented and variable names were more understandable.
• Other observations - it simply delivered what it was asked to do. No namespace which I had to include myself but that's not a deal breaker. It also wrote a unit test method for me which saved me a lot of time for testing outcomes of others.

Google Gemini

• Response Time - Slow > 3 seconds
• Initial Output Accuracy - 0/10 - Out of 5 test cases, only one managed to pass but the logic was still not correct
• Readability - 5/10. Their focus was on line minimization and the code was hard to understand for beginners. There was no commenting.
• Other observations -Though I had asked for a method to write, it wrote a whole program class for me including the main method and namespace. The most frustrating point is that I spent 10 minutes with it for corrections and despite several suggestions and multiple regenerations of code, I could not achieve the right outcome.

Unit Test Results

If you wish to run it yourself or want to see the code, here is the link

https://dotnetfiddle.net/H0ig57

Test cases for method MyMethod
Income: -100,     Expected: 0,     Actual: 0.0,     Result: PASS
Income: 150000,     Expected: 0,     Actual: 0.0,     Result: PASS
Income: 300000,     Expected: 15000,     Actual: 15000.0,     Result: PASS
Income: 500000,     Expected: 35000,     Actual: 35000.0,     Result: PASS
Income: 600000,     Expected: 55000,     Actual: 55000.0,     Result: PASS
Income: 800000,     Expected: 95000,     Actual: 95000.0,     Result: PASS
Income: 1000000,     Expected: 135000,     Actual: 135000.0,     Result: PASS
Income: 1100000,     Expected: 165000,     Actual: 165000.0,     Result: PASS


Test cases for method Gemini
Income: -100,     Expected: 0,     Actual: 0,     Result: PASS
Income: 150000,     Expected: 0,     Actual: 0,     Result: PASS
Income: 300000,     Expected: 15000,     Actual: 0,     Result: FAIL
Income: 500000,     Expected: 35000,     Actual: 20000,     Result: FAIL
Income: 600000,     Expected: 55000,     Actual: 30000,     Result: FAIL
Income: 800000,     Expected: 95000,     Actual: 35000,     Result: FAIL
Income: 1000000,     Expected: 135000,     Actual: 35000,     Result: FAIL
Income: 1100000,     Expected: 165000,     Actual: 55000,     Result: FAIL


Test cases for method ChatGpt
Income: -100,     Expected: 0,     Actual: 0,     Result: PASS
Income: 150000,     Expected: 0,     Actual: 0,     Result: PASS
Income: 300000,     Expected: 15000,     Actual: 15000.00,     Result: PASS
Income: 500000,     Expected: 35000,     Actual: 35000.00,     Result: PASS
Income: 600000,     Expected: 55000,     Actual: 55000.00,     Result: PASS
Income: 800000,     Expected: 95000,     Actual: 95000.00,     Result: PASS
Income: 1000000,     Expected: 135000,     Actual: 135000.00,     Result: PASS
Income: 1100000,     Expected: 165000,     Actual: 165000.00,     Result: PASS


Test cases for method CoPilot
Income: -100,     Expected: 0,     Actual: 0.0,     Result: PASS
Income: 150000,     Expected: 0,     Actual: 0.0,     Result: PASS
Income: 300000,     Expected: 15000,     Actual: 15000.00,     Result: PASS
Income: 500000,     Expected: 35000,     Actual: 35000.00,     Result: PASS
Income: 600000,     Expected: 55000,     Actual: 55000.00,     Result: PASS
Income: 800000,     Expected: 95000,     Actual: 95000.00,     Result: PASS
Income: 1000000,     Expected: 135000,     Actual: 135000.00,     Result: PASS
Income: 1100000,     Expected: 165000,     Actual: 165000.00,     Result: PASS
        

Verdict

In conclusion, while AI tools have made significant strides in assisting programmers, they are not yet a complete replacement for human ingenuity and problem-solving skills.

Tools like Copilot and ChatGPT have demonstrated their potential to boost productivity and code quality, but they still require human oversight and intervention.

As AI technology continues to evolve, the collaboration between humans and machines will likely become even more seamless, leading to more efficient and innovative software development processes.

If you have ever tried AI tools, please share your experience in the comments below.