Streamlining API Integration with AI: A ChatGPT-Assisted Development Journey

Introduction

I can now tell you first-hand that coding with AI is awesome.

For the following project, AI:

• dramatically accelerated up my development time
• gave me an initial codebase to iterate on
• helped me debug and complete my project ahead of schedule

And I’m not alone.

92% of programmers are using AI tools - according to a recent study by Github and Wakefield Research. https://www.zdnet.com/article/github-developer-survey-finds-92-of-programmers-using-ai-tools/

And it’s leading to rapid efficiency improvements.

In a 2023 study, programmers with AI assistance completed the task 56% faster than the control group https://arxiv.org/pdf/2302.06590

McKinsey claims up to 100% efficiency improvements in their study https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/unleashing-developer-productivity-with-generative-ai

The following is an example of a challenging API integration problem I was faced with - and how I streamlined the development process with AI.

The Challenge

When a client wanted a flat file, I was floored.

We have a proprietary data asset with many real-time API integration patterns. The API is built for easy downstream integration via an optimized search API - but doesn’t have explicit functions for extracting flat files.

The client wanted a flat file for:

• human-readable purposes as an evaluation of a sample set of our data
• Integration purposes - to support flat-file integration models based on the latest data in our API

I had no idea how to do that, but knew I probably needed to use code.

The main challenge - I’m not a full time developer. I have a degree in computer science and can understand code, but am not up to date on my syntax or libraries.

Challenges included:

• How was I going to start building this script?
• Where were my “bare bones” to start iterating??
• How was I going to refine that script and get it to completion with minimal time investment on my part?
• How would I keep up with code considerations? (Security, Clear / Elegant code, Easy to follow / adapt, Well-commented / described, Follows best coding practices and API usage practices)

Also - API integrations present their own challenges:

• Different authentication models
• Different pagination models
• Different response and JSON structures

And I had a tight timeframe on top of that.

The AI Solution

I explained this to my CTO, he sympathized and suggested I try coding with ChatGPT.

So that’s where I started.

I went to ChatGPT and typed in the following prompt:

ChatGPT Prompt:

write a python script that hits an api endpoint and paginates through the result set and saves to a csv file

And lo and behold - it generated some code!

But the code was flawed.

So I decided to provide ChatGPT with some clarification to rewrite the code.

First, it had no authentication.

Our API requires an API key in the header in the form of “authorization : Token your_api_key”. The generated code had no authentication.

So that was my first fix:

ChatGPT Prompt:

Our API requires an API key in the header in the form of “authorization : Token api_token_here”? - rewrite the code

And it did a pretty good job!

It was also doing the wrong pagination model.

For example, the script was looking for a variable called “has_more” - and looping pagination based on whether it was populated.

Our API just provides a variable for “total_count”.

That was my next fix.

ChatGPT Prompt:

Our API doesn’t have a “has_more” key and instead returns “total_count” in the “parameters” object in the JSON response - rewrite the code to paginate through the total count by page size

And again - the code looked pretty good!

It was also the wrong data model.

It was looking for the returned data in an array called “results” but ours is called “data”.

That was my next fix.

ChatGPT Prompt:

The JSON response contains an object “data” that is an array of JSON objects representing the results set - rewrite the code

Now I had my “bare bones” for iteration - and I didn’t have to “write” anything yet!

Debugging and Error Handling

I created a new Python script in VSCode, copy / pasted the AI-generated code, hit run - and immediately got an error.

This happened many more times than I’d care to admit.

Debugging can be so incredibly time intensive if you don’t have many insights and don’t know where to start.

But who better to help me debug than ChatGPT!? After all, they wrote the initial code.

For example, when I got an error from running my script, I entered the following prompt to ChatGPT:

ChatGPT Prompt:

I’m getting the following error: <error from console>

Help me debug it

And help me debug - it did!

It modified the script to include debugging mechanisms:

• Printing out values of variables at different points in the code to give me insight into where it was breaking down
• Try / Catch exception handling
• Other error handling and messages

I could then modify the code with a few manual tweaks and ask ChatGPT for any syntax help needed along the way.

Then

1. run again
2. see where it broke again
3. Use ChatGPT to debug / fix
4. rinse and repeat!

Until finally I had a working product.

This saved me so much time

• I didn’t have to bug my coworkers or post questions on stack overflow
• ChatGPT wrote the error handling items - breakpoints, error and variable outputs, etc. to add to my script to get exact insights into where my code was breaking.
• I could answer my own incredibly specific questions I needed to complete the fix and progress to the next part of the code.

And the resulting product was high quality

• Code complete with exception handling and good security standards
• Well-commented and easy to follow
• Easily adaptable, didn’t rely on “magic numbers”, etc.
• Flexible to create many different types and formats of flat files for different integration needs

Learnings

1. Next time, I would start with a much more specific and detailed prompt to generate the initial code. Include authentication model, pagination model, data model, and any additional details I learned I needed through my edits / debugging process
2. Example ChatGPT Prompt: write a python script that hits an api endpoint and paginates through the result set and saves to a csv file Our API requires an API key in the header in the form of “authorization : Token api_token_here” Our API pagination model is to include “total_count” in the “parameters” object in the JSON response and allow you to specify “page” and “page_size” in the API call parameters The JSON response contains an object “data” that is an array of JSON objects representing the results set
3. Look first to understand the code that was written, then adjust. It’s well-commented.?Understand what the code does and where it might be flawed. Running the code, getting an error, and trying to debug the error can be time consuming

Conclusion

Generative AI is awesome for coding. It’s not the silver bullet that will kill your developers just yet - but it is a significant accelerator.

Benefits include:

• Efficiency and Productivity Gains
• Innovation and Competitive Advantage (Code writing, Code review, Quality assurance / testing, Debugging, Design and Architecture)
• Scalability and Quality Improvement (Improved code quality, Accelerated learning of the codebase)
• Developer Satisfaction and Talent Attraction

I personally loved the experience. It’s rekindled a love for code that I haven’t felt in awhile.

For example, I hate learning and looking up syntax (ie. is it console.print or console.log or print(“text”) or printf? - I don’t give a $%^# as long as it prints!). And syntax changes drastically from language to language.

ChatGPT will write the syntax based on your description of what you want the code to do - the more descriptive the better. No more sifting through function documentation and code examples.

I love how ChatGPT helps me code!

How will you incorporate AI into your development stack today?