Engineering AI Applications (Part 10): Developing a Video Frame Analyzer Using LLM-Generated C/C++ Code

In this short post, I would like to share a few lessons that we have learnt while developing the AI applications.

Recently, I have created a simple desktop application in C/C++ to analyze video frames (streaming sources will be added later). This time I thought of developing the application using LLM. It is really a pleasant experience how LLMs can "help" in software development. I would like to share a few lessons I learnt.

Scope of the Application:

1. A simple desktop application in C/C++ to analyze video frames.
2. User selects a video input file (stream source will be added later).
3. The application analyzes each frame of the video and produces a report (JSON).
4. The metrics of each frame: brightness, contrast, blur, noise, stability, motion, illumination uniformity, dynamic range, time of day (day or night), light source intensity, histogram uniformity, color balance, white balance score, sharpness score, haze factor, rain detected?, fog detected?, glare severity.

Essential Environment

1. Desktop (Linux)
2. OpenCV + GStreamer
3. ImGUI
4. Multi-threading
5. Programming: C/C++
6. Project: CMake
7. IDE: CodeLite
8. Use of C/C++ libraries
9. LLM: Claude 3.7 Sonnet, OpenAI GPT-4o mini

How I used LLMs

I wanted to test if I can take help of LLMs in generating code with the above specification. I tried using LLMs to develop the application from scratch. The project structure generated was not so impressive. It is OK kind of project structure.

The first code generated did not give desirable results and code appeared to be not so impressive.

I was a bit disappointed. I could not use LLMs to generate a good quality code :-(

Next Attempts

1. I am reasonably proficient in C/C++ and worked on large code bases. I tried to bootstrap the project structure, identified the required libraries and created initial "Hello World" with the project structure that I wanted.
2. I created a high-level design on paper first.
3. I ensured the design is modular.
4. I made some cautious decisions on multi-threading, launching threads per processor core, synchronization mechanism, queues etc. This is the step I started using LLM and building the context for our application. Surprisingly, I started seeing results and validated my design first.
5. I am convinced that LLMs (context) are ready for the code generation for the application.
6. When I started implementing the design, LLMs became my "Team". It was nice experience working with LLMs.
7. I could develop a fully working, and even production grade code in C/C++.

Tips using LLMs

1. Scope your application.
2. Prepare the project structures that suits best of your preferences.
3. Create high-level designs and ensure the designs are modular.
4. Divide the coding tasks.
5. Build the context with small proof of concepts using LLMs. Correct the LLMs with appropriate prompts.
6. Ensure that you instruct (prompt) the LLMs of your choices like coding style, standards, expectations etc.
7. Review the code manually - even though the LLMs can review the code.
8. Do not be in a hurry to generate code. Have some patience :-)

Development Effort:

1. I would say use of LLMs reduces the software development costs significantly. For general purpose applications, I guess LLMs can save 50% to 65% costs provided the project is overseen by an experienced engineer.

A new era in software development has begun. Let's embrace the change and take advantage of AI to build cost-effective solutions.

#AI #LLMs #OpenCV #VideoAnalytics

Affiliations

I write posts for Vidcentum Technologies and VTWAYS where I am founder Director. Some of the articles are published in my blogs / articles / posts as well as on the company timeline / posts.

If you are interested to know more about it, drop a message either to [email protected] or to Maruthi Pathapati or in the comments.