ChatGPT on Steroids: My Techniques to Accelerate OpenAI up to XX Times (+code & templates)

I leverage OpenAI to tackle problems and tasks daily. Exploring AI technologies has led to encountering progressively intricate and captivating challenges. Currently, 80% of my tasks go beyond the capabilities of the chat.openai.com interface, requiring the use of the OpenAI API.?

Examples of tasks I solved:

• Labeling data: Get the theme of the influencer according to his content and bio (100.000 + dataset).
• Structured data extraction: Extract emails, phones, and websites from tons of Instagram bios.
• Content generation: Write 1k telegram-channel posts.
• Data preparation for fine-tuning: Prepare dataset in JSONL for smooth fine-tuning.
• Speech-to-text recognition: Analyze 10.000 hours of call recordings.
• Computer vision tasks: Analyze call to action of 20.000 product images.

I want to share my own ways to speed up OpenAI based on my experience. You can ask for the desired method in the comments:

• #1 "Nocode Template" - I will send a JSON template for the Make platform.
• #2 "Spreadsheet Template" - I will send a spreadsheet with App Script
• #3 "Python Processor" - I will send a Python multi-thread script.

Important notes:

1. You need a OpenAI account with API key.
2. There are limits on the number of tokens processed per minute (TPM) and the number of requests sent per minute (RPM). For newly created accounts, the restrictions are more stringent.
3. Not all the models are available through the API, even if you have a Plus subscription.?
4. Plugins available to you in Plus don't work via API.

The methods are of different labor intensity, with pros and cons. All methods I have tried on myself, I describe only my experience.

Method #1 / No-Code + Google Sheets

First of all we prepare all the data in Spreadsheet. We build a script in the visual editor Make that processes the original Spreadsheet line by line and writes the result into a neighboring cell.

?? Pros:

• It is the most elementary solution. There are some ready to use templates from the No-code platforms.
• It doesn't require specific technical skills. Quick and simple for beginners.
• You can add some additional logic without delving into development.

?? Cons:?

• You need to use paid platform for No-Code.
• Lowest speed.

?? Speed: synchronous execution, approx. one per second.

Write #1 "Nocode Template" in comments  I will share a JSON template for the Make platform.        

Method #2 / Google Sheets + AppScript

It is similar as the first method. It does not require No-Code platform, flexible and customizable.

The formula (custom app script) used to access Openai servers.

The only thing is that you need to find a working script that interacts with OpenAI API.

?? Pros:

• If you have a ready-made script, it's easy to start using it. Copy the workbook, insert your token, prompt data. Pulled the formula -> got the result.

?? Cons:?

• You need to work with the workbook very carefully. For example, after you get the result, you need to save the data as text. Otherwise, after reopening Spreadsheet, it starts updating the data again.
• It is tricky to work with large amounts of data. If you stretch the formula too much, hundreds of queries will launch. It may cause errors by TPM RPM restrictions.

?? Speed: about 3-5 queries per second when processing a document line by line.

Write "#2 Spreadsheet Template" - I will send a Spreadsheet with App Script.        

Method #3 - Interacting with the API using code

We directly interact with the OpenAI API using any programming language - the speed and result depend on our code. Interacting with OpenAI servers is possible in both single-thread and multi-thread modes.?

I started with a single-thread operation. I reached multithreaded implementation with the following options:

• Work with the number of tokens per minute TPM and requests per minute RPM.
• Multiple attempts in case of request failure with exponential delay
• OpenAI functions and model parameters work.
• The code uses as many threads as possible to fit within the TPM and RPM constraints.

Example: Extracting structured information from a data set

?? Pros:?

• The fastest way I know of to work with the OpenAI API in batch mode.
• This solution is recommended by OpenAI and posted in the cookbook. I just refined it for myself: Openai functions and a few other things didn't work.

?? Cons:?

• You need to prepare the input JSONL-file with all data.
• Handler of the output JSONL-file with results required.

??? Speed: 10+ queries per second, 1.000.000+ tokens per minute

Write "#3 Python Processor" - I will share my Python multi-thread script.