Considerations for adopting OpenAI in your applications - Part 2
In Part 1 I broke down the considerations for integrating OpenAI into your applications, in this article I am going to show you some simple API calls you can make to create simple functionality powered by AI on GPT-3 and GPT-4.
This is more of a technical article with code samples.
What were we doing again?
So the idea (which was nothing short of revolutionary, not), was to recommend blog article topics for the end user based on their profile information captured from social networks during their sign-up and login.
The application I have has a basic blog functionality.
The two seemed to marry nicely.
What ideas could I steal from you?
Here are some product ideas I could think of (probs already done somewhere!).
Disclaimer, some of these would warrant a private instance of GPT on Azure OpenAI Services down to the level of personal information used. Remember training data also stops at September 2021, so this would be anecdotal and desk research at best. Private, fine-tuned instances could be built to do way more.
Ok, enough of that, show me the code.
Bang, your database is sleeping...
Quick side note: When you use Planetscale and have databases on the free tier, after a period of inactivity, they goto sleep and you need to manually wake them. Quick press of the "Wake Database" button and voila, everything is alive again.
This is one of those perils of using cloud technologies for local development and proof of concept'ing. Personally I find it an acceptable compromise in exchange for the ease of use and zero-infrastructure hassle.
Back to the application
Here is a quick screenshot of the application, it shows the Posts page and gives the user an simple UX to create a New Post +
When I click through to New Post + I am then presented with an editing screen, with a very modern look and feel. I really like the clean pre-filled, yet slightly greyed out text. This style combined with the "grey boxes" during load is called skeleton loading and is a common UX strategy used in web apps of this nature. Loading UI is built into Next.js and is a very handy feature.
The intention is to add recommendations above the "Untitled Post" input box.
So let's start with access to the API
Firstly, I need to prototype my prompts and choose the APIs. I will be using OpenAI credits that I have purchased. Start here;
Once you have done this, the Overview screen will change and show you a heap more content as a paid user.
I am worried about spending, or don't want to spend money...
Well, there are a couple of options for you here, feel free to check them out.
Ok, so your sorted and you have access to the API.
Deciding which API/Model.
It really doesn't matter which model we choose at this point, but it does matter which API we choose. The two we would choose from here are Text completion and Chat completion APIs.
Chat completion is where you can access GPT-4 and is optimised for chat style interactions where you building a more conversational experience. When you instantiate an interaction you would give it "system context" such as "be an over-zealous Australian tour guide". But it also works well for traditional completions.
Text completion is more straightforward and computes based off the string of text you enter. Such as "Give me some blog articles." You get blog articles.
Here is one for you though. I asked the text-davinci-003 model the following question "Whats the difference between the Text completion and Chat completion APIs provided by OpenAI?". Now, according to the docs, this has training data up to Jun 2021... so either these API's existed before that, or something else is going on. I'll let you decide. Anyway, this is what it output.
The Text Completion API takes a piece of text as input and attempts to predict the most likely completion of that text, while the Chat Completion API takes a conversational context as input and attempts to predict the most likely response to a given statement. The Text Completion API is optimized for more traditional, structured writing, while the Chat Completion API is designed to generate natural-sounding conversation.
Which one are we going with?
I have decided to go with the Chat completions endpoint, and use the gpt-3.5-turbo model. I feel this will give me the most flexibility and creative answers for my project. Be aware though, this is a more expensive API in terms of usage.
Why didn't I use GPT-4? Honestly, because I am still on the waitlist! Yes, even though I am a paid subscriber of both ChatGPT Plus and OpenAI, I have to wait.
Ok, enough, so tell me about the Prompt
Using the Playground, I have selected "Chat Beta", gpt-3.5-turbo is the default when working on this API. When you do this, the UI changes to show you the "system" content in a seperate window, although in the API this will be defined as a "message" with a role called "system".
The "User" and "Assistant" messages construct comes up on the right hand side. In my screenshot, the Inputs are in Green, the Outputs are in Red.
One annoying feature of the Chat mode in Playground is that it won't generate code snippets for you. If you want this feature, you maybe better off sticking with the Complete mode or cross-referencing the documentation. Here is an example of what it spits out for you, you can choose different languages in the drop down.
How do I take this to my application?
Disclaimer as mentioned in previous articles and for those of you that know me, I am a 1st class amateur developer. So this article does not constitute programming best practices and you are welcome to criticise any of my code.
You might recall that my application is built using Next.js.
领英推荐
So the first thing I plan on doing is creating a Component that will sit just above the two Text Areas already provided by the Starter Kit.
Being an article about AI and a little bit of an AI fanatic at this point, with paid access to GitHub Co-pilot, ChatGPT Plus and the APIs, I will get attempt to solve this, by letting the tools write the code for me.
Show me the code! Hold on.
The first thought was to goto the Playground and use the Complete mode.
Here was my prompt.
Write me a Next.js component that makes an API request to OpenAI Chat API and displays the response on the screen.
It didn't really work. It gave me code, but there was some context lacking. The Green is the prompt and the Red is the response.
I was able to see some issues here:
This was kind of ok, but I realised I needed to work on my prompt and I was hoping for references to the latest APIs, and ideally some context around what the code was doing. After trying 2-3 more prompt changes, I wasn't happy.
So I switched to using the Chat mode in Playground and it started to give some some more detailed responses. What I like here, is it appeared to speak to Next.js best practices. It created code that was using a NextJS API, it dropped the use of Axios. But I did have to prompt it to provide the API itself.
Lastly, it provided some more context around what it was doing. This is where the two modes differ. Complete simply gives you a straight answer, whereas Chat aims to explain itself and be contextual in its output.
If you remember from earlier in the article, Chat requires a System message to set the context. I wasn't really sure what to put in here, but you can see from the screenshot, I kept it simple.
So, now I am stuck, I have rudimentary code from one mode (Complete, GPT-3, davinci-003) that will need a lot of modernising and refining (I don't have the patience), and I have very detailed, complete code from the other mode (Chat, gpt-3.5-turbo), which will likely require me to research Next.js documentation on the best way to inject it into my project.
For my POC, I am looking for the holy grail...
How do you reckon my chances???
ChatGPT Plus has entered the chat
So I decided to see what ChatGPT Plus would give me. Why?
Well, as I mentioned, in the Playground and APIs I am on the waitlist for GPT-4. But with ChatGPT Plus, I am able to access GPT-4. Promising better context and although it is trained on same data as GPT-3, it appears to provide more up to date code examples and context.
So I am pushing on here with GPT-4.
My initial prompt was the same;
Write me a Next.js component that makes an API request to OpenAI Chat API and displays the response on the screen.
And the response was similar to Playground on gpt-3.5-turbo.
Now, because this is ChatGPT, the UI (which btw is also built using Next.js) is a blazing example of how you can interact with the Chat API, with all of the bells, whistles and preconfigured settings. It is designed to showcase and harness the power of OpenAI models. Including syntax highlighting, rich formatting and conversational context.
I have recorded a few short videos to showcase how I got the code to a working state. This took initially about 15 minutes to bring into my application.
So, I am happy that I have some code that I understand enough to start copy and pasting into my application and attempt to get something working.
What next...
At the end of this video you can see that we have a working integration with the API and results being pumped out to the screen. So what's next.
Let's make it Artificially Intelligent, and look nice
To wrap up this POC and integration, I would like to implement some features that make it more contextual and relevant. That being;
(See video, 9 minutes long, shows code and the working app!);
Here goes.
Wrapping up
So hopefully you feel a little more informed and confident in how you can utilise the OpenAI APIs. Here are my takeaways;
As always, comments welcome and if you would like to discuss this in more detail and see behind the scenes of the code, hit me up in the DMs.
Happy AI'ing.