Considerations for adopting OpenAI in your applications - Part 1
Yet another article about AI.
In this article, I dive into the foundations of OpenAI and things you need to consider when integrating to your applications. In Part 2, I walk through some code examples from a recent POC application.
So you have tried ChatGPT...
It is fair to say that many of us have tried ChatGPT, which for most end users is an interface into GPT 3.5. For paid users of ChatGPT Plus (which is different to an OpenAI subscription), you will have access to ChatGPT 4. Albeit, OpenAI are rapidly improving, testing and changing how they go to market, so this could change at anytime.
I highly recommend subscribing to ChatGPT Plus, as an end user it provides a higher level of depth and understanding, and it seems to be able to course correct throughout the life of a conversation much better.
I'll save some common prompts I have been using that I will save for another post. But suffice to say, I have found it extremely useful with many day-to-day tasks.
Disclaimer
This is not a post about which AI and ML platforms are "the best". This is about the basics of OpenAI, real-world gotcha's you need to look out for and how I was able to quickly use ChatGPT to integrate GPT-4 into an application!
There is a huge world of capability out there and your use cases will likely need access to a specific model, your own model, or even an amalgamation.
I highly recommend following these people who regularly have the lo-down on the best AI, ML and Generative AI platforms;
First, lets talk about Models and API's
For those of you building "products", such products that end users and businesses need to interact with, ChatGPT Plus (which is a UI, not a model or API) is nothing more than a test ground and simple interface to the AI world.
As such, you don't integrate with ChatGPT, you integrate with OpenAI, their models and their API's.
If you are serious about integrating AI technology into your application you will need to decide which model you need and the API's, moreover, depending on your business requirements, you may need to access the API's in a specific way.
Here is a quick overview of the models, taken from the Open AI website;
It is important to note that whilst OpenAI has tried to streamline their API to be "model agnostic" and help you grow in the future, each of these Models has certain features, flags that are access through the respective APIs. The API is extensive and has a large range of endpoints for common operations, typically most applications would use the Completions, Chat and Image APIs.
There isn't much point in me listing it out here, so I recommend you read about the APIs available to you here.
Let's start with a simple use case for AI
Like all endeavours of this nature, if you don't have a plan, it might not work out how you expect and you probably shouldn't be spending time on it. But, in fairness, that plan, may be to uncover use cases for AI in your organisation (especially when AI is currently the #1 topic of CEO's and Boards across Australia https://iot-analytics.com/what-ceos-talked-about-in-q1-2023).
Now, there are probably 1000's of use cases for AI that never really need developing or building for. But like many technological advancements, the initial wave of hype typically breeds lots of "innovation", but few of them make it.
Side note: If you haven't read the Gartner Hype Cycle for AI, you should dive in; What’s New in Artificial Intelligence from the 2022 Gartner Hype Cycle?. This is essential reading for any stakeholder in the AI space.
Now, the use case for this application is relatively simple, referring back to the recent POC application I started, it includes blogging functionality. The use case I have chosen here is to collect a small piece of information from the user, and recommend blog article topics that the user can start their blog from.
The reason why OpenAI and GPT is a good candidate for this is down to the training data it has. GPT-35 Turbo and GPT-4 models are up to Sept 2021.
Side note: Using the fine-tuning feature, you can prepare your own data to add to your model. You can read more about fine-tuning here: Fine-tuning - OpenAI API. This is an advanced topic and might be best left for later.
How does this idea breakdown?
Whilst my application is quite simple, it does collect some personal information during the sign up process. The act of recommending blog articles could use that information for better prompts such as age, location, bio.
So, what can I do from a recommender perspective with this;
Although I am using ChatGPT here, this is a good way understand how prompts work with the information you have and depth and context in the responses. I typically take notes about different prompts and what I liked/disliked about them. This helps me remember and optimise later.
You can also use the OpenAI Playground, but this requires a paid OpenAI subscription and not easily available to all users. Playground - OpenAI API.
These prompts, after an hour or two of testing and refining work for me.
领英推荐
Easy, give me the code, I am ready...
At this point, we need to address some things when it comes to privacy and security. Many use cases of AI require some data to be input from your end users, and deciphering what is personal, private and identifiable is going to be a very large problem that will need governance and best-practice.
Here are some things you need to know when using OpenAI products;
Right, privacy, what does this mean for me?
Just to be clear, privacy is a responsibility that is on YOU!
Data sharing with 3rd party services is heavily regulated in almost all major geographies, in Australia the umbrella is The Privacy Act 1988. Which includes the Australian Privacy Principles. All of which are relevant to the use of Personal Information and AI. Principle #1 is probably the most important "open and transparent management of personal information".
If you are putting any data into any 3rd party, you need to be 100% sure that
If you haven't already, you will likely need to modify your Privacy Policies and documentation to that effect. Engaging privacy professionals early is an important part of your product development journey.
What about more advanced security and capabilities
Organisations are going to come across a number of concerns with the prevalence of AI, let's do a quick overview on the top ones;
Ok, now I am worried, what does this mean!
It means that beyond defining a solid use case for OpenAI and writing code to integrate, once you get serious you are going to need features that go beyond just the simple APIs, including;
Whilst some organisations may have these solutions and controls in place, they may not be suited to next-generation AI services, or they may require an extra layer of services and capabilities to suit. I plan to talk about potential integration architectures and patterns for AI services in a future post.
What is important is that integrating directly with OpenAI may not be appropriate for your organisation and a layer of services in front could be required.
Azure OpenAI Services has entered the chat
It may seem obvious, it may not, but OpenAI is actually hosted on Microsoft Azure already. But there is a dedicated Microsoft Azure product called Microsoft Azure OpenAI Services. On the Microsoft Learn website, there is a solid article that explains what it is, What is Azure OpenAI Service? - Azure Cognitive Services | Microsoft Learn. Save this link for later.
It is also important to note that AI and Machine Learning are not new to Microsoft Azure, they have a pedigree in this space and many years of experience in delivering Cognitive Services and Advanced Language Models.
Where the Azure OpenAI service excels is the "enterprise" layer it delivers on top of OpenAI. For those companies who typically develop within the Microsoft Azure world, it will be a familiar experience. For those who are using competing clouds and/or alternative solutions, it should not put you off.
With Azure OpenAI, customers get the security capabilities of Microsoft Azure while running the same models as OpenAI.
In the context of this article, there are a couple of critical things (that are correct at the time of writing, April 2023) that Azure OpenAI does, beyond OpenAI direct.
You can read about all the advanced features on the Microsoft Learn, What is Azure OpenAI Service? - Azure Cognitive Services | Microsoft Learn.
These enterprise features also have somewhat of a hidden benefit, by which I mean, enterprise adoption. In my experience, when petitioning an organisation for approval of such services, the Microsoft badge carries some serious credibility. Whilst no technology endeavour should be signed off flippantly, you will in many cases encounter decision makers who have been successful in the when sponsoring technologies through the Microsoft platform.
There are some limitations though, you will need to request approval from Microsoft, some APIs and response types are not yet fully supported, and you will likely be waiting a short while behind OpenAI's releases for it to appear, although I expect this will reduce significantly over the coming years.
As with other Microsoft Azure products that have embedded 2nd or 3rd party technology (such as Microsoft AKS), this approach has not caused any significant disadvantages to the majority of users.
If the capabilities you need are exposed through Microsoft, and you are not disadvantaged, then I would choose this route.
When will I need these "layers", how do I choose?
Honestly, that is subjective to your project, but here is my super-short checklist that should suit most projects getting off the ground;
Useful Tools & Links
Summary
It's time to bring this Part 1 to a close, I hope that so far you have picked up a few handy hints and tips on where to start with OpenAI development and things to consider as you ramp up your practice internally. For those who have existing AI/ML practices, you will be well versed in how to approach OpenAI, for those just starting, the most important thing to steer clear of, is putting in any data that can compromise your organisation.
In Part 2, I'll show you how I made the decision to stick with OpenAI direct, and develop API code for the OpenAI API, using ChatGPT!
Appreciate your feedback in the comments.