Considerations for adopting OpenAI in your applications - Part 1

Considerations for adopting OpenAI in your applications - Part 1

Yet another article about AI.

In this article, I dive into the foundations of OpenAI and things you need to consider when integrating to your applications. In Part 2, I walk through some code examples from a recent POC application.

So you have tried ChatGPT...

It is fair to say that many of us have tried ChatGPT, which for most end users is an interface into GPT 3.5. For paid users of ChatGPT Plus (which is different to an OpenAI subscription), you will have access to ChatGPT 4. Albeit, OpenAI are rapidly improving, testing and changing how they go to market, so this could change at anytime.

No alt text provided for this image

I highly recommend subscribing to ChatGPT Plus, as an end user it provides a higher level of depth and understanding, and it seems to be able to course correct throughout the life of a conversation much better.

I'll save some common prompts I have been using that I will save for another post. But suffice to say, I have found it extremely useful with many day-to-day tasks.

Disclaimer

This is not a post about which AI and ML platforms are "the best". This is about the basics of OpenAI, real-world gotcha's you need to look out for and how I was able to quickly use ChatGPT to integrate GPT-4 into an application!

There is a huge world of capability out there and your use cases will likely need access to a specific model, your own model, or even an amalgamation.

I highly recommend following these people who regularly have the lo-down on the best AI, ML and Generative AI platforms;

First, lets talk about Models and API's

For those of you building "products", such products that end users and businesses need to interact with, ChatGPT Plus (which is a UI, not a model or API) is nothing more than a test ground and simple interface to the AI world.

As such, you don't integrate with ChatGPT, you integrate with OpenAI, their models and their API's.

If you are serious about integrating AI technology into your application you will need to decide which model you need and the API's, moreover, depending on your business requirements, you may need to access the API's in a specific way.

Here is a quick overview of the models, taken from the Open AI website;

No alt text provided for this image

It is important to note that whilst OpenAI has tried to streamline their API to be "model agnostic" and help you grow in the future, each of these Models has certain features, flags that are access through the respective APIs. The API is extensive and has a large range of endpoints for common operations, typically most applications would use the Completions, Chat and Image APIs.

There isn't much point in me listing it out here, so I recommend you read about the APIs available to you here.

Let's start with a simple use case for AI

Like all endeavours of this nature, if you don't have a plan, it might not work out how you expect and you probably shouldn't be spending time on it. But, in fairness, that plan, may be to uncover use cases for AI in your organisation (especially when AI is currently the #1 topic of CEO's and Boards across Australia https://iot-analytics.com/what-ceos-talked-about-in-q1-2023).

Now, there are probably 1000's of use cases for AI that never really need developing or building for. But like many technological advancements, the initial wave of hype typically breeds lots of "innovation", but few of them make it.

Side note: If you haven't read the Gartner Hype Cycle for AI, you should dive in; What’s New in Artificial Intelligence from the 2022 Gartner Hype Cycle?. This is essential reading for any stakeholder in the AI space.

Now, the use case for this application is relatively simple, referring back to the recent POC application I started, it includes blogging functionality. The use case I have chosen here is to collect a small piece of information from the user, and recommend blog article topics that the user can start their blog from.

The reason why OpenAI and GPT is a good candidate for this is down to the training data it has. GPT-35 Turbo and GPT-4 models are up to Sept 2021.

Side note: Using the fine-tuning feature, you can prepare your own data to add to your model. You can read more about fine-tuning here: Fine-tuning - OpenAI API. This is an advanced topic and might be best left for later.

How does this idea breakdown?

Whilst my application is quite simple, it does collect some personal information during the sign up process. The act of recommending blog articles could use that information for better prompts such as age, location, bio.

So, what can I do from a recommender perspective with this;

  • With Age, I can prompt the API to tailor the response to information from a specific era or relevance to that persons age. For example, if the users bio contains "film fanatic" and I know they were born in 1985, I could prompt the API to be contextual like this.

No alt text provided for this image

  • With Location, for example, if the users Bio contains "Fishing, hiking and adventure", I could prompt the response to deliver this type of context.

No alt text provided for this image


Although I am using ChatGPT here, this is a good way understand how prompts work with the information you have and depth and context in the responses. I typically take notes about different prompts and what I liked/disliked about them. This helps me remember and optimise later.

You can also use the OpenAI Playground, but this requires a paid OpenAI subscription and not easily available to all users. Playground - OpenAI API.

These prompts, after an hour or two of testing and refining work for me.

Easy, give me the code, I am ready...

No alt text provided for this image

At this point, we need to address some things when it comes to privacy and security. Many use cases of AI require some data to be input from your end users, and deciphering what is personal, private and identifiable is going to be a very large problem that will need governance and best-practice.

Here are some things you need to know when using OpenAI products;

  • OpenAI continually uses wording about their products and models such as "beta", "prototypes", "research". This, by design (and it isn't a bad thing) gives them some flexibility when it comes to data privacy.
  • ChatGPT and their consumer focused, non-API services, for example, when used via chat.openai.com can and will use your data to train the model. You can read more about this here, Data usage for consumer services FAQ | OpenAI Help Center.
  • You can opt-out of data sharing when using Consumer services, but this is not retrospective.
  • OpenAI models accessed by their own API's will not use your data. You can read more about this here, API data usage policies (openai.com).
  • Non-consumer services, through the API can opt-in to data sharing. This can be beneficial depending on your use case.

Right, privacy, what does this mean for me?

Just to be clear, privacy is a responsibility that is on YOU!

Data sharing with 3rd party services is heavily regulated in almost all major geographies, in Australia the umbrella is The Privacy Act 1988. Which includes the Australian Privacy Principles. All of which are relevant to the use of Personal Information and AI. Principle #1 is probably the most important "open and transparent management of personal information".

If you are putting any data into any 3rd party, you need to be 100% sure that

  1. you either own it, or
  2. have the rights to use it, and
  3. you are clear how that data will be used.

If you haven't already, you will likely need to modify your Privacy Policies and documentation to that effect. Engaging privacy professionals early is an important part of your product development journey.

What about more advanced security and capabilities

Organisations are going to come across a number of concerns with the prevalence of AI, let's do a quick overview on the top ones;

  1. Employees using ChatGPT, with free and open access to ChatGPT, employees will be likely be tempted to trial it. Ars Technica reported that ChatGPT had hit 100 million actively monthly users. Quite something! As employees face numerous articles with "useful prompts" and "how to use AI to be better at work", there is risk key financial information, company business plans or even PII data from excel tables can make its way in.
  2. Technical resources prototyping, in my experience, rarely do teams deep into SDLC's have bulletproof testing capabilities, such as fully abstracted test API's with test data, and a framework for quickly deploying testable versions of their applications. I know there are plenty who do, but even those who do, are often senior and therefore also left with access to Production API's just the same. What I am saying, is it is murky! In the early stages of feature development, it can often be too easy to get caught up with some level of production data or IP mixed up in these systems. Be mindful.
  3. Operational security (OPSEC), assuming you have made choices that are legally verified, privacy centric and technically sound, as with any technology platform, the OPSEC model comes into play. Some of the fundamentals of OPSEC include minimum access, device restriction, network control and change management. This is something rarely offered by an API directly from the provider, instead achieving your organisations OPSEC standards is often passed off iPaaS and API management providers such as Azure APIM, Boomi and Apigee.

Ok, now I am worried, what does this mean!

It means that beyond defining a solid use case for OpenAI and writing code to integrate, once you get serious you are going to need features that go beyond just the simple APIs, including;

  • Data Retention Controls
  • Data Encryption
  • Private Networking
  • Managed Identity
  • Content Filtering
  • Moderation Controls & Rules

Whilst some organisations may have these solutions and controls in place, they may not be suited to next-generation AI services, or they may require an extra layer of services and capabilities to suit. I plan to talk about potential integration architectures and patterns for AI services in a future post.

What is important is that integrating directly with OpenAI may not be appropriate for your organisation and a layer of services in front could be required.

Azure OpenAI Services has entered the chat

It may seem obvious, it may not, but OpenAI is actually hosted on Microsoft Azure already. But there is a dedicated Microsoft Azure product called Microsoft Azure OpenAI Services. On the Microsoft Learn website, there is a solid article that explains what it is, What is Azure OpenAI Service? - Azure Cognitive Services | Microsoft Learn. Save this link for later.

It is also important to note that AI and Machine Learning are not new to Microsoft Azure, they have a pedigree in this space and many years of experience in delivering Cognitive Services and Advanced Language Models.

Where the Azure OpenAI service excels is the "enterprise" layer it delivers on top of OpenAI. For those companies who typically develop within the Microsoft Azure world, it will be a familiar experience. For those who are using competing clouds and/or alternative solutions, it should not put you off.

With Azure OpenAI, customers get the security capabilities of Microsoft Azure while running the same models as OpenAI.

In the context of this article, there are a couple of critical things (that are correct at the time of writing, April 2023) that Azure OpenAI does, beyond OpenAI direct.

  • Azure OpenAI Services can be deployed in a region of your choice. Regions are limited right now (check out which models are support in which regions here) and not available in Australia yet. In contrast to OpenAI, where you do not get a choice where your data is processed.
  • Azure OpenAI Services supports encryption with customer managed keys.
  • Azure OpenAI Services supports private networking and endpoints, allowing you to connect to your VNETs and applications in the same manner as your existing integrations.
  • Azure OpenAI Services

You can read about all the advanced features on the Microsoft Learn, What is Azure OpenAI Service? - Azure Cognitive Services | Microsoft Learn.

These enterprise features also have somewhat of a hidden benefit, by which I mean, enterprise adoption. In my experience, when petitioning an organisation for approval of such services, the Microsoft badge carries some serious credibility. Whilst no technology endeavour should be signed off flippantly, you will in many cases encounter decision makers who have been successful in the when sponsoring technologies through the Microsoft platform.

There are some limitations though, you will need to request approval from Microsoft, some APIs and response types are not yet fully supported, and you will likely be waiting a short while behind OpenAI's releases for it to appear, although I expect this will reduce significantly over the coming years.

As with other Microsoft Azure products that have embedded 2nd or 3rd party technology (such as Microsoft AKS), this approach has not caused any significant disadvantages to the majority of users.

If the capabilities you need are exposed through Microsoft, and you are not disadvantaged, then I would choose this route.

When will I need these "layers", how do I choose?

Honestly, that is subjective to your project, but here is my super-short checklist that should suit most projects getting off the ground;

  1. Define your user stories
  2. Choose an appropriate model
  3. Create sample prompts for each user story
  4. Prototype them using ChatGPT or Azure Studio
  5. Optimise your prompts (you'll need this long term!)
  6. Do not put any personal information in these systems yet
  7. Produce a solution statement that defines use case, model, API, request/response, data privacy and so on.
  8. Decide OpenAI vs. Azure OpenAI Services, vs. other.
  9. If you have privacy requirements, go Azure.
  10. If you plan to fine-tune with custom data, go Azure.
  11. You may choose to go direct for early stages of your engineering and Azure for Production, you may choose to go straight to Azure.
  12. Continue developing

Useful Tools & Links

Summary

It's time to bring this Part 1 to a close, I hope that so far you have picked up a few handy hints and tips on where to start with OpenAI development and things to consider as you ramp up your practice internally. For those who have existing AI/ML practices, you will be well versed in how to approach OpenAI, for those just starting, the most important thing to steer clear of, is putting in any data that can compromise your organisation.

In Part 2, I'll show you how I made the decision to stick with OpenAI direct, and develop API code for the OpenAI API, using ChatGPT!

Appreciate your feedback in the comments.

要查看或添加评论,请登录

Anthony Hook的更多文章

社区洞察

其他会员也浏览了