How Do We Use Generative AI like GPT from OpenAI?

(1) For Individuals and Consumers

We use ChatGPT or Gemini or Claude directly from a web browser. See section #3 for the limitations.

(2) For Corporations and Organizations

The way to leverage the nascent technology like Large Language Models (LLMs) is a bit different.

When developing an interface, for instance, web application, to access an external LLM such as OpenAI's GPT-4 model or GPT-4o model via its API, one should be aware of the following important parameters:

A) "temperature"

This is a parameter that would determine how accurate or how creative the LLM response would be.

The default value is 0.6 for OpenAI's models. The lower the more accurate, the higher the more creative. Hence, a value of 0.1 would be fitting for R & D purpose, and 0.7 or 0.8 would be desirable for marketing and sales.

B) Cost Consideration

Because LLM providers such as OpenAI will charge a company based on a base/fixed monthly fee as well "token" usage fee. For simplicity's sake, consider one "token" as one word, that is, company will also need to pay a fee for words used in their "prompt" as well as words in the generated "response". Usually they are very cheap, but they can add up. Hence, your application should consider setting up a maximum token limit.

C) Because LLM like OpenAI's GPT-4 are general purpose your application will perform better if you provide it with "system instructions".

(3) Limitation of using ChatGPT or similar Tools (vs. using Corporate LLM Applications)

a) You are unable to set "temperature"

b) You are unable to set "max token"

Workaround. You can add some sort of "length" or "size" wording for a response such as "Please respond with just one sentence" or the like.

c) Limited ability in managing and organizing output/responses over a period of time.