Clearing up ChatGPT pricing
11 June, 2023 — This guide to ChatGPT pricing may be out of date.
ChatGPT can clarify a lot of things but not its own pricing: "As a model based on information available up until September 2021, I don't have access to current pricing information."
Prices have dropped dramatically since paid ChatGPT access was made available earlier this year. The ChatGPT Pro pilot service ($42/month) was halved to $20/month when it was launched as ChatGPT Plus.
In March when GPT-4 was released, GPT-3.5 API access was reduced by a factor of 10 to $0.002 / 1k tokens or, put another way, $2 per 1 million tokens. That's $1.54 to upload all three volumes of The Lord of the Rings (576,459 words total, and a third as many more tokens).
That's so cheap that here at GoodHabitz where we have been building AI-powered learning activities for our online training courses it took us months to spend our $5 of free credit.
That's all great news for us AI developers but there are a couple of gotchas with the pricing. I got surprised a few times, so hope this clears things up!
Accessing ChatGPT
Firstly, there are three ways to access ChatGPT:
Anyone may sign up for a free account for ChatGPT to get access to GPT-3.5. It's also ad-free at the time of writing.
Free accounts compete for the service, which is under heavy demand, so you might see some timeouts. For $20/month, you can switch from the free plan to ChatGPT Plus and this gets you:
ChatGPT Plus is a good fit for power users, with casual or intermittent users making use of the free accounts until they outgrow them.
Now, time for the gotchas...
ChatGPT Plus ≠ playground access
That was initially confusing for me. I had/have unlimited access to ChatGPT and its GPT-3.5 and GPT-4 models via chat and also, for a while, in the platform playground. What I hadn't noticed is that there was $18 of introductory free credit and, when that expired recently, I received this message when I tried chatting in the playground:
That was confusing because the same credentials were still working in chat, and had worked in the playground for months before. What's going on here is that the playground is paid for by the underlying platform access.
I didn't realise it at the time but you pay for the platform (with a credit card on a pay-per-token basis), and that gives you both playground and API access. You can think of the playground as a thin UI in front of the platform API. In fact, the playground has all the parameters you pass into the API like temperature, model, frequency/presence penalties, etc.
We've had API access for a while, and we're transitioning now to two separate organisation IDs: one for prod and one dev. The solution for me here was to add me as a member of our dev platform organisation.
API access ≠ GPT-4 API access
After signing up to the platform (platform.openai.com/login), you can create API keys. That will enable you to embed GPT-3.5 in your applications.
To get API access to GPT-4 you may join the waitlist here: openai.com/waitlist/gpt-4-api. Our request took a couple of weeks; not sure how long the waiting time is now.
OpenAI is prioritising GPT-4 access for companies who are building innovative AI products and for whom GPT-3.5 doesn't quite cut it. We had some use cases where GPT-4 really made the build / no-build difference, and we submitted those with our application.
User ID = organisation ID
Employees access chat via their own accounts using their work emails. Those could be free accounts, ChatGPT Plus accounts (fixed cost per month) or platform accounts (pay-per-token).
Every sign-up creates a new user, which is confusingly not given a user ID but, instead, an organisation ID (a UUID-looking string starting with "org-", e.g., org-CYMgtPyC5jjIEkq7xBVd0nly).
Any user may invite other users to join their organisation by supplying the invitee's email addresses. A user who joins an organisation that way may be part of many organisations (I'm part of both the prod and dev orgs as the orgs' owner), so there's no sense of the organisation owning the members.
Cost tracking = separate prod & dev
At some point, you may want to reconcile your OpenAI invoices to specific AI applications. We keep a token count for the API calls made by each of our AI learning activities so that we can accurately price the end-user cost of providing these services, and also to see how different AI-powered learning activities compare in price. We may also wish to A/B test different models from other vendors in the future ??.
For example, we have an Personal Learning Assistant (PLA) who has been trained on our courses and who can be invoked at any time to answer questions, provide additional education, recommend related courses, assess comprehension, and generally be a supportive guide to the student in their learning journey.
It's an incredibly powerful feature, and we'd like to know all the metrics that you'd expect: duration of interactions, frequency of invocations, and the average number of tokens exchanged so that we can estimate the average cost per use.
领英推荐
If you want 100% reconciliation with the invoices, then the engineers' interactive use via the platform playground is going to slightly bump and perturb the numbers.
To separate the costs of providing AI services on your product platform from the costs associated with their development, it makes sense to have one org ID for prod (which has only an owner and a recovery account as members in it), and another for dev that contains engineers and other internal users who need access to the playground.
ChatGPT = manual opt-out
By default, prompts shared with the non-API consumer products ChatGPT and ChatGPT Pro may be used by OpenAI to improve its models. You can opt out of that by submitting this form here. Unfortunately, that will need to be done once for each user.
For the platform products (API and playground), OpenAI will not use shared prompts to improve its models, unless you explicitly opt into data sharing (here).
Samsung has banned ChatGPT after some of their staff unintentionally shared trade secrets and upcoming plans. Samsung has the might to build its own internal LLM, which is what it's doing now, but that's not an option most of us have.
Instead, we have developed an in-house best practices guide for our staff so that we can support widespread productivity improvements across our organisation while putting in place guidance on the sharing of sensitive information.
We ask all ChatGPT and Pro users to complete the opt out form themselves.
ChatGPT Pro ≠ cheapest option
Given how inexpensive tokens for the platform playground and API are, it would take a true power user to exceed the cost of ChatGPT Pro in a month. They would need to exchange 10 million tokens (~7.5 million words) with GPT-3.5 to spend the $20. Power users are probably using ChatGPT Pro anyway in order to get its other benefits, such as plugins and access to beta features.
For regular or slightly more advanced users who need more than the free ChatGPT service but who aren't power users, playground access is a cost-effective solution.
That allows you to save on ChatGPT Pro licenses. ChatGPT has a nice simple interface and $20/month is not a lot, but it adds up if everyone in a team would like it.
By adding mid-level users to your platform, you're essentially getting their access for free. Switching to units that are a bit more relatable, one upload of The Lord of the Rings with GPT-3.5 costs $1.54. How many months or years would it take someone to type or read that much in chat? Average typing speed is 40 words/minute, so imagine someone types continuously into the playground interface for an hour a day without pausing to think or read the responses; that multiplies out to almost exactly 1 LOTR per year, or $0.13/month. If those users' needs are met by GPT-3.5 then that gets you those users for next to nothing. GPT-4 is 150x more expensive than GPT-3.5. One LOTR per year would cost $1.92/month; that's still a saving over ChatGPT Pro.
But, of course, prompt engineering requires a lot of playing, thinking, pausing, re-phrasing, and carefully considering the responses, so that one hypothetical hour per day of non-stop typing probably corresponds to roughly one heavy-duty user over an eight-hour day.
Under the platform pricing model, spending $1,000/month would be practically impossible, as that's 870 LOTRs. But you could spend that just by giving ChatGPT Pro to 50 people!
So, that's my advice: provide ChatGPT Pro to your power users, and make your staff who need more than the non-free account capabilities members of your dev organisation.
Lastly, another reason to split prod and dev is that you may want to separate prod's billing method from the one used for dev. If an engineer were to write a script that contained a fault and a loop and left it running, and were you to run up a massive invoice, you wouldn't want that maxing out the same credit card used to pay the production account. It's a bit hard to imagine that given how cheap the service is but, as a matter of governance and maybe auditing too, keeping prod upstream dependencies isolated from dev platform access (and with separate credit cards) makes sense.
ChatGPT Pro = no off-boarding
It's easy to off-board members from the platform organisation: you just remove them as members and they can't log in (actually, they can log in but they see the usage limit error message I shared earlier when they try to use it). Off-boarding works because, although a departing member of your team might still have a copy of the API key, accessing the API also requires supplying the credentials of a member of the organisation (which they won't be any more).
The same is not true of ChatGPT Pro. Those accounts are attached to the company's credit card so, when someone leaves, they can still log in with their work email and the password they chose even if that email address has been de-provisioned.
You can't reset the ChatGPT password on those accounts or terminate the subscription, at all. Maybe you could reach out to OpenAI to terminate the subscription of an account with an email address from your organisation. The better way would be asking the departing employee to terminate their subscription before they leave, but you can't really verify they've done that other than asking them to show you, or looking for the absence of the monthly credit card charge. There's no support for automated off-boarding.
SSO will come to ChatGPT subscriptions sometime, but it's not here yet.
This is another good reason to avoid deploying ChatGPT Plus accounts widely within your organisation.
Final advice
A quick summary of the above:
This journey with ChatGPT so far has been amazing. In a short time, we've built some breathtaking new student experiences at GoodHabitz powered by generative AI.
In addition to our Personal Learning Assistant trained on our courses, we have interactive role-playing for managers (think giving performance reviews or having tough conversations), AI comprehension testing, and fun engaging learning activities that weren't possible before.
The pricing is already low, it has reduced by a factor of 10 this year, and it will only get cheaper. If you stick to the pay-per-token model of the platform playground and API, then you're paying the best rates available for all but your most demanding power users.
With the power of these generative AI capabilities, and the creativity we have seen that they have unlocked across our organisation, we can develop products and services we never imagined before. We have barely scratched the surface of what is now possible.
To contact GoodHabitz to talk about our online training or its generative AI capabilities, email [email protected].
AI/Machine Learning
1 年I'd add that there are lots of cost effective alternatives right now including Palm and text-generator.io We use all three for our AI Analyst askFelix.ai Trick is to know the strengths of multiple models, try the best ones first and have fallbacks because reliability is still low right now for all these AI services and response quality varies largely too, Text gen is great at fast autocomplete and doesn't have the sub token issues that other LLM providers have which is a serious issue for most autocomplete solutions. Evaluating if we should use huggingface models too but need their pro offering/then the models are often not as good as commercial ones yet so holding off for now. Another tip of the day is that get on the bleeding edge and try the free offerings like the edit API which is still free after all this time. It can improve text and tends to not change things if its not sure.
Staff Software Engineer
1 年Great read, Steve! ???? I slightly diverge on "Avoid ChatGPT Pro". Being a ChatGPT Pro subscriber, I find it valuable despite my access to GPT-4 API. Reasons: 1. Early Access: Pro subscribers get new features first. Currently, I can use plugins and internet browsing with GPT-4. 2. Cost-Effectiveness: The “ChatGPT Pro ≠ cheapest option” analysis overlooks how chat interfaces use the GPT API. Behind the scenes, the interface sends the entire chat history plus the latest input per request, subject to a max context of 8k tokens for GPT-4 (32k coming soon). This means even a single word can entail up to 8k tokens per request as ChatGPT lacks innate memory and must be ‘reminded’ of the conversation. As a heavy GPT-4 user, relying just on the API would make costs approach or exceed ChatGPT Pro’s $20/month. I verified that in practice. Nonetheless, I do agree with the other points you mentioned for enterprise usage. I'd love a follow-up once we have more metrics on production usage!