Two-Minute Recap of OpenAI DevDay + Insights

Highlights of the Past Year:

• 2M devs using OpenAI AI
• Over 92% of fortune 500 developing apps using the ChatGPT API
• Over 100M weekly active user on ChatGPT

What’s New:

• Introduction of GPT-4 Turbo:
• Expanded context length, now handling up to 128k tokens. (300 pages of content)
• JSON mode - the output is guaranteed to be a valid JSON + the ability to call multiple function calls
• The ability to upload documents or connect databases that will be used to answer (Retrieval - RAG)
• April 2023 is the new cutoff for training data
• Text To Speech (TTS) API ($0.015 per 1k tokens for normal quality - $0.03 / 1k tokens for HD Quality)
• General access to Vision API - you can also use images to prompt ChatGPT by API - 1024 x 1024 px = $0.00765
• DALLE 3 available via the ChatGPT API - you can now output images not just text using ChatGPT API
• Whisper v3 - a new version of the state of the art speech to text model
• Fine-tuning now generally available for gpt3-16k and for gpt 4 they launched an experimental access program
• Custom Models - for a select few OpenAI researchers will help build custom models from scratch
• Higher rate limits (2x Tokens per minute TPM & 2x request limits) + the ability to request rate limits increase - super good for apps that build on top of OpenAI and scale fast
• Sam mentioned that OpenAI doesn’t train on data generated from using ChatGPT or ChatGPT Enterprise.
• Custom GPTs
• Create custom gpt by talking with gpt
• You can upload files to extend the knowledge custom gpts
• You can make custom GPTs: public, private or enterprise
• OpenAI will launch a custom gpt store: you can list a custom GPT and make money with revenue sharing

Assistants API - build assistive experiences for your own apps:

• Threads - conversation state API
• You can upload documents and connect databases to implement Retrieve Augmented Generation RAG without having to handle embeddings or chunk documents.
• Code interpreter + python code sandbox + Function calling - you can translate queries from users in english to custom code or a series of actions in your app.

New Pricing:

• Gpt-4-turbo costs 3x less for input 2x for output than ChatGPT-4 - $0.01 for 1k tokens of input and $0.03 for 1k tokens of output.
• Gpt 3 turbo 16k is also 3x cheaper for input and 2x less for output than before
• Gpt-4-turbo will be the default ChatGPT - including the free version - also no more dropdown to select plugins

My thoughts:

• Customer support will be disrupted with the Assistants API - with a bit coding you can teach GPT to use any software product IMHO this will eliminate at least 80% of customer support tickets and will make software more usable for the average user.
• There will be a gold rush to build and monetize custom GPTs - people who already have an audience - aka influencers - will have an advantage. But the tools released by OpenAI to build custom GPTs seem incomplete - being good at prompt engineering will bring an advantage
• Since gpt-4 will be the default in ChatGPT and you can make custom GPTs that use data - influencers that make money from education will kinda be forced to adopt this technology - the ones that do it first will dominate their niche since chat gpt will freely answer any question they might have in that niche - my hunch is that this is the start of a new kind of info product.
• Many no-code custom ChatGPT apps will not be able to compete with OpenAI - but there’s room for improvement, for example: custom GPTs don’t learn from usage - you can’t give feedback on its answers in order to make it better and custom GPTs don’t have the proper abstraction of state in order to handle complex scripts - which are required in many business applications - voxcode has both - so if you can’t get the results you need with custom GPTs give voxcode a try.