New plugins mean ChatGPT can carry out live web searches

Open AI has revealed details of ChatGPT’s new plugins, which allow the chatbot to access up-to-date information, and use third-party services. Oh, and it can now write its own code.

ChatGPT has been given a live internet connection, supercomputer maths skills and the ability to network with more than 5,000 everyday web apps, including Gmail, HubSpot and Salesforce.

And if that wasn't enough, ChatGPT can now write its own code, although the company has been quick to stress they've taken measures to ensure anything produced by an AI "does not have unintended side-effects in the real world".

Open AI announced the launch of its new plugins today, as well as the big-name companies that are part of the first wave, including Expedia, FiscalNote, Instacart, KAYAK, Klarna, Milo, OpenTable, Shopify, Slack, Speak, Wolfram, and Zapier.

"Users have been asking for plugins since we launched ChatGPT (and many developers are experimenting with similar ideas) because they unlock a vast range of possible use cases," wrote the Open AI team on the company blog. "We're starting with a small set of users and are planning to gradually roll out larger-scale access as we learn more (for plugin developers, ChatGPT users, and after an alpha period, API users who would like to integrate plugins into their products)."

The Wolfram plugin provides computation, math, and real-time data access through Wolfram|Alpha and Wolfram Language. The Zapier plugin allows ChatGPT to interact with over 5,000 apps, including Google Sheets, Trello, Gmail, HubSpot, and Salesforce.

The plugins will allow the language model access to information that is too recent, personal, or specific to be included in the training data. In addition, Open AI says the plugins can enable the language model to perform safe, constrained actions on the user's behalf, increasing the system's usefulness overall.

Open AI's new browser plugin builds on the company's previous work with WebGPT, as well as GopherCite, BlenderBot2, LaMDA2, which allow language models to read information from the internet, expanding the amount of content they can discuss, going beyond the training corpus to new data from the present day.

"The plugin's text-based web browser is limited to making GET requests, which reduces (but does not eliminate) certain classes of safety risks," says the Open AI team. "This scopes the browsing plugin to be useful for retrieving information, but excludes 'transactional' operations such as form submission, which have more surface area for security and safety issues."

Browsing retrieves content from the web using the Bing search API which provides the benefits of Microsoft's work regarding source reliability and a "safe mode" to prevent the retrieval of inappropriate content. The plugin operates within an isolated service, which separates ChatGPT's browsing activities.

OpenAI has also created a code interpreter plugin, which provides a working Python interpreter in a sandboxed, firewalled execution environment which can handle uploads and downloads.

"The primary consideration for connecting our models to a programming language interpreter is properly sandboxing the execution so that AI-generated code does not have unintended side-effects in the real world," writes the Open AI team. "We execute code in a secured environment and use strict network controls to prevent external internet access from executed code. Additionally, we have set resource limits on each session. Disabling internet access limits the functionality of our code sandbox, but we believe it's the right initial tradeoff."