Bing vs ChatGPT: A Tale of 2 Chatbots

Last week, OpenAI left us in awe with the release of the groundbreaking GPT4. Microsoft has also captured the attention of businesses and individuals alike with its Bing Chatbot, integrated seamlessly with Office365 products and Copilot. Not to be outdone, the search giant of the past two decades, Google, unveiled its own Chatbot, Bard, yesterday (March 21, 2023). With this, the AI Chatbot wars have officially begun, and who knows what other players will enter the field and shake things up?

In this article, I will evaluate the performance of ChatGPT and Bing chat by exploring their current strengths and optimal use cases and analyzing their responses to use-case-based prompts.

Bing vs ChatGPT

Here is a quick comparison between Bing and ChatGPT at the moment.

Which one is better?

Deciding which chat to use would solely depend on what your use case or need is.

Let's take a few examples:

General Knowledge

Let's take a prompt

Tell me about the first moon landing

Bing gives us a short concise answer with links to sources of information. Bing chat is rather limited when it comes to the length of the results it can return. It does, however, provide helpful follow-up questions below the response.

That said, when I asked it to give me the response in 2000 words, it shut me down and terminated the current chat! I understand the intent behind this - but it shows that giving your chatbot agency isn't necessarily a good thing. This can be especially irritating as I may have been guiding the chat to respond to me in a particular way in the chats before. Bing chat already restricts you to 15 chats in one thread. Allowing it to terminate a thread like this can be very frustrating to a user.

ChatGPT on the other hand gave me a long well-written piece. It had no qualms about providing me the data in the format I need - and as long as I wanted it to be. However, I had to specify explicitly that I wanted it to provide me with its sources of information (it does not provide these by default).

IMO, in terms of verifiability of information - I would give the edge to Bing, although in terms of quality of output, I prefer ChatGPT (As long as what you are looking for is before Sep 2021)

Winner: Bing for short responses and more recent information, ChatGPT for longer responses

Real-time information

There is no competition here. Connected to the internet, Bing can provide real-time information. However, it often does not give you results you are expecting. It returns links to where it believes one would find information and expects you to do the work. Moreover, the chat terminates involuntarily if asked for more information. Sometimes Bing shuts down the chat stating there is nothing more to add and that we should start a new chat.

Accuracy also appears to be a problem for Bing at the moment. For example when I asked it to provide a profile about me - here's what it gave. It got my education wrong although it appears to have pulled in the information for the profile from my Linkedin account.

Winner: Bing hands down

Creative writing

Write a short story about a magical forest in India

Bing's response in creative mode

ChatGPT's response

Both provided short stories. However, I personally preferred the ChatGPT version. It had an introduction, middle and suitable conclusion to the story whereas Bing's response felt a little incomplete (possibly due to the character limit for its response). Overall, I believe that ChatGPT is far superior to Bing's offering here.

That said, for those of you looking to use the output of these directly, both get caught out by AI content detectors - although interestingly, the parts where they talk about the main character get flagged as human content.

Winner: ChatGPT

Summarizing and critiquing

I had written an article on how I used ChatGPT to critique my previously written work. When I put Bing through the same grind. Here are the results

Overall, I feel ChatGPT was better again although Bing also did a very good job of critiquing my articles. Both can help highlight the problematic areas and help rewrite content. Bing unfortunately is hindered by its character limit in the responses. ChatGPT on the other hand can be as detailed as possible. In fact, as you keep interacting with it on the chat it gets more and more helpful whereas Bing can tap out after a few responses (or 15) or flat out refuse to do more work depending on its "mood".

Winner: ChatGPT

Analysing Data

I love watching and playing cricket. IMO, it is the greatest sport in the world - a good balance between strategy and action - a thinking man's game. However, as in any other sport - there are always debates on the best players in the game which would never have any sort of resolution. I want to use data to attempt to present my point of view with a metric called "Impact Score". I believe that Virat Kohli is the best or most impactful batsman over the past two decades (2000-2020). Let's see how the chatbots can help here.

First, I want to come up with an impact score formula for batsmen. So I prompt the chatbots with the following:

I want to create an impact score formula for batsmen in the sport of cricket. Can you help me create one for all 3 formats of the game?

Right off the bat, I appreciate the fact that ChatGPT recognized that there were 3 different formats of the game. Although I had to massage both with the parameters I required for the impact score, ChatGPT was able to come up with a more nuanced index which would possibly give me a more nuanced response. However, Bing was able to accept the formulas given by ChatGPT as a starting point to calculate my score. Unlike Bing which flat-out rejects any further help in analyzing, ChatGPT would use its training data to calculate this using this index as well. However, I believe Open AI is starting to reject such complex computations now due to server overloading. That said ChatGPT did give me a way to do this offline whereas Bing asked me to find a relevant data source and provide it information.

That said, I believe the winner, in this case, would clearly be ChatGPT, as I can feed in as much information to ChatGPT for analysis whereas Bing would only be able to take in a certain amount of data. It would be interesting to see how Copilot tackles the analysis of data in Excel. Bing Chat, however, is clearly not made for this purpose.

Winner: ChatGPT

Summary

There are plenty of other uses that we can compare the two on - such as root cause analysis, coding assistance, creating new content based on existing content, etc. but it seems clear where each shines at the moment.

The Bing chatbot shines for searching for information on specific information. It is generally more accurate in its responses and provides links to its data sources although the data it presents is incorrect (at times). Its biggest strength is its connection to the internet and ability to respond to queries from any time frame without additional training

ChatGPT, IMO, is far superior for tasks, in terms of the quality of output produced, for creative tasks, and anything related to the analysis of information from its training data (Sep 2021). It can be trained on more recent data. However, this can be a lengthy and painful process. So it is not very useful for content beyond the training date of its model at the moment.

Let me know what you think about this content. As always, feedback is welcome!