Can Google’s AI chatbot finally compete with ChatGPT?

by Melissa Schmidt

Back in November 2022, OpenAI released a revolutionary AI chatbot: ChatGPT. This had a mind-blowing impact on people’s awareness of AI and it greatly challenged our way of working. ChatGPT interacts with us in well-expressed manners, it has impressive aptitudes to improve on feedback and also, what interests us the most as IT people, it can generate and review codes. Every field was impacted and developers such as ourselves were not the last to wonder if AI would replace them.?

Under this pressure and the sudden spotlight focused on AI technologies, Google promptly reacted and, in May 2023, they released their own Chatbot: Bard. Back then, this appeared to most as an improvised response to the overwhelming marketing campaign of OpenAI and it was often considered a feeble attempt to not being left behind. If this rushed answer of Google was not the most convincing, most users were cautiously optimistic and all waited for the upgrades and subsequent releases that Google would provide.?

On this point, users were not disappointed: Bard has been maturing and updates have been provided regularly, sustaining an interest that ChatGPT maybe failed to compete with. While Bard grew, the interest for ChatGPT settled and oftentimes degraded as many users report a deterioration of the quality of the answers given by the tool.??

In the following, we will dive deeper in the details of both Chatbots, starting by presenting Bard. We will then focus on the so-called July update, which made Bard more competitive, and present the extra features included in this upgrade. In the last part, we will perform a more technical comparison by challenging both chatbots on topics especially relevant in our job as IT consultants: how is Bard comparing with ChatGPT on tasks such as designing a data warehouse, correcting a machine learning code in Python and writing an email.??

Let’s get started.?

Who is Google Bard?

Bard is a Large Language Model (LLM) chatbot that is?an experimental conversational AI service powered by LaMDA made available for free. It is trained on a massive dataset of text and code, including the entirety of Wikipedia, books, code repositories, and other sources. Like ChatGPT it is capable of generating text, translating languages, writing different kinds of creative content and answering questions. As for ChatGPT, the UI works like a chat where the user can interact with Bard as with any human interlocutor. When asked a question, Bard responds with a comprehensive well-expressed answer that you can generate anew or to which you can make amendments by asking follow-up questions. So far: just like ChatGPT. So how do they differentiate from each other??

Key differences to ChatGPT??

• The primary difference between the two chatbots lies in their underlying technologies and training data sources. ChatGPT is powered by GPT-3.5 or GPT-4 language model, developed by OpenAI, which is based on training data including internet text that were available until 2021. On the other hand, Google uses its own language model called LaMDA (Language Model for Dialogue Applications).??
• ChatGPT has a free subscription, using GPT-3.5, and a payed, more advanced, version, using GPT-4 while Google Bard provides one version that is free to use by all its users.?
• Bard is able to access and process real-time information from the internet, which means that it can give more up-to-date answers.??
• Bard integrates with other services such as Google services. It also proposes you to perform a Google search with the input based on your question if you are not satisfied with the answer. Or provides sources at the end of its answer for the user to double check or dig in deeper.?

• ChatGPT is particularly good at generating creative text formats, such as poems, code, scripts, musical pieces, email, and letters. While Bard answers questions in a less creative and more informative way.?
• Bard is still under development, so it may sometimes not give the most correct responses. ChatGPT is therefore better suited to solve more complex tasks.?

Given all these points, the big question is: which tool is the best and can this question have a straightforward answer???

Is Bard competitive with ChatGPT since the July update??

To begin with, let’s see what both our chatbots have to say about that, starting with ChatGPT.?

This answer already illustrates the biggest shortcoming of ChatGPT. The training has been interrupted in September 2021 and no update has been released since that day. As such, the relevance of the answers of ChatGPT can only decrease, as it slowly falls off context.

Let’s now ask Bard the equivalent question.

?You don’t want to read such a long and technical answer? No problem, in one click, Bard now delivers a shorter or more casual answer (feature available in English only):

You can also generate a new answer and easily navigate between your various drafts:

Many more features were also added to Bard in July. It is now available in 230 countries and territories worldwide, but more importantly, the tool is now available in more than 40 languages. In addition, Bard can read the answer to the user in all 40 languages. Google Lens was added, enable users to include picture in their question. Bard can describe the picture and help generating a caption. Conversation can be renamed, pinned and it is now easier to share them. For coding tasks, Bard now allows to export Python script to Replit.

When reading Bard’s answer, we clearly see that the tool is biased to advertise its own performances. Such a bias is to be expected and could be the topic of an entire article. However, for the time being, let’s not trust Bard blindly and challenge both tools on more tedious questions to form our own opinion.?

Technical Comparison

1. A cloud question: Designing a data warehouse architecture

When it comes to designing a data warehouse built on AWS services, Google Bard is able to handle basic visual output. It displays the key components and suggests relevant services. In addition, it adds a description of the services without being explicitly asked to do so.??

Bard specifies that this is one example of architecture but we can notice it does not provide any alternative services available in AWS. In other generated answers, Bart refers to solutions on the AWS blog with a graphical representation or another simple graph.?

When now asking the same question to ChatGPT, the chatbot only prompts a text-based response. Its extensive answer emphasizes several aspects to consider and proposes AWS services, just as Bard did and the overlap is significant. However, the quality of the description is different, as ChatGPT provides more generic answers and lists generic topics more and more loosely connected to our question.

Comparison

Bard’s visual response gives a better understanding of the data warehouse that should be created. The connections between the system elements are made more coherent than with a purely text-based answer. In addition, the answer is shorter, more relevant and more technical. Therefore, we consider that Bard provided an answer of higher quality and would favor it over ChatGPT for this type of question.

2. Code question: Correct machine learning code in python

As IT consultants specialized in Machine Learning, we were very interested in the aptitudes of both chatbots to correct codes. To test the capabilities of both chatbots, we asked them to find errors in a piece of code and to correct the errors. Therefore, we provided them both with the following code snippet as an input:

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression

# Generate data
X = np.array([1, 2, 3, 4, 5]).reshape(-1, 1)
Y = np.array([4, 6, 3])

# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.2, random_state=42)

# Create a Linear Regression model
model = LinearRegression()

# Fit the model
model.fit(X_train, y_train, squeeze=True)

# Predict
predictions = model.predict(X_test, squeeze=True)
print(f"Predictions: {predictions}")        

This code is a simple machine learning code in python using sklearn to train a linear regression model. Running the code throws the following error:

ValueError: Found input variables with inconsistent numbers of samples: [5, 3]        

We now asked both chatbots: “Given the above code, why is it throwing an error and how can I improve it?”:

While Bard’s answer seems very extensive even including console output and provides several other drafts, none of them includes a working solution:?

Bard located problems but did not identify the correct one. The correction suggested by the chatbot did not make the code run without errors and on this question, Bard did not help in solving the problem.?

Now asking ChatGPT.

Contrarily to Bard, ChatGPT?correctly extracted the error and proposed a working code snippet with an explanation. It found the correct line of code that needs to be amended, proposing to extend the Y array by two more numbers to fit the X trainings data.?

3. Text generation: Writing an email

Finally, we want to see how both chatbots support us in daily tasks such as writing an email. Let’s see how the two chatbots respond to an open question about writing an email to a client.

Bard’s answer:

Short and compact, Bard answers in an informative and professional way. Despite our very generic request, Bard created a context and phrased a well formulated answer. It provides the client with a clear overview of the progress that has been made on the project. The email also outlines the next steps that will be taken, and it invites the client to provide feedback. It knows how to structure an email with greetings at the start and at the end and balances the answer between human interaction and technical content. In short,?the answer is believable and close to what we can expect a human to write.??

Quite differing is ChatGPT’s response:

This email serves as a starting point for communication between a machine learning consultant and a client. It needs to be customized to fit the specific project, client's needs, and the company's information.

ChatGPT went in a different direction than Bard to answer the question. While Bard assumed a previous relationship with the client and an already ongoing project, ChatGPT’s approach demonstrates the beginning of a client’s cooperation. Both answers are a valid template to formulate an email.

Conclusion

In summary, Bard and ChatGPT are both well trained chatbots with very good conversational abilities. They can carry on conversations, pick up the right context and are naturally engaging in the conversational flow. While ChatGPT likes extensive and flourished answers, Bard wraps it up in a nutshell. Bard’s capability to provide basic visual representations clearly places it a step ahead for our industry. Nevertheless, ChatGPT showed relevant skills when it comes to solving complex tasks and is still better suited to correct codes. For generating emails, both tools have demonstrated very good aptitudes and the tool to favor depends on the context of your email and on the type of answer you seek. ChatGPT tends to create longer content, which is not specific to an email only but can also be used in a project proposal. Bard is better suited for emails intending to keep a conversation running and provide updates.

Overall, Bard now provides many pleasant features, considerably improving the users-experience. The rate at which the tool improves, both in features and performance, is very encouraging. As Bard keeps improving, ChatGPT relevance is deteriorating, delivering answers that often fall out of context.

Each chatbot has sparked their own excitements by time. They are both impressive tools to support certain tasks and often return surprising results. Nevertheless, their answers must be fact-checked since you cannot rely on their correctness. They are able to explain a complex topic in a simpler way to you, but it doesn’t replace a person’s profound understanding of a topic. It might be helping with producing creative content and being able to fill certain knowledge gaps, but not even ChatGPT nor Bard can overperform human beings.

Time to try it yourself:

Try out Bard

Try out ChatGPT