ChatGPT or Gemini: Which is Better? - A Quick Comparison

Author - Aditya Rawat, MBA, Coventry University, UK (Ex. Deloitte, MOODY's Research, and HCL-British Telecom)

Note: The views shared in this blog are based solely on my personal experiences and research. Others may have different perspectives and findings.

The rise of artificial intelligence (AI) has led to the emergence of chatbots, which have become essential tools for businesses and individuals alike. Two prominent players in this space are - ChatGPT and Gemini (formerly Bard), both offering advanced AI-driven chatbot capabilities.

OpenAI introduced ChatGPT in November 2022, generating significant interest in generative AI. This led to GenAI becoming a dominant theme in the tech world in 2023. Microsoft backed OpenAI in early 2023 with a multimillion-dollar investment to accelerate its AI technology development. Google entered the GenAI space in March 2023 with Bard, which was later rebranded as Gemini in February 2024, featuring an improved AI chatbot.

ChatGPT and Gemini have fueled considerable buzz around GenAI, which uses machine-learning models to generate text, images, and videos. Both OpenAI and Google continue to enhance their large language models (LLMs) to improve their ability to generate human-like text. Despite AI hallucinations, incorrect answers generated by AI, ChatGPT, and Gemini are being adopted by businesses and consumers for automating tasks.

What can you do with ChatGPT and Gemini?

• Content Generation: Both platforms can help generate content like emails, social media posts, and blogs. They can provide suggestions, write drafts, or even generate complete pieces based on given prompts.
• Text Summarization: Both can summarize long texts into shorter, more digestible versions, which is useful for condensing information or creating abstracts.
• Language Translation: Both platforms can translate text from one language to another, helping users overcome language barriers and reach a wider audience.
• Code Generation: Both can assist in generating code snippets for programming tasks, which can be particularly helpful for developers seeking quick solutions or learning resources.
• Learning and Education: These platforms can serve as educational tools, providing explanations, answering questions, and offering interactive learning experiences.
• Building Virtual Assistants: Both platforms can be used to develop virtual assistants that can interact with users, answer questions, perform tasks, and provide guidance.
• Research Assistance: Both platforms can aid in research by providing relevant information, summarizing research papers, and assisting in literature reviews.
• Entertainment Applications (Games): Both can be used to create interactive and engaging games or entertainment experiences, leveraging their natural language processing capabilities.

Key Features of ChatGPT

• Strong Conversational Abilities: ChatGPT can carry on natural-sounding conversations, adapting its tone and style to fit the context.
• Text Generation: It excels at generating various creative text formats, such as poems, scripts, emails, and more.
• Diverse Applications: ChatGPT can translate languages, summarize information, write creative content, and answer questions informatively.
• Adaptability and Personalization: It learns and adapts over time, improving its understanding of user preferences and communication style.
• API Access: ChatGPT's open-source API allows integration into various applications and projects, making it versatile and accessible to developers and end-users alike.

Key Features of Gemini

• Multimodal Understanding: Gemini processes information from various sources, including text, code, and visuals, for tasks like summarization, translation, and analyzing codebases.
• Up-to-date Information: Gemini continuously crawls webpages, providing access to the latest updates and breaking news.
• Human-like Explanations: Gemini explains answers clearly and concisely, providing relevant and insightful results.
• Comprehension and Improvement: Gemini can analyze codebases, identify potential bugs, suggest solutions, and explain their reasoning, improving code quality and saving developers time.
• Focus on Responsible Development: Google emphasizes responsible development and deployment of Gemini for positive impact.

Which Chatbot is Better?

To assess the effectiveness of AI models, researchers utilize benchmarks like the Massive Multitask Language Understanding (MMLU) test. This test evaluates an AI's capacity to comprehend and resolve issues across various language-related tasks.

According to Google, Gemini AI outperformed ChatGPT in nearly all of its academic benchmarks. Here's a comparison between Gemini Ultra and ChatGPT-4 using the benchmarks tested by Google:

1. General Understanding: Gemini Ultra achieved an impressive 90.0% in Massive Multitask Language Understanding (MMLU), showcasing its ability to understand 57 subjects, including STEM, humanities, and more. ChatGPT-4 achieved only an 86.4% 5-shot capability in a similar benchmark.
2. Reasoning Abilities: Gemini Ultra scored 83.6% in the Big-Bench Hard benchmark, demonstrating proficiency in a wide range of multi-step reasoning tasks. ChatGPT-4 exhibited similar performance with an 83.1% 3-shot capability in a similar context.
3. Reading Comprehension: Gemini Ultra achieved an 82.4 F1 Score in the DROP reading comprehension benchmark. ChatGPT-4V attained a slightly less impressive score with an 80.9 3-shot capability in a similar scenario.
4. Commonsense Reasoning: Gemini Ultra scored with an 87.8% 10-shot capability in the HellaSwag benchmark. ChatGPT-4 demonstrated a slightly higher 95.3% 10-shot capability in the same benchmark.
5. Mathematical Proficiency: Gemini Ultra excelled in basic arithmetic manipulations with a 94.4% math score. ChatGPT-4 maintained a 92.0% 5-shot capability in Grade School math problems.
6. Math Problems: Gemini Ultra could tackle complex math problems with a 53.2% 4-shot capability. ChatGPT-4 scored slightly lower, with a 52.9% 4-shot capability in a similar context.
7. Code Generation: Gemini Ultra could generate Python code with a commendable 74.4% capability. ChatGPT-4 did not perform as well, scoring a 67.0% capability in a similar benchmark.
8. Natural Language to Code (Natural2Code): Gemini Ultra showed proficiency in generating Python code from text with a 74.9% 0-shot capability. ChatGPT-4 also scored well, maintaining a 73.9% 0-shot capability in a similar benchmark.

Independent Reviews

Several reviews offer valuable insights:

• Carnegie Mellon University researchers compared ChatGPT and Gemini, finding Gemini stronger in complex reasoning and non-English translations but weaker in math and with aggressive content filtering.
• Wharton professor Ethan Mollick concluded Gemini Advanced can compete with GPT-4, each having strengths and weaknesses.
• Futurist Bernard Marr highlights ChatGPT's conversational nature and Gemini's efficiency in processing information and automating tasks.

Final words

ChatGPT and Gemini are powerful AI chatbots that offer unique features and capabilities. The choice between the two ultimately depends on your requirements and the tasks you need the chatbot to perform. Whether you're looking for a conversational interface or multimedia capabilities, ChatGPT and Gemini have something to offer in AI-driven chatbots.