AI Versus BI, Part Two
Time for the second part of my AI versus BI duel between ChatGPT and Copilot. This one really focuses on Copilot and could be called, “What were you thinking?”
Here’s a quick recap of this interesting side problem. I loaded the Billboard Hot 100 data into both ChatGPT and Copilot via Excel file, then asked which performers had the most #1 hits. ChatGPT claimed Michael Jackson had seven chart toppers, while Copilot said the King of Pop scored the top position nine times.
If I were brand new to SQL queries I’d say that ChatGPT was correct. Seven distinct songs reach #1 with a performer of, “Michael Jackson.”
However, I’m not new to SQL, nor 80s music, so I know to check for collaborations. Micheal made it to the top with Paul McCartney with “Say Say Say,” and again with Siedah Garrett with “I Just Can’t Stop Loving You.” It appears Copilot was right, right?
Maybe. Let’s check.
“Based on only the dataset in the Excel document I provided, what are the songs that hit #1 with Michael Jackson as a performer?”
“Where does ‘Black or White’ appear in the dataset? I don't see it.”
You are correct. Upon reviewing the dataset from the Billboard Hot 100 - 80s.xlsx file, "Black or White" does not appear in the dataset. This song was released in 1991, which is outside the scope of the 1980s data in the file.
I thought I’d been specific enough by saying, “Based on only the dataset…” but perhaps not. Let’s try asking a different way.
“When you calculated that Michael Jackson had nine songs which reached #1 in the dataset I provided, which songs did you include?”
领英推荐
Based on the dataset from the Billboard Hot 100 - 80s.xlsx file, the songs by Michael Jackson that reached the #1 position are:
These are the eight songs by Michael Jackson that reached the #1 position on the Billboard Hot 100 chart during the 1980s. I apologize for the earlier mistake in stating that there were nine songs. If you have any more questions or need further analysis, feel free to ask!
Apology accepted, but we still have a problem.
“What about ‘I Just Can't Stop Loving You?’ According to the data file, is that a Micheal Jackson song that peaked at #1?”
Yes, according to the data in the Billboard Hot 100 - 80s.xlsx file, "I Just Can't Stop Loving You" by Michael Jackson did peak at the #1 position. This song should be included in the list of Michael Jackson's songs that reached the #1 position during the 1980s.
“So, how many Michael Jackson songs reached #1, and what are they?“
Based on the dataset from the Billboard Hot 100 - 80s.xlsx file, Michael Jackson had nine songs that reached the #1 position on the Billboard Hot 100 chart during the 1980s. These songs are:
This time Copilot listed the correct songs.
Conclusion: Unfortunately, Copilot’s original correct answer of nine songs was NOT more insightful analysis nor added value. It seems to have been a lucky guess. Or even worse, a fortunate mistake.
I’m curious if Copilot learned anything from this progression. The experience felt very much like teaching a clever child rather than referencing a trusted resource. If the system did improve through the interaction, that’s great, but I’d still caution against having the AI build your time machine or analyze your P&L.
Demystifying Your Passion & Purpose So You Can Live It Every Day | Best Selling Author, The 25 Mindsets: Understand Anyone, Even Yourself
3 个月I primarily use Copilot and have found it "lying" to me on several occasions. For example, it flat out made up a song and lyrics by Tyler Childers that didn't exist (in my search to see if he, a Progressor mindset, had lyrics about thinking for yourself). Thanks for sharing your experiments and insights!
Experienced Leader in Customer Success, Service Delivery, Enterprise Support and Operations
3 个月Good article Jeff, entertaining and insightful