o1 vs. 4o, is Strawberry really delicious?
Announced to the world on 12th September 2024, the new o1-preview ChatGPT has been the current trend in media. Most of people who are learning or working with AI would be aware of this new release and might already tried.
I tried myself too, of course. And as a evidence oriented person, I have done some tests to compare between 2 models: 4o and o1. My objective is just to see why the new o1 is receiving positive feedback from many people.
OK, here we go. For the quick test purpose, I will not make complicated prompts or too complex requests (actually they are quite simple).
Here are the result (Disclaimer: again, this is my result limited to my knowledge on how to use ChatGPT so far, and not to be a reference for any serious studies)
TEST 1 - Review "Sapiens: A brief history of Mankind” by Yuvaral Noah Harari
Request: “summary for me the key content of "Sapiens: A brief history of Mankind" of Yuval Noah Harari, and reviews from famous business leaders in the world”
With 4o assistant
The result is really brief, breaking down the book content into 5 sections with short summary for each, which does not show the major difference from this book and other Mankind history books. It also summarizes reviews from Bill Gates, Mark Zuckerberge, Barrack Obama.
Satisfaction score: 5/10. For a person who hasn’t read this book yet, I believe this summary just indicates that it a normal boring history book.
With o1 Strawberry
The content structure of the answer is the same, but the information is richer, pointing out the interesting ideas within the book. Example:
“Harari also delves into the concept of imagined realities—shared myths such as nations, corporations, and legal systems—that exist because of collective belief. These constructs have immense power in shaping societies and driving collective action. The book concludes by pondering the future of humanity, especially in light of biotechnological advancements and artificial intelligence, questioning what it means to be human in an age where Homo sapiens might evolve into a different kind of being.”
This model also lists reviews from Bill Gates, Mark Zuckerberge, Barrack Obama, but again, it’s more informative and more inspirational for people who is considering reading this book.
Satisfaction score: 8/10.
TEST 2 - Sales Introduction Email suggestion
Request: “you are a sales manager with 10 year experience working in software development industry, specialized in ecommerce platforms. Our company is Secomm Solutions Consulting who is positioned as Ecommerce Solutions Provider. I want to approach a new potential client called XYZ Fashions who already have multiple stores in Vietnam. Help me to write an email to the Ecommerce Manager of that brand.”
I will not show the exact result here as they are too long, but my thoughts on those 2 assistants are below:
With 4o assistant
Casual style, straight into the main purposes of introduction and Call to Action.
Satisfactory score: 5/10, which means you will not be able to use this as a base.
However, the plus point is that somehow it knows we Secomm use Magento and Shopify Plus to provide solutions to our customers, while the Strawberry does now know that at all.
领英推荐
With o1 Strawberry
Formal style, clear introduction with helpful information about Secomm services, some insights from fashion industry in Ecommerce trend and how the others in this industry is growing with Ecommerce (although I am still questioning whether those figures are accurate or not, probably the model invented by itself.)
This new model also express the values that Secomm would bring to the customer business and then Call to Action.
Satisfory score: 7/10. This result is absolutely usable as a base to come up with a good Sales Email.
TEST 3 - Software Solution Design
Request: “You are a software solution architect with 10 year experience. Help me to design an online application for a High-end Wine bidding broker business which allows 3 user types to interact with each other: Wine owners, Bidders, and Brokers. The Wine for bidding will need to be verified the authority, origin, quality, etc... and the bidding process will need to be secured as well as the payment process.”
Again, the answers are very long, I will just include summaries and my thoughts on the result.
With 4o assistant
Be able to work out the basic required modules, suggested tech stack, potential integrations.
And that’s it. This is useful for a student though.
Satisfaction score: 5/10.
With o1 Strawberry
Now it becomes more interesting. o1 is known by its ability of thinking, working out solution with steps on its own.
Strawberry’s solution have more required sections of a software system design document, which describes many different aspects of an online application. For example:
Impressive answer, to be honest. And this is really a sweet Strawberry that I expect to see. Although the information it brings do not show anything I don’t know as I am a solution architect, but it does show a great potentials of even deeper thinking for more difficult problems, which I will probably try on our real projects in the near future.
Satisfaction score: 9/10
CONCLUSION
Via 3 tests from a very simple one to a bit complex request, o1 Strawberry does show potential and its capability to be helpful assistant for many different purposes and use cases.
I hope this article will give you some more insight about AI and ChatGPT.
Your future however is still in your hand, not in AI’s hand. Make yourself an expert at your job plus AI assistant will bring you a irreplaceable postion with great values.