Test Driving Multi-Modal ChatGPT

ChatGPT is apparently in the process of rolling out its new multi-modal features to users with a paid account. I noticed the new icon in the interface last week but only got round to test driving it today.

TLDR: WOW! This is some crazy ￡*("*

Once again it blew my mind. So here are some examples...

My very first shot was to ask it to compose a poem based on this picture (not a great photo I know)

It came straight back (in a couple of seconds) with this:

In a cozy corner by the stove, A feline finds a place to doze. On a rug so plush and white, She curls up snug, in pure delight.

Whiskers twitch and eyes half-close, Dreaming of adventures, I suppose. The fire's warmth, the room's soft glow, Cradles her in a gentle throw.

Black as night, with patches fair, A regal presence fills the air. In her kingdom by the flame, She rests and dreams, wild and tame.

By the hearth, she finds her spot, A cherished, warm and loving thought. In the dance of firelight's beam, The cat drifts off, to softly dream.

Cute! The references to the plush white gentle throw cradling her are good. It correctly identified her as a female cat - and a regal one at that. The "eyes half closed" and "patches fair". All quite impressive.

OK.... so lets crank it up a bit...

Prompt: This is one of my kitchen cupboards. Can you write me a shopping list to replenish this cupboard for me.

Response:

Certainly! Based on the contents of your cupboard, here's a suggested shopping list to replenish it:

Peanut Butter (crunchy)

Yeast Extract

Barley Malt Extract (Meridian brand)

Olive Oil or cooking oil

Canned Green Peas (or similar vegetable)

Canned Chickpeas

Canned Cannellini Beans

Canned Black Beans

Canned Pinto Beans

Canned Chopped Tomatoes

Canned Whole Tomatoes

Canned Bean Mix

Canned Sweetcorn

Canned Fish (Tuna or Salmon, or similar)

Dried Herbs or Spices (based on the bottles seen)

Other essentials or staple items you frequently use.

Interestingly it doesn't seem to know what to make of the Golden Syrup - I would have thought Lyle's was a fairly well known brand. Lets push it on this.

OK... now for a different image...

Prompt: What does this look like to you?

It looks like a pot of diced vegetables, possibly tomatoes and other ingredients, being cooked on an induction stovetop. Are you preparing a particular dish?

It's actually stewed apples and strawberries - but I don't think I could have told that from the image alone. What blew me away though was the identification of the induction hob. But then I looked closely and I could see that it does say in the corner "induction". Was this what tipped it off?

OK... lets get a bit surreal (or as surreal as I could manage in my kitchen at 11pm)...

Prompt: What is wrong with this picture?

From the image, it looks like there's a pair of glasses inside a mug, which is an unusual place to keep them. Typically, mugs are used for beverages, not for storing objects like glasses. The placement could potentially damage the glasses or make them dirty. Additionally, the risk of accidentally pouring a drink over them increases if one forgets they're inside the mug.

I couldn't help wondering about the "risk of accidentally pouring a drink over them"... slightly disappointed by the answer...

What about counting and identification?

Prompt: How many map pins are there in this picture?

Finally I wanted to test its OCR capabilities. Once again these were staggering:

Prompt: Can you summarise what this is saying?

In under a couple of seconds it replied with...

Certainly!

The text discusses the concepts of "safe-life" and "fail-safe" in the context of aircraft design:

1. Safe-life: Refers to the life span of an aircraft during which the probability of failure due to fatigue is acceptably small. If an aircraft's safe-life doesn't match the service life based on fatigue inspections, it would be too costly. An economically viable aircraft should last its intended lifetime with minimal modifications and repairs, primarily because it has a sufficient safe-life. This ensures that the aircraft can handle dangerous situations arising from failures.

2. Fail-safe: Ensures that when a failure does occur, it doesn't lead to catastrophic consequences. If an aircraft has a low percentage of fatigue-sensitive structures, the weight penalty of using safe-life stress levels in the design would be economically feasible. By incorporating fail-safe design features, the risk of cracks initiating and leading to fatigue damage is minimized. This type of design aims for longer service life with minimal inspections. However, inspections will always be necessary due to risks like corrosion, unexpected damage from ground handling, and other sources, which might not always be noticeable.

3. Conclusions: The text highlights a specific finding related to aluminium structures. To maintain safety standards, for every standard deviation in the logarithm of the number of cycles to failure (Log10N), aluminium structures should have a mean fatigue life that is 0.2 times their required service life. This is to ensure the risk of a catastrophic fatigue failure remains very low, specifically between 10^-6 and 10^-7.

In essence, the text emphasizes the importance of designing aircraft with adequate safe-life and incorporating fail-safe features to ensure longevity and safety.

From my first interactions with it I'm very impressed and I'm sure I'll come up with some more fun and creative uses for it over the coming weeks.