Prototyping is thinking

Motivation to write this is utterly selfish. I had these thoughts for years and decided it's finally the time to understand it. I'm using this post as a prototype to help me think about it. Here you are, dear reader, playing a forcing function so I don't slack?off. Hopefully it will also be beneficial for you in some way.? This topic probably warrants a book but who has the time for that. Instead,? I'm going to keep it shallow but cover most of the areas a book of that kind would.

Main idea is simple:?Thoughts are messy and loosely connected. When you turn thoughts into matter, they get structured. Now they are not thoughts any more, it's an object. It is the only real way to get feedback from other people. Moreover, it allows me to consider my own thoughts from the 3rd?person perspective. So even if you don't share it?with other people - it's enough for me?to perceive it as another human would. This is the feedback loop which inspires me to think even deeper.??

Let's do a brief scan of what prototypes can be, some common mistakes when?prototyping?and an outlook in regards to AI and our jobs in the future. Here we go.?

Writing is thinking

As I type these words and look at them each one is affecting what will come out next. Each sentence branches my thoughts forcing me to rethink what I actually wanted to say. A lot has been written on this topic so I won't stay here long. This blog by Paul Graham captures it succinctly:

Writing about something, even something you know well, usually shows you that you didn't know it as well as you thought. Putting ideas into words is a severe test. The first words you choose are usually wrong; you have to rewrite sentences over and over to get them exactly right. And your ideas won't just be imprecise, but incomplete too. Half the ideas that end up in an essay will be ones you thought of while you were writing it. Indeed, that's why I write them.

https://paulgraham.com/words.html

Where I find writing to be successful is if I'm trying to think deeply?about a phenomenon. To help me understand what other people will think if they read it and use that as feedback mechanism. I'm sure most of you are familiar with?Future Press Release?method when developing a product. It's a way to prototype how press will react to the release of your product or feature and to rethink what your product means to other people.

Where writing is not that useful: Reading is a conscious act. It requires participation from the reader. So, writing will help you understand what you want to make but it won't help you understand if anyone will pay attention to it.?

Drawing is thinking

Where writing fails, drawing excels. For the most parts humans are visual beings. The decision to engage with anything in the world is made subconsciously. Unlike writing which goes into?z-axis as you read, drawing is always on the surface helping you stay connected to the initial moment of interaction.?

In the book Build, Tony Fadell explains how they first designed the Next thermostat. The first thing they did was design the box and place it on the shelf in BestBuy. The whole product has been thought from that first point - what it will do, what is the installation procedure, what is the tech behind it - all of that is a reflection on how people will discover it for the first time.?

Almost everything we have interaction with in our lives started as a paper prototype. From the buildings we live in to apps we use every day. In my photo gallery I have a small collection of these prototypes that are now used by millions of users. I'm sure some of you do too.?

Talking is thinking, but now always

Somewhere in 2022, the thing I was excited the most about GPT3 was that it finally looked like it?could solve voice input. As someone who uses mobile for the best part of my day and who had worked on mobile apps for that long - it looked like a final solution for mobile editing. I was so happy about it.?Even in the early, web-only stages of ChatGPT you could hack it using the transcription on iPhone, which was not great but actually good enough that GPT could understand it. It worked.

From then to now there was a series of better models for voice that improved experience a lot, but somehow it didn't replace writing as thinking for me. It did replace scenarios where I already know exactly what to say, so it can translate conversational language to a written one, but to think about the topic and come to new conclusions - it just doesn't work for me. So, I was wondering - why?? I started paying close attention to real world interactions as I knew those are great for brainstorming. It turns out when you speak, voice feedback from another person is just a small and pretty insignificant feedback signal. Even if you are having a monologue, the feedback from the listener is what keeps you thinking. Micro expressions on their face, nodding and body language - those cues are shaping your thoughts in real time. So, I understood listener is an important in talking as the speaker, maybe even more.

To think as you talk, you need a listener, and it's hard to find people that are great listeners.? I actually tried to build this prototype. An AI Avatar with a sole purpose of being a great listener. It was quite a journey trying to learn Unreal engine, and to some success - but in the end I couldn't get the latency right. Looks like our brains work on such speeds even a small latency is difference between your thoughts flowing or actually struggling to get them out. This also made me wonder if this is the reason why video-conferencing doesn't feel like face2face yet.?

Prototyping?is thinking. In all forms.

Depending on what you make, writing, drawing or talking will not be enough. People impute qualities to products just on their impressions. If you work on a physical product you will have to think about touch as well. It is well known fact that people consider drinks in glass bottles to be of higher?quality than in plastics - even when they are exactly the same.?

Same goes for?prototyping?scent for live events or taste for food.?

When?prototyping?goes wrong

I'm sure many of you had this interaction with your friends recently:?

Friend: I'm building this travel plan app with GPT. Plan looks awesome, can you take a look?

Me: I didn't know you write those plans before traveling??

Friend: I don't.

Me: So how do you know it's useful??

When?prototyping?you are both the creator and the audience. Creating can be hard, but being able to judge is much more important. You can't prototype anything if you don't know how good feels like. Even talking with the customers, doing user research, all of that doesn't help because you will not be able to internalize the problem they are facing and you will always get conflicting signals. That's the hard, sobering thing about product thinking. Btw, it doesn't mean you have to be good at that thing, you can suck at it. It's just important you care about it. And that you do it often.?

Second danger is in endowment effect. It's a powerful evolutionary bias that makes us place higher value on things we consider our own versus the rest of the world. Obviously, this is immensely helpful to keep the family together but makes it hard to throw away a prototype once you got learnings out of it. As a rule of thumb, the more time you spend on something the harder it gets to get learnings from it objectively and the harder it gets to throw it away. So, speed is critical for?prototyping.

Golden age of thinking?

It was a long-held belief that thinking is closely coupled with language, that language is necessary for thinking. In the last couple of years this has been completely debunked in neuroscience. Here is one of the?research?on the topic.?

Furthermore, even with LLMs, language is just one of the possible token streams. We are still in early days of omni-models, models that are capable of consuming diverse token streams to reason over them and output in different modalities.? This is remarkedly human like. Humans get inspired to write a book based on a song, to paint based on a scent of a flower. Creative people try to express the same idea in many different modalities so they can understand it better.?

With transformers we are now entering a golden age of thinking. As we have seen, transformers can be useful for a whole range of applications, but hardly anything matches their capability better than?prototyping. It's easier than ever to write, draw, create a video and code. Of course, it's still very hard to get into the top 10% in any of these disciplines but something that will get you thinking - that's where transformers shine.?

Less soldiers, more tinkerers?

The obvious question comes to mind - if transformers are that good with?prototyping?can we just remove slow and unreliable humans from the process all together?? I believe the answer to be no, or at least not entirely. ?

As we have seen above humans are both the creators and the audience. The creator part gets easier and easier, even turning more into curation than creation, but the audience part requires introspection, a human reaction to the proposed solution.? We can think of?prototyping?as data creation process, but the data is created at the end of cycle when human is interacting with the prototype. Even if there is only one human, that's infinitely more than AI doing it on its own. One possible solution would be for AI to predict the reaction in advance and reinforcement learning can help to the extent. But internal human reactions to something never seen before are hard to predict based on historical data. It's far easier to predict the stock market.?

No doubt we are going to see profound impact of this technology on our society. How I think it will play out is that we will need less people to follow exact orders and more people who are able to think independently. We will put premium on people that tinker with the world, create new data that way and feed it back to the system.?