ChatGPT: Part 1, limitations

Having worked in software for 20+ years, I see very few things that strike me as being 'new' and 'different'. Most innovation is an evolution of what is before. ChatGTP is not this - it is really new and has a lot of implications for the future of both software development and the end user applications that are produced.

My interest is (and has been for decades) how do we decrease the cost of automation and make systems 'smarter'. It seems like ChatGPT is an interesting new tool for this - however, in current form there are limitations.

For anyone not familiar with ChatGPT, here is a very non technical (and largely inaccurate) explanation. ChatGPT is a chat robot that allows you to have a conversation with an 'intelligent' computer. You can ask it to produce some artifact or answer a question - and it is uncanny in its ability to produce something that resembles a human response.

ChatGPT can:

• produce the content of business letters
• write essays
• write poems and songs
• write code
• analyze code
• etc, etc, etc

It produces things that look really great - but are not always that accurate or consistent. It may produce different (and sometimes incorrect results) to the same question. Given this, the consumer of these conversations needs to be able to understand and guide the bot to get a usable result.

As a tool for a senior person with the ability of to understand and correct the results, it is a huge time saver. I have been using it to help me write code, and often it takes 3-5 cycles to get what I am after. I have to be able to understand what it is producing and ask it to correct. If I was a less technical person, I might take the result and just use the first one. The limitation here probably is a result of my not explaining what I wanted correctly on the first go around.

Business software is really complex, often involving multiple architectural layers, business rules, data structures (which by their nature also infer business rules) user interfaces, system interfaces, etc. When asking for some input, trying to explain the entire landscape and exactly what is needed is complex - maybe even more complex than the benefit that is gained. In current form, ChatGPT gives you about 4000 tokens (think of a token as a word or part of a word) for both the request and response. So, even if you wanted to provide sufficient data there is a limitation of how much can be given to the system.

Finally, its slow. Not slow like it is irritating to wait for with spinner, but too slow to be used in the middle of high volume transactions. I cannot imagine asking it for a scoring of 10,000 items. At 5-60 seconds per response it would take forever.

I am excited about ChatGPT! I think, however, the trick is to find use cases that it can be useful without doing harm. I am assuming as the models improve and we (humans) learn to phrase requests with more specificity that some of the limitations will be reduced.

In future posts, I am going to explore use cases that:

• produce artifacts that are easy to understand and consume
• do not require a deep understanding of the entire system architecture
• are not limited by the relatively slow transaction speed.