It still takes effort to make a really good bot

Come on then conversation designers – who’s bored by the lack of challenges in our industry?

What, nobody?

It’s no surprise really. We conversation designers have to be comfy with change. It seems like things are in constant flux, and that leads to new challenges. At the time of Siri’s release (2011) there were only a few hundred people in the world designing conversations with things. Then many conversation designers were newly minted during the Alexa/Assistant era (2014 – 2022), but best practices were still hard to find. Then along came ChatGPT in late 2022, and the widespread focus on LLMs, and for a tiny moment it seemed like we conversation automators could be made redundant by software.

We designers have to prove our value all the time. People were asking, if machines can have natural language conversations with people that weren’t designed, then who needs a conversation designer?

As if! That’s just one of the many subjects that were expertly tackled by Rebecca Evanhoe and Wally Brill when they answered a dozen or so crowdsourced questions with warm and thoughtful advice.

Do you want a mediocre result?

Conversation designers are more relevant than ever. Thanks to LLMs, there’s now millions of digital conversations happening every minute, and they don’t always go well. While an LLM can theoretically connect anyone to any piece of information, how that is done has wildly variable results. A conversation with an LLM can be helpful, or like talking with someone who thinks they know it all, a manipulative social chameleon or someone who’s a bit high.

Sadly, the myth that conversation designers were made irrelevant by LLMs still lives on. As Rebecca said, “I have a little anecdote. I gave a talk about LLMs. A nice guy came up to me, and he was a developer and said, ‘Oh, is conversation design even a thing anymore?’ And I said, ‘Yeah,’ and I said some stuff [about conversation design]. And the guy said, ‘Well, I can do that. I can have an LLM do all this stuff.’ So then I was like, ‘But is your bot good?’ And he was like, ‘I don’t know.’ It’s easy to make a mediocre bot, but it’s still a high effort thing to make a really good bot.”

Conversation designers are full-time educators

And designers need to be strong and persistently remind people what’s important for a great user experience. According to Wally, “people are going to want to do the minimum code possible, right? So they’re going to ask you to reuse things over and over and over again. Like, ‘let’s have a generalised error prompt that goes everywhere. Why should we rewire it?’ … And you know, it’s really about holding your ground as a designer and saying that shortcuts aren’t okay, because they will affect the user experience. Over time, there will be calls lost, there’ll be money lost, there’ll be transfers to agents that you don’t want.”

For Rebecca, educating people about conversation design is a huge part of her job. She said, “you have to teach people. I think that’s something that can burn designers out. We’re not just doing our job. We’re constantly teaching. As you go forward in your career, you kind of develop your little toolkit of examples and activities. You see people get the lightbulb moment. If you give this example, people go ‘oh, I see.’ I did a workshop that had some higher level folks, and I had them do prompt writing and a little training, data sorting, and then tried to get those things together. It was a very simple activity. But one of the PMs that I had been working with afterwards said, ‘I get it now. I didn’t get it before, but I get it now.’ Even if you can just get people to sit here with me, just spend an hour with me and play around, then that can really help. And it’s part of the job.”

We’ll find you an answer

The work of the conversation designer is easy to overlook. Frequently, conversational AI is seen as a ‘tech’ challenge, but AI and humans aren’t easy conversational partners. Designers are needed, and the job is far from easy.

That’s why we’ve started the Agony Agent series. We’re crowdsourcing the issues that conversation designers deal with, and we’re getting experts to answer them.

So, do you have any burning questions? You can send them to us here, and we’ll try our best to get them answered.