Slow down! ChatGPT requires data strategy

The marketing and education worlds are all a-buzz about ChatGPT. The ability to create content using AI - just think about how much time and money will this save organizations! The reality is that ChatGPT is a communications tool. And if you use it, you're going to need a data strategy.

Let's talk about how we've cheated ourselves in the past by hastily adopting new comms tools, what ChatGPT actually is, and some of the risks to consider.

Misinformation brokers are ALWAYS early adopters

I gave a brief history of how misinformation brokers have always been the early adopters of communication technology innovations in the Defcon Misinformation Village last year. From newspapers to movies to radio, people that want to mislead others love figuring out these technologies before the mainstream can catch up.

They certainly co-opted social media platforms, almost from the beginning. Misinformation weaponized social media, and most of us are just catching on to what happened. I have been warning about what would happen if we didn't set up defenses against misinformation brokers misusing social media for over a decade.

As I watch the ChatGPT hysteria, I am worried we haven't learned a thing.

What is ChatGPT?

ChatGPT is an AI model built and trained by OpenAI. This is how OpenAI describes ChatGPT:

We’ve trained a model called ChatGPT which interacts in a conversational way. The dialogue format makes it possible for ChatGPT to answer followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests. ChatGPT is a sibling model to InstructGPT, which is trained to follow an instruction in a prompt and provide a detailed response.

ChatGPT is built on the language model (LLM for Large Language Models) GPT-3. "GPT" stands for Generative Pre-trained Transformer. These modeles are based on Transformer technologies developed by OpenAI.

This article by Patrick Meyer gives a very good primer on how ChatGPT came to be. Here is his description on what was used in the training process:

GPT-3 has been trained on hundreds of billions of words from Common Crawl, WebText2, Books1/2, and Wikipedia in English. It has also been trained on examples of programs coded in CSS, JSX, Python, etc. It accepts 2048 tokens as input, which allows it to handle very large sentences of about 1,500 words (OpenAI considers a token to be a part of a word of about four characters and gives as an example that 1,000 tokens represent about 750 words).?

Please check out his article to learn more details, it is very interesting!

Who is rushing to use ChatGPT?

I'm seeing lots of marketers and educators rushing to embrace ChatGPT. How great would it be to give sales a tool to find that content that they constantly say they need? Although if sales never bothers to look where you tell them to find it now... I'm not convinced they will talk to an AI bot that will just tell them again. :)

But we are imaging what *could* happen. Imagine the works that could be saved when the product gets new features. All you'll need to do is train ChatGPT on the new features and then ask it to rewrite the content with the updated features. AMAZING!! Everyone will be so happy!

What could go wrong?

Use ChatGPT strategically

Marketers and educators, it is incredibly important view these APIs as tools. It is something to add to your data strategy, another tool to serve your audiences. It's very important that you understand how these models are trained, and how they can be trained.

It is really important to decide if this tool will fit the needs of your audience!

For instance, lets be honest about most sales teams. They are not going to change just because you get content to them faster. If the content doesn't give them what they want (e.g. how your product is differentiated in the market, talking points that will engage customers), or even worse if the data is outdated or wrong - sales is still going to complain.

Don't just jump on the bandwagon to get any old content out. Plan what needs to happen, investigate any API tool you chose, and get an SME to review what comes out of this tool!

Consume content generated by ChatGPT critically

We need to expect that misinformation brokers are going to use ChatGPT to spread disinformation. This journalist gave ChatGPT a few prompts for a story, and was surprised to find it made up quotes from a CEO. Other researches have found that ChatGPT makes up sources.

We can be sure that misinformation brokers are going to use a tool a powerful as ChatGPT. But how can we use digital literacy to counteract their misinformation when their AI tool makes up sources and speaks so authoritatively?

Consider all of the risks

The ACM article "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" (available here, membership required) by Emily M. Bender, Timnjt Gebru, Angelina McMillan-Major, and Shmargaret Schmitchell outlines risks to consider:

Environmental.

The environmental cost of running the infrastructures required for these models grows with the size of the model. Training a single (BERT) model on GPUs costs about as much energy as a trans-Atlantic flight!

This impacts marginalized communities the most, because the materials sourced for these infrastructures are extracted closer to the. And because marginalized folks are least likely to benefit from progress achieved by large learning models, it is like they are charged twice with nothing to show for it.

Documentation debt.

As these training models grow, it gets harder to understand what is in the training sets. The article states, "While documentation allows for potential accountability, undocumented training data perpetuates harm without recourse."

The authors suggest setting aside budget for curation and documentation, and don't create datasets larger than you can document. After all, you're going to be responsible for the data you create using these AI models.

Encoded bias

The authors point out that ChatGPT "absorbs the hegemonic worldview from their training data. When humans produce language, our utterances reflect our worldviews, including our biases. As people in positions of privilege with respect to a society’s racism, misogyny, ableism, etc., tend to be overrepresented in training data for LMs , this training data thus includes encoded biases, many already recognized as harmful."

Since the sources ChatGPT used are online sources, mainly authored by men with privilege, it must be expected that their unintentional (as well as intentional) bias are used to train the language model. This means you have to expect that ChatGPT's creations will carry those same biases.

Do we really want to go backwards 100 years, just to use a new tool? This is something you must consider and put in the guardrails to protect your organization.

What does all this mean?

Technology is always a tool. Tools can be used to mislead or exploit others, even when it's not our intention. When you use any new technology, especially when you rely on it to cretate content, you must also have a strategy. Make sure you are building the right content for you users, and also be sure to build a way to protect those who could be harmed.

Let's not get to 2034 to start having conversations on how this tool could be used to harm our society. I may be too close to retirement to have this discussions again. :)