AI without the blunders

This morning I came across an article from CIO titled "10 famous AI disasters ":

and it got me thinking about the different tips and tricks we use to avoid issues in our AI systems after working in this area for 10 years at the highest levels.

At it's core, though, it constitutes safe and responsible research and development.

Detailing the issues and the ways around

What's listed here is the biggest blunders the industry has seen in the last few years and what we need to do to get around those issues.

I usually subscribe to policies instituted at Big Tech companies not only because they're the most robust, but also because they are very comprehensive and transferable to other companies.

Air Canada: The lying chat bot

While a relatively minor overall issue, it still doesn't look great for company reputation that this happened.

The crux of the issue was someone unable to take advantage of bereavement fares and it's assistant lying about it.

Chat bots can be hard to get right when we're really relying on the underlying model to have done more due diligence.

Keep in mind though this is a shared responsibility and constitutes we put sufficient testing at multiple levels instead of just standing up a basic wrapper for another software product.

Sports Illustrated: Publishing fake writers

Big companies can't do it all and they often rely on outsourced work and arbitrage to fully fulfill the demand we're seeing in the market.

How this went down was a third party group's writers used pen names which is ok, but didn't keep it human-generated or human-assisted.

Ghostwriting is still ok too, but complete fakes is a blow to journalistic integrity that the writers should have imagined would be common-sense.

My take here, if you come to an unknown gap in the road, act in a way that empowers the people around you and have a conversation.

Gannet: AI provides poor writing

As we move into the AI-assisted area, we have to be vigilant around what and how we produce.

I have seen AI produce both bad and good writing with the differences being:

1. Length of the model's context window
2. Number of parameters present in the model
3. Use of multi-agent workflows that can correct each other to produce high quality
4. Great writing pieces that the AI can build off of already.

In Gannet's case, they didn't handle replacement of inserts well either which led to a number of issues.

iTutor Group: The Ageism Effect

Diversity, equity, and inclusion for all is a massive commitment, especially when it comes to implementing great software systems.

In this case, AI recruiting software was rejecting female candidates 55+ and male candidates 60+ with around 200 people total affected.

What this comes out to is proper care and design in the backend for how to design to these standards and is an evolving area of research.

The court case files

In 2023, a lawyer found himself in trouble for relying on ChatGPT as a case research tool.

It is possible to build a tool that can take care of this for people, but it is additional work on top of ChatGPT, not just research with the model itself.

An interesting parallel to this is research paper tooling that has come out making handling of this side of things easier.

While it may or may not have made a lot of traction, it's just a next step into an age of convenience that can be helpful.

AI algorithms can't identify COVID-19

When COVID-19 hit, a mad scramble happened to understand the virus and see what we could do to address any gaps in knowledge.

Unfortunately, this led to a lot of poor research design that didn't meet standards laid out by existing frameworks for a variety of reasons.

When implementing systems, move slower and opt for a more rigorous design process as this will provide users a better end product in both the research and engineering processes.

Zillow's algorithmic home-buying disaster

Through Zillow Offers, the company would make cash offers on houses with plans to flip them as soon as possible.

Unfortunately a multitude of issues hit:

1. Large inaccuracies according to the business needs
2. Black swan events including COVID-19 and home renovation labor shortage

It's very easy to account for number 1 by instituting controls around levels of accuracy and refining the R&D to make target guidelines.

Number two may have surfaced through operational monitoring, but is hard to know without being there.

These in particular were chocked up to being an outlier year by the wider engineering community and exlcuding them from analysis.

Inequitable high-risk healthcare

A famous case happened across an algorithm used in hospitals and insurance companies to recommend patients for "high-risk care management" where Caucasians were more likely to be recommended than BIPOC individuals.

This should have been addressed in data design and throughout the research and development process with testing specifically poking and prodding around key areas of software concern.

If it still came to be inequitable even after testing, then scrap it, but it doesn't mean you can't build empowering systems handled correctly with the right people.

MSFT's troublesome chatbot

In 2016, Microsoft built a chatbot called Tay which was trained on Twitter data and made available online for others to interact with.

After some time, users had it spewing hateful rhetoric not reflective of MSFT as a company and it took down the bot.

What I largely see here is the inability of a group of users to act responsibly.

We account for this now with appropriate cleansing of data and interactions early in the process, but it would save a lot of trouble just to throw out any interaction with a tool like this that doesn't meet responsible user guidelines.

Amazon's AI recruitment tool

In 2014, Amazon developed an AI recruiting tool largely based off men's resumes that would rank candidates on a scale of 1-5.

Because of implementation, women didn't get equitable access and were thus denied proper care.

Ultimately, the project was scrapped, but what they should have kept an eye on was proper data design from the outset.

10 years later, we have the benefit of hindsight, but it goes to show the need for care in the development process.

Working through the disasters

AI research, engineering, policy, and strategy doesn't have to be hard.

In many cases, we get the benefit of hindsight in the market to get around issues, but it is all about proper care and due diligence in the R&D process with intentions towards empowering the next wave of growth with humanity at the forefront.

Structure AI can help with this as we keep even the deepest client needs in mind throughout the process and help them safely navigate the waters of challenging policy and software implementation.