Stop worrying about ChatGPT taking your job; harmful AI is already here and you probably didn’t notice it

AI has captured the public’s attention over the past year, especially with advances in generative AI (e.g. ChatGPT) that are accessible to the average user. Along with this public attention has come speculation about how this AI might be used for humanity’s benefit, and its detriment.

The day may come when these new advances in AI are used in harmful ways on a large scale, or when large numbers of workers are displaced by AI-based tools. Until then, you’re worrying about the wrong thing. Other types of AI are already hurting you, and society at large, on a massive scale and a daily basis.

This post is about one of those harmful uses, one that (in an article last month) Harvard Business Review called a “decisive differentiator” that allows a business to increase its profits by “double-digit percentages”. Sounds great, right? Here’s what it really is: a specialized type of AI model that is invisibly manipulating nearly every financial decision you make.

You probably aren’t even aware of it. Maybe you sense it in subtle ways from time to time. Something just feels different; things are more stressful, more strained than they used to be. You might blame global inflation, social media, the pandemic, or your favorite political scapegoat, but you can’t quite put your finger on it.

There’s a lot going on in the world, and more than a few reasons to feel this way. But, there is evidence about one potential cause, one that has gained widespread usage over the past 2–3 years. Let me tell you where this AI technology is being used and how it works, and then we’ll take a look at why this is harmful to you personally, and society at large.

To understand this better, we need a bit of background. How does a business decide how to price its products?

Early in my career, I worked in a warehouse for a business that sold and serviced heavy machinery (trucks, excavators, etc.). When we sold parts to a customer, we understood the margin that we needed to earn in order for the company to make a profit (it was around 15–20%). So, we’d take the wholesale cost we bought the part for, add 20% to it, and sell it for that price. If we really liked that customer, or they claimed a competitor was offering them a lower price, we might discount by only marking it up 10% or 15%. Selling at a reduced margin was better than not selling at all. This was, in the minds of the family that owned my company, an honest way of doing business.

It was a pretty naive approach, but it was how many companies set their prices then.

Successful companies have since learned that competitive pricing is much more complex than cost-plus-fixed-margin and realized that they need to consider other factors, including the customer’s perception of the value of the product, brand recognition, the number of alternative products the consumer could buy instead, the prices that competing products are selling for, macroeconomic conditions, seasonal buying patterns, and so on, what economists call pricing elasticity. Large retailers had entire departments of people focused on gathering this type of information and setting prices for their products. At it’s simplest though, pricing was still a basic attempt to present an economic cost that the customer would consider fair relative to the value of the product, in order to maximize revenue.

Before online retail and electronic restaurant menus, changing the price of a product was expensive and time consuming. Remember being a kid in a department store watching people walk around with those sticker guns covering the price on each clothing item with a new price tag? In those days, a price was only changed if there was a compelling reason that outweighed the expense of updating the prices manually.

The advent of online retail meant that prices could be changed much more easily for a large portion of a company’s sales and pricing strategies adapted to some degree. Businesses in highly seasonal industries (like airlines and hotels) or luxury product markets adopted more aggressive pricing strategies, adjusting prices higher or lower based on expected demand.

All of that has now changed. Enter this powerful new type of AI that is already in widespread use.

This AI collects more information than an individual human can possibly comprehend. It is watching the websites of every competitor and tracking their prices. It is measuring consumer sentiment, economic conditions, the weather, social media trends, buying habits at specific times of day and week in each geographical area, estimates of your income level, and thousands of other variables. Then, it’s using data centers the size of a city block to do one thing: find (down to the penny) the absolute highest price you are willing to pay for a product at any given moment.

More than that, it is continuously experimenting on you and learning. It plays with the price of a product and observes consumer behavior. Did the customer leave and buy elsewhere? I’ll charge the next customer a few pennies less. Did my competitor raise their price? I'll instantly raise mine too. Is it going to snow next week in Chicago? Let’s see how high I can raise the price of gloves and shovels there before customers start choosing alternatives. Then, let’s use that information to set prices before the next snowstorm. Is my competitor out of stock on an item? Raise the price, because now customers have no choice. How big of a “service fee” can I add on at the last minute and still get a purchase?

And it’s doing all of this in real-time, in milliseconds. Since its invention a few years ago, almost every routine purchase you make is now under the influence of this type of AI. It’s not just luxury or discretionary items; it’s even basic essentials like clothes, household goods, even groceries. If you rent your home or apartment, your next lease will likely be priced using this type of AI.

Don’t just take my word for it. Here is a step-by-step guide about how to implement this at your business, courtesy of Harvard Business Review. You can also buy the use of this AI technology from companies like Turing, DynamicPricing.AI, and McKinsey’s Periscope.

Here is a real-life example using Keepa.com, a website that tracks price changes on Amazon. Below is the price history for a 5-day window on my favorite protein bar. On May 18th, the price fluctuated 5 times in a 6-hour period, ranging from $22 to $27!

What changed? Did the cost Amazon paid to put those bars on the warehouse shelf change? Did the contents of the box of bars change 5 times that morning? Did Amazon encounter “supply chain issues” between 4am and 5am? Nope! For some unknown reason (probably unknown even to the people at Amazon.com that built the AI) the AI model determined that it could charge that customer $27 instead of $22 for a brief period of time and still make the sale. Maybe it knew that Thursday mornings are when busy parents are most likely to notice that stock on the kids’ breakfast is running low, and reorder without price shopping first. Whatever the reason it made that pricing decision, be assured that the decision was to Amazon’s advantage, not yours.

It’s price-gouging and collusion on an individual scale. From the customer’s perspective, this AI is essentially learning the moments of weakness in your financial decision-making process and exploiting them with higher prices. Additionally, in the view of some experts, it is indirectly coordinating with competitors to raise prices against you. And it’s all completely legal.

If you prefer to shop in brick-and-mortar stores, you aren’t safe from this either. Retailers like Walmart have already started to apply this in-store, replacing price tags with digital signs that can be updated instantly and automatically. Research firm Gartner predicts that by 2025 all major retailers will have implemented “in-store real-time pricing”. Even restaurants have started experimenting with electronic menus that use AI to raise prices during peak dining times based on factors like the weather, traffic, and how busy other nearby restaurants are.

Now that we know how it works, we can probably agree that it feels a little shady. It offends our sense of fairness and equity. You may wonder, “Why should I pay more just because I live in a certain area, or chose to buy at a more convenient time of day or week?”

Further, this is obviously impactful to your household budget. If sophisticated AI is learning our purchasing decisions and finding ways to charge us more for nearly everything, that means we’ll have less money than we used to have. I’m sure that you can relate to that!

But, is it really harmful? I think so, and here’s why.

First, a brief bit of context: Behavioral economics is the study of why people make the economic decisions that they do. It’s a fairly new field of study, pioneered and popularized by researchers like Dan Ariely at Duke University. Their research has identified a psychological phenomenon that occurs as part of an economic exchange (e.g. the purchase of a good or service) that they call the “pain of paying”.

It works like this: Every time you experience or are about to experience economic loss (in this case, paying for something), you experience psychological pain. The greater the perceived loss, the greater the pain you experience. This pain is somewhat reduced by the value you receive in exchange for your loss. If you perceive that the value of the product is worth the financial loss, you might call that a “good deal” or a “fair price”. If the pain is too great, you will make the decision not to purchase.

Researchers have explored this phenomenon further and learned that when the brain is observed via fMRI during these decisions, the pain of paying activates the same brain functions as physical pain. In other words, this affects your brain in nearly the same way as actual physical pain.

So, what happens when an AI technology purpose-built to find the exact maximum amount of “pain” you are willing to accept at any given moment is continually working against you in every economic decision you make throughout the day?

The long-term effects of various types of pain are well-documented. At the least, it negatively affects your mood and increases irritability. At worst, it can lead to depression, anxiety, and other forms of mental illness.

I hypothesize that this implementation of AI across nearly every area of our economic lives over the past few years has embedded an elevated level of continual psychological pain and fatigue within our daily lives that was never before possible. No longer does the individual feel that they are receiving fair value for their economic loss, in nearly every transaction they engage in.

Instead of feeling as though you’ve gained something, with every transaction you feel that you have lost, but had no better option in the losing. Put plainly, you continually feel like you sorta just got ripped off. There’s a vague feeling of frustration, threat, or danger to your well-being in every transaction.

The past few years have been challenging and tumultuous for most people, and there are many factors contributing to it. I think we’re overlooking a huge one. In this case, AI is being used to inflict tangible harm on society, in the name of increasing (already record-high) profit margins.

Unfortunately, I’m not sure there’s much we can do about it.

Just being aware of it helps a little bit. We can also push our legislators to strengthen price-gouging laws to address the more egregious examples of this behavior. We can push them to demand more transparency into the pricing decisions these AI tools are making. It’s very likely that if someone were to take a closer look, they’d find that these AI models are discriminating against specific groups of people in unacceptable ways. Research on this is currently underway.

If you enjoy the thrill of a bargain hunt, there are online tools you can use to track price changes on your favorite retail sites and pounce when the AI model has lowered the price of something you want to purchase. This requires a level of vigilance that most people don’t have the time for, but it may help restore a sense of control for some.

As AI continues to advance, I believe that it will benefit us in many ways. However, the potential for harm won’t be realized in cataclysmic events wrapped in sensational headlines. The harm will be insidious and subtle, but impactful nonetheless.

Lastly, it’s important for us to remember that AI does not have moral agency. It does what it is built to do, and the builders are humans. Maybe, the best we can do is hold those humans accountable, both the builders and the users of this technology. For those of us who work in the development of AI models, this also means examining the potential human impact of our model and refusing to proceed if we foresee harm, even if we know that our competitors do not have the same moral fortitude.

Note: Opinions expressed are my own and don't represent any current or past employer/client.