Do The AI Genies Work For Us, Or For Them?

(Original Post on Searchblog)

Of all the structural problems “Web 2” has brought into the world — and there are too many to list — one of the most vexing is what I call the “meta-services” problem. Today’s commercial internet encourages businesses and services to create silos of our data — silos that can not and will not connect to each other. Because of business model constraints (most big services are “free,” revenues come from advertising and/or data sales), it’s next to impossible for anyone — from an individual consumer to a Fortune 50 enterprise — to create lasting value across all those silos. Want to compare your Amazon purchase history to prices for the same goods at Walmart? Good luck! Want to compare the marketing performance of your million-dollar campaigns between Facebook and Netflix? LOL!

For the past 15 or so years, I’ve written about a new class of “meta-services” that would work across individual sites, apps, and platforms. Working on our behalf, these meta-services would collect, condition, protect, and share our information, allowing a new ecosystem of services and value to be unlocked. OpenAI’s recent announcement of plugins , along with their already robust APIs, has brought the meta-service fantasy tantalizingly close to reality. But it’s more likely that, just as with the “open internet,” the fantasy will remain just that. Internet business models have been built to collect short term rent. Truly open systems rarely win over time — regardless of whether the company uses the word “open” in its name .

But maybe — just maybe — this time could be different. After all, it’s happened a few times before — once with the printing press, and again in the mid 1990s, with the birth of the Internet itself.

So what’s it going to be?

I Dream of Genies

I’ve admired the meta-service problem endlessly -here’s a wishful post from twelve years ago:

While a year ago I’d only see a “service connection” happen between an app and Facebook or Twitter, lately I’ve noticed such connections happening all over the place — with LinkedIn, Google, Foursquare, and many others. I think it’s only a matter of time — and not much of it — before we have a “metaservice” hit on our hands — an entirely new and delightful service that curates our digital lives and adds value above the level of a single site.

Well, no. It’s been a lot more time than “not much.” A few years later, I thought maybe mobile deep links were going to help fix the problem. Again, I was wrong.

Frustrated, in 2018 I suggested a regulatory fix through what I called “The Token Act ” — a proposal that would force companies to offer up the data they hoard as “tokens” that consumers can trade. (This was before the crypto craze went mainstream, and was not about crypto per se). I imagined a fantastic new innovation economy built on “freeing the data.” I imagined the rise of “user agents” that watched everything you did and suggested ways to enrich your life based on data you’d created or shared. But of course, expecting enlightened regulation that delivered what amounted to an innovator’s dilemma for the reigning tech oligarchs was pure fantasy.

Now, with generative AI, all my imaginings seem possible again — but only if we get the business models right.

I’m not optimistic we will.

First, while at-scale platforms like ChatGPT or Bing Chat (or Bard, if and when it launches at scale) are fun and cool, to me they’re not where the revolution lies. To me the critical element is what, during the open web/search era, we used to call “personalized search” or “desktop search” — the application of the technology not at scale, but in a bounded environment specific to the individual using it.

In the context of generative AI, this concept might be called “Local Language Models” — bounded, specific corpus of information upon which at AI is trained (the term already exists — Ben Thompson revived it in the context of off-cloud instances earlier this month). In short, we’d train GPT “meta-services” to pay attention to all of our personal data, then apply the intelligence gathered to negotiate on our behalf with services all around the web.

It’s worth pausing and really thinking about this for a minute. Imagine a GPT chatbot that you own, trust, and control. Let’s call it a genie, because honestly, that’s the most appropriate word for this new entity. This genie has access to everything you do on your phone, your computer, and sure, why not — your Alexa, your car, basically every digital surface with which you interact. Imagine it’s bounded by immutable rules that state you and you alone can tell it what to do, what information to share, what services to connect to, on what terms, and so on. Now imagine you can ask that genie to perform all manner of magic on your behalf — pretty much any question you can think of, it will figure out an answer. That Amazon/Walmart query at the top of this post? Dead easy. How about remembering the password you used once to log into your healthcare provider three years ago? How about negotiating a way better deal with that service by threatening to move to competitors? Done! Once you’re happy with that healthcare provider, can you ask your genie to file all your claims and make sure you get reimbursed by checking your bank statements? Why yes you can! Your wish has been granted!

The value you could unlock by having natural language control of all the information you interact with every day is?… mind boggling. At some point I’ll write tens of thousands of words imaginging all the questions you could ask such a data source — not to mention what might happen if you started federating and sharing it — but let’s just agree it’d be amazing.

But the key to genies is that they work for you. If you have a choice between having your own health care genie coded by genius Valley hackers on a mission to keep you healthy (and not broke), or using the one provided by Blue Cross Blue Shield Anthem Empire Incorporated and given the stamp of approval by Apple or OpenAI, which one would you want to use?!

Enter Terms of Service

Whatever we call these magical meta-services (and I like genies), they’ll never come to pass if we continue our current model of platform data dominance. Before Google consolidated the search market, and before all our data dissolved into iPhones and/or “into the cloud,” consumer tech had multiple domains and markets. Yes, there was “internet search,” but there was also “desktop search” — for a brief moment it seemed to make sense to have a search engine for the realm of our personal computing — back when we kept important files and communications locally on our own devices. It seemed sensible to apply the wizardry of Google’s algorithms to our personal data — but as our information migrated to the services and devices owned by Apple, Google, Microsoft, Amazon, and a few others, desktop search waned as a category.

In essence we slouched toward an architecture that favored convenience over consumer/user leverage. It was convenient to let big platforms manage our documents, photos, communications, and interactions with apps and service. But we lost a lot of potential in the trade. Instead of encouraging an ecosystem that allowed consumers to wield their data as an instrument of value, we accepted an ecosystem that delivered that data to large platforms, who built exactly what any rational actor in a capitalistic system would build: Walled gardens that favored their own services. (Cory Doctorow has been ruthless in his recent documentation of this phenomenon, which he calls “enshittification.”)

Large corporate entities, no strangers to enshittification, are already figuring out the magic of AI genies. In short, they have begun to train GPT services against their private data storehouses, and then leverage the insights and output they gather in any way they see fit. Morgan Stanley, for example, “built a tool to train an instance of GPT-4 on roughly 100,000 Morgan Stanley internal market intelligence documents .” Across thousands of large corporates and trillions of dollars of our economy, big companies are busy building corporate genies that they hope we’ll have to use. But imagine the same power for all of us? Why can’t we treat our interactions with every digital service as a ‘market intelligence document’ that empowers us all? Could we perhaps leverage GPT tools to escape the “enshittification” of today’s internet (and by extension, our interactions with all large corporate entities)?

Perhaps, but unfortunately, the same forces that built last year’s model of the Internet are now hard at work building this one. OpenAI’s “plug in” architecture is bound by its Developer Terms of Service , and as I’ve written elsewhere, absent enlightened government intervention, Terms of Service (TOS) have become the defacto regulatory framework for the entire Internet .

Here’s what OpenAI’s plugin TOS have to say when it comes to what you can and cannot do if you create a plugin, which I like to think of as the equivalent of a website circa 1998.

(e) Restrictions. Your API and Plugin Responses will not: (i) pose a security vulnerability or threat to our users, us, or any third party; (ii) interact with our users in a manner that is deceptive, false, misleading, or harassing; (iii) return or contain illegal, defamatory, pornographic, harmful, infringing, or otherwise objectionable content; (iv) include any malware, viruses, surveillance, or other malicious programs or code; (v) interfere with, damage, or access in any unauthorized manner any software, technology or services of ours or any third party; (vi) use Plugin Requests to develop models that compete with OpenAI; or (vii) send us any personal information of children under 13 or the applicable age of digital consent.

That’s a lot of wiggle room, plenty enough to allow OpenAI to deem any plugin “competitive” and any third party to complain that any given plugin “interferes” with their business. Imagine those kinds of restraints in the late 1990s, when any and everyone was throwing websites up just to see what would happen. The commercial Internet as we know it would have simply — not happened. Imagine OpenAI’s TOS regulating the Internet in the early 2000s, when Mark Zuckerberg lit up Facebook — certainly Friendster or MySpace’s founders might have seen Mr. Zuckerberg as “interfering” with their businesses.

Put bluntly, there’s nothing in OpenAI’s terms of service that make me feel like anything is going to change once AI becomes mainstream — and that’s just?… tragic.

We can’t entirely blame the platforms for this sorry state of affairs. Startups like OpenAI may have made quite a show of being “open,” but in the end they just want to be the next Google (don’t be evil!) or Apple (bicycles for the mind!). Building an ecosystem driven by a consumer’s ability to instrument their own personal data would mean coughing up quite a technical and socio-economic hair ball. It would require the coordinated effort of all major platforms — and that’s like asking them all to declare a truce in a four-decade long war for market dominance. That ain’t gonna happen by itself.

If generative AI really does mark a moment similar to the dawn of the Internet, or hell, as once senior Big Tech executive recently quipped to me privately, “a moment as big as the introduction of the Gutenberg printing press” — well damn, folks, maybe we should think about taking advantage of that fact, instead of crossing our fingers and hoping the next Google or Apple turns out better than the last one?

There’s a ton more to say about this, including a deeper dive on whether advertising is going to play a role (the more I think about it, the less I think it will). But I’ll save that for the next post.?

—

You can follow whatever I’m doing next by signing up for my site newsletter here. Thanks for reading.