Make Them Dance - Lessons from history for the online future - Part Three

Rewiring the modern dance

Over two hundred years on from abolition, the subjects of dancing and coercion reappear to provide perhaps the most chilling moment (one of many, many such moments) in The Age of Surveillance Capitalism.

“We are learning how to write the music, and then we let the music make them dance,” an internet of things software developer explains, adding, “We can engineer the context around a particular behavior and force change that way. Context-aware data allows us to tie together your emotions, your cognitive functions, your vital signs, etcetera. We can know if you shouldn’t be driving, and we can just shut your car down. We can tell the fridge, ‘Hey, lock up because he shouldn’t be eating,’ or we tell the TV to shut off and make you get some sleep, or the chair to start shaking because you shouldn’t be sitting so long, or the faucet to turn on because you need to drink more water.” Shoshana Zuboff, The Age of Surveillance Capital

Ms Zuboff’s immense and important project exposes an entirely modern problem, arising from the rapid and relentless rise, across only twenty years, of a small number of technology giants.?

William Wilberforce took on the narratives that supported the slave trade - in the face of entrenched resistance from its powerful practitioners and beneficiaries, and sustained apathy from the wider public - by surfacing, one by one, the hidden trade-offs that enabled its continued enormous profits. As we have noted above, while the fighting ranged across a number of fronts, it was by successfully reframing the narrative of slavery as an abuse of human rights - at that time a startlingly modern notion - that he was able to win over an audience that ranged from the hostile to the indifferent.

Ms Zuboff, in both her book and in her subsequent media appearances, fronts up to this century’s ostensibly less horrific, yet comparably urgent, equivalent: the surveillance trade. Her unique challenge, analogous in many ways to that of Wilberforce, is that while human rights are today a more or less familiar touchstone, and hardy, credible campaigners such as Jaron Lanier have already broken the ground, their forensic defence, in the immense and almost entirely unregulated Wild West of cyberspace, has never before been attempted.

As a result, she has to work extra hard to make us understand the full set of hidden trade-offs on which the wealth and power accumulated by the surveillance traders to date, and their continuance into the future, depend. Surveillance Capital is a long and demanding read, and not just due to its sheer length. The depth, breadth and carefully researched and detailed evidence that carry the argument are sometimes overwhelming. And the passion and skilful rhetoric that Ms Zuboff sustains throughout her masterpiece can, it must be admitted, be emotionally wearing for the reader.

Fortunately for those short on time and attention span, Ms Zuboff has summarised her narrative in substantial thought pieces for both The New York Times and the Financial Times.

Here, for example, in an article titled “You Are Now Remotely Controlled” she lays out the headlines of the hidden trade-offs that we buy into as ‘users’ of Google and Facebook.

It’s not surprising that so many of us rushed to follow the bustling White Rabbit down his tunnel into a promised digital Wonderland where, like Alice, we fell prey to delusion. In Wonderland, we celebrated the new digital services as free, but now we see that the surveillance capitalists behind those services regard us as the free commodity. We thought that we search Google, but now we understand that Google searches us. We assumed that we use social media to connect, but we learned that connection is how social media uses us ...

All of these delusions rest on the most treacherous hallucination of them all: the belief that privacy is private. We have imagined that we can choose our degree of privacy with an individual calculation in which a bit of personal information is traded for valued services — a reasonable quid pro quo. For example, when Delta Air Lines piloted a biometric data system at the Atlanta airport, the company reported that of nearly 25,000 customers who travelled there each week, 98 percent opted into the process, noting that “the facial recognition option is saving an average of two seconds for each customer at boarding, or nine minutes when boarding a wide body aircraft.”

In fact the rapid development of facial recognition systems reveals the public consequences of this supposedly private choice. Surveillance capitalists have demanded the right to take our faces wherever they appear — on a city street or a Facebook page. The Financial Times reported that a Microsoft facial recognition training database of 10 million images plucked from the internet without anyone’s knowledge and supposedly limited to academic research was employed by companies like IBM and state agencies that included the United States and Chinese military. Among these were two Chinese suppliers of equipment to officials in Xinjiang, where members of the Uighur community live in open-air prisons under perpetual surveillance by facial recognition systems. New York Times, January 24th 2020.

We should take note of that sharp and rapid Through The Looking Glass flip, from Utopian service innovation to Dystopian obfuscation and manipulation. It provides a dramatic example of dropping below the ostensibly harmless, value-added surface trade-offs whose narratives we accept as unquestioning users, and the extensive, deeply disturbing underlying trade-offs with which we need to come to grips.

On the subject of unquestioning and obfuscation, Zuboff digs into the maze of service agreements that we are required to sign, in order to get access to the full range of services on offer from the surveillance traders. University of London research in 2017 determined that “were one to enter into the Nest ecosystem of connected devices and apps, each with their own burdensome terms of service for third-party data sharing, the purchase of a single Nest thermostat would entail the need to review nearly a thousand so-called ‘contracts’.”.

Later on in the same New York Times article, we are carried persuasively - irresistibly - from the ‘What’ of the surveillance trade … to the mechanics of the ‘How’.

Early on, it was discovered that, unknown to users, even data freely given harbors rich predictive signals, a surplus that is more than what is required for service improvement. It isn’t only what you post online, but whether you use exclamation points or the color saturation of your photos; not just where you walk but the stoop of your shoulders; not just the identity of your face but the emotional states conveyed by your “microexpressions”; not just what you like but the pattern of likes across engagements. Soon this behavioral surplus was secretly hunted and captured, claimed as proprietary data.

The data are conveyed through complex supply chains of devices, tracking and monitoring software, and ecosystems of apps and companies that specialize in niche data flows captured in secret. For example, testing by The Wall Street Journal showed that Facebook receives heart rate data from the Instant Heart Rate: HR Monitor, menstrual cycle data from the Flo Period & Ovulation Tracker, and data that reveal interest in real estate properties from Realtor.com — all of it without the user’s knowledge.

These data flows empty into surveillance capitalists’ computational factories, called “artificial intelligence,” where they are manufactured into behavioral predictions that are about us, but they are not for us. Instead, they are sold to business customers in a new kind of market that trades exclusively in human futures. Certainty in human affairs is the lifeblood of these markets, where surveillance capitalists compete on the quality of their predictions. This is a new form of trade that birthed some of the richest and most powerful companies in history. New York Times, January 24th 2020.

This is how - indeed it’s perhaps the only reliable way in which - a hitherto unshakeable dominant narrative can be questioned, immense power and influence challenged, and public apathy stirred into protest. It must be dug under, in order to uproot the beliefs, the logics and the assumptions that enable it to control and steer the discourse.

The ‘Why’ of the surveillance trade is, by the way, in the midst of our shock and outrage, a rare instance of sadness and regret. The innocent early narrative of Google’s own, initially idealistic and principled founders, is kicked firmly into touch by the harsh realpolitik of Silicon Valley investment. This switch from ‘customers uber alles to ‘customers as unpaid gold miners’ now informs the proposition - declared to hungry investors, concealed from ‘users’ - of more or less every start-up seeking to emulate the success of Google and Facebook.

Yet there is a two-sided nature to surveillance capitalism that makes it so dangerous, in Zuboff’s view, concealing the dark reality behind the public illusion. Google’s users are not its customers, which means it is radically indifferent to their real interests. Advertising-supported search engines will always prioritise those who pay the bills over those who use its services, so long as they remain hooked.

That also used to be the view of Sergey Brin and Larry Page, Google’s founders, who presented a paper in 1998 highlighting the perils of advertising. “We expect that advertising-funded search engines will be inherently biased towards the advertisers and away from the needs of consumers. This type of bias is very difficult to detect but could still have a significant effect on the market,” they wrote.

That worldview changed when Google realised that the behavioural insights it could draw from its data were a potential gold mine, offering far greater rewards than other advertising-driven businesses such as commercial television.?John Thornhill, Financial Times, January 4th 2019

The limits of the remit

One of the very few criticisms that have been aimed at Ms Zuboff’s book is its perceived failure to propose tangible solutions to the looming human rights crisis caused by the rise and dominance of the surveillance trade.

If this is such a big deal, where should we go from here?

As I understand it, the author is too smart to fall into that trap, and I’d like to think, also too humble to dream of standing as the tiny David against these immense Goliaths. To propose a set of solutions - to go up against these powerful, wealthy and well-connected traders - would not only play directly to the strengths of the opposition.?

We are reminded of lessons from The Art of War … to take on the enemy when conditions are so clearly advantageous to them would be suicidal to her cause. Ms Zuboff has set out - just as Wilberforce did in his maiden abolition speech in 1789 - not to end the war, but to begin it. As such, her primary concern is to unpick and lay bare the underlying narratives that provide the current foundations of the surveillance trade. She has, to my mind, entirely succeeded.

The rest is - and we really don’t want to hear this - up to our governments. We see promising regulatory signs this year in the EU, which is not in such utter thrall to Silicon Valley as the US, which makes do with feeble sabre rattling (has this changed under Biden?). But judging by the recent record of, for example, the awful Johnson government in the UK, we must conclude that it’s pretty much up to you and I.

The battle lines have certainly been clearly drawn for us. Are we able to rouse ourselves from the pervasive, Brave New World lethargy that plagues us today?

Or shall we just keep on dancing?