Wasted years: How personalization lost 20+ years heading into the wrong direction

'People keep telling me they know me. I’m afraid no one does.'

- Rey (Star Wars: The Rise of Skywalker)

I have lost the count of the posts of mine who start with Gonzalo Martín throwing me a curveball, with this time being helped by another friend and expert, Borja Adsuara:

Coming from a spanish newspaper article, with the header being "Spanish consumers are among the most willing to share personal information with brands"

That leads to an Accenture study that you can find here:

I have taken my time to write this article because I didn't know if the number of wrong statements that were present in the Spanish article were the result of the study or was something that got somehow lost in translation.

And well, it's just a mixture. Beyond anything else, the study is...

BLAND.

does not provide significant insights. The fact that the questionnaire is not available (As hinted by Gonzalo as well on the thread), which leaves a reasonable suspicion that some nudge could have been used here and there, and the fact that I have been exposed to first-class research from my partners at Singular Solving has vaccinated me against all kind of middle of the ground, biased research.

I am pretty conscious that the study is made to sell the programmatic services of Accenture, and therefore I could not expect a full-frontal assault on the premise, but it's not the first article that I have come as of late trying to ride the Personalization horse again.

And again as well, with the saddle turned the wrong way.

Let's dwell on the issue.

My personal story on the matter

It was 2002 when a bunch of misfits that included me, with backgrounds on Business Intelligence, Data Management, and Cx/Ux decided that what was made at the moment in terms of CRM, customer knowledge and profiling was lacking a lot of real punch. We were fascinated with the vision that people like Peppers and Rogers were painting (Later, very much like Sci-fi aficionados, we noticed that our authors talked about teleportation without knowing jack-shit about how to really execute it) and the chances of people stopping being anonymous mass to become individuals.

I didn't realize at the time that what it started as some kind of side Intelectual pursuit (I was a CTO for a UK tech consultancy at the moment at the time) would end up as some kind of lifetime professional pursuit of sorts.

And it was not because it became a boom and an easy way to cash in; because big companies rushed to it, like Social Media or E-commerce were in its time, which fueled some explosive (and mostly shortlived) growth for some companies. Broadvision comes to mind for example, with its moniker "Personalizing e-business"

(I could talk as well at length at the use of a platform as a vehicle to sell a vision, but that's another article. I wrote a bit about it here)

In 2004, we wrote "Personalización" (Personalization) which was the first book on the matter in the Spanish language, and as I found much later, the first one on setting the bases of a discipline that is much spoken of, but, seldom really known.

We tried very hard to live off it, with several startups created around the concept and some big projects pulled off in the process. And beyond everything, it was an incredible source of learning and discovery. The more we dig it, the more water the dwell provided. It was an unending trove of learnings and constantly open, unexplored paths for the adventurous explorers to plunge into.

On the entrepreneurial side of it, it proved too soon and we were an inconsistent cast to make a proper company because talent is not everything regarding setting up a company. At the end of the day, I am the only one left from that initial bunch of misfits who keeps pushing it, and who has expanded the intellectual corpus around it, which only proves that I am too stubborn to give up or too stupid to get the messages.

Or maybe, the fact that in the meantime, I have noticed that while interest in personalization has been ever in the rise (Very much like IA, which is another clear victim of the disparage between the vision and the capability to execute it) the degree of understanding of the basic underlying principles is not precisely keeping up the pace.

Let's dwell on it.

Visions vs. Execution

I have already mentioned several times that there is a significant distance between a vision being incorporated into culture and technology and knowledge being able to put that vision to work. I usually mention the case with teleportation, fully incorporated into the culture (From movies to TV series to Videogames like Portal) but still at large in terms of implementation.

AI is another classic case of the same situation. The image has been embraced by culture in the fullest, but the discipline is still struggling to provide the kind of competitive advantage that is expected on it, especially with the kind of money that has been thrown to it. Because, as usual, people have put a completely insane amount of expectations on it. I did a full rant on the matter here.

On the other hand, the personalization case seems to be a case of people suffering of two different, distinct maladies at the same time.

• Identifying personalization with the lowest common denominator and
• Not being able to grasp basics of what makes it really tick

Now that I come to think of it, on the aforementioned rant I mentioned that the trouble with culture vs. expectations debate, the personalization and AI cases are somehow intertwined:

At the same time, and much more subtle and lurking under the technical glitz, the dream of a machine able to be able to react accordingly and identifying the proper personal traits of its owner has been as well there since the beginning. But as AI algorithms have make progress by leaps and bounds, regarding personality, we are pretty much in the same place we were 40 years ago.

Sorry for quoting myself, but as the classic said, it adds spice to my conversation.

So, let's try to dwell on the aforementioned maladies.

Personalization has not yet reached a real Summer

I already mentioned in the aforementioned rant that AI has had more winters and summers than I can recall (On my professional life, I can count three o them) but Personalization has never had a real Summer as the one AI is going through now, even if it ends again in another winter.

What there has been bad been several personalization piggybacking efforts. Several people have tried to sell to the world a personalized future vision and on those premises, selling books, speeches, consulting, software, hardware or eyeballs.

So, let me state a quick recap of how many times the industry has tried to bank on personalization.

• The Peppers and Rogers Wave
• The E-commerce Wave
• The advertising Wave
• The AI Wave

All of them completely missing the point for different reasons.

Let's dwell on them. And remember this crude timeline by Forrester, not because it shows incredible insight, but because it points the arrow in the right direction, and makes more puzzling the fact that personalization seems to be still and uncracked safe box even on the fact that the demands for it have never been higher.

The Peppers and Rogers Wave

Starting in 1993 with the classic "The One to One Future" by Peppers & Rogers, which was an incredible demonstration of an "all vision and no delivery" approach that successive books on the matter didn't improve in a significant way.

Well, they came from the Marketing camp and didn't have a clue on how to land the ship. At least they were pioneers on bringing the matter to the table. As I already mentioned, the matter has been around forever since we somehow crossed the line regarding mass production age and we dwell into the "something more" department.

The E-commerce Wave

The aforementioned Broadvision example was possibly the poster boy of the generation, but no e-commerce startup deck on that time was without its personalization invocation. It was the first wave of two distinct things that have plagued the personalization camp since then: The terrible turf war in order to achieve personal relevance, brilliant and succinctly resumed in this Eli Pariser phrase from The Filter Bubble (Penguin Press 2011)

“In 1995 the race to provide personal relevance was just beginning. More than perhaps any other factor, it’s this quest that has shaped the Internet we know today”

and the start of another neverending trend: The use of personalization to sell extremely expensive hardware and software which promised to deliver it with the click of a button. (Hint: You are going to end up with a very overpriced paperweight).

The Advertising Wave

Spearheaded by Facebook and Google, we are just in the middle of the phase where we noticed that we got it completely wrong again. I just made a complete rant already on the matter here, on the advent of the so-called S4 companies, so not dwelling into it further, but let's notice again that the matter was perceived as nothing short of the Holy Grail on its time, as this sentence of Ben Kunz, on the Thoughtgadgets blog on May 2nd 2011 proves:

“Personalization has become the Great Pumpkin of the ad universe, always almost here, and when it someday arrives it will be really, really big.”

As a coda, this tweet is as beautiful as suggesting that in fact, what really made advertising working in its heyday, were its capabilities to build commonly shared stories and that it's basically incompatible with personalization. I don't know if this is a classic tale of the fox and the grapes, but hey, at least it explained why broadcast advertising worked until the fragmentation and infinite inventory turned the model into Blockbuster ashes. What is painfully clear is that there is no working personalizable advertising model coming from pivoting from the old industry mechanisms.

Ah, and in the process, it states the obvious: The promises made by the S4 are bigger than their delivery capabilities. Shocking, right?

The AI Wave

Again I was in the middle of it (Now that I come to think about it, it seems like a ritual of sorts). It started with autonomous agents in the 90's (I was working with them on my IT nerd phase) followed by the Recommenders Wave on the '00s. I can recall attending Recommenders 2006 in Bilbao and setting up a couple of startups myself based on recommender technologies.

The problem again lied on the lack of real personalization efforts and the reliance on weak personalization methods, like collaborative filtering and such. If you still wonder why your recommendations on your VOD platform still sucks, it's because we hit the ceiling regarding the path taken a long time ago. In fact, Netflix has improved its basic algorithms very little in the last decade. In fact, since they prized a psychiatrist in 2009 for trying to shake it up a bit with some lateral thinking.

In fact, we tried to overcome the limitations of the existing algorithms and developed a complete architecture based on something called balanced recommenders with mixed personetics and recommender engines.

It worked like a charm, but we were not able to push it to the next level. We were academic nerds that failed miserably on the funding chapter of startups and we were not in a friendly environment to help us (Wrong country, wrong ecosystem, wrong choices from our part). Well, shit happens.

Well, at least we are somehow vindicated by the fact that even at 2020, most recommender systems still show ample margin of improvement because they picked a cul-de-sac regarding how to use personetics and context management.

For example, this is an example of ours more than ten years ago. And yes, it worked. We found checking NHK (Japanese television) program taxonomy that there was a state of watching TV called "Nap/Background noise" and that there were contents that induced you that drone effect. And that was the same person who could pay a lot of attention under other circumstances. That information was nowhere to be found on past behavioral data. That's real zero party data and real-time context management in 2009!!

So, what the hell is happening?

Problem is, core architectural beliefs that have underpinned the way people have been implementing personalization have been completely misguided. They have been clouding it in some kind of technological hocus pocus in order to hide that in terms of core personetics, they are completely lazy exercises based on several trivialities and platitudes.

And the results are the ones you could expect: Bland efforts and not kept promises. Don't want to be exhaustive, but will pick some examples all headed in the wrong direction:

• Pseudorandom experiment mixing basic IoT, Big Data, and the magic algorithm in Lanzarote, Spain, to determine how many customers are coming. "Digital Oracle" is mentioned at some point. Yes, really. https://retina.elpais.com/retina/2019/08/30/innovacion/1567171473_651450.html
• I can recall a joint effort in 2011 with players like Technical University of Munich and Swiss Post, to name a few. It finally, as usual, ended up in half-cooked efforts like this one. It made some waves, and we were hopefuls that it would but the underlying message was "Hey, that would be great if we could just nail it, but we don't have any clue on how to do it beyond some hasty, trivial, non-gauge moving efforts". another exercise on "how awesome would be to be able to teleport".
• IBM is a personal totem of mine regarding making waves around personalization based on complete crap. I already wrote a rant exclusive on them three years ago and it seems I was not particularly wrong as they made a divestiture of their whole marketing apparatus https://www.ibm.com/supply-chain/acoustic-divestiture?

I could go on forever, but I hope my point is through. Tons of spaghetti has been thrown to the wall on the hope that the personalization promise would crystalize somehow, and the results have been disappointing.

The whole concepts that have dominated the broadcast scenario are basically untranslatable to hyperpersonal spaces and have derived into thew S4 nightmare that I already mentioned in the aforementioned article.

Advertising is completely lost, and I already mentioned that programmatic was not going to save the day either.

So what now??

The problem with complex taxonomies: The sexuality example

One of the most outstanding problems we have with Personetics is that we suffer a lot with the day to day complexities. We like to categorize quickly, it excites our heuristics. And our brain is designed on centuries of evolution with energy-saving in mind. We loathe escaping from our cookie-cutting wired recipes.

And one of the most needed and most neglected camps in Personetics is building complex taxonomies. We loathe complex taxonomies. And we loathe getting into complexities like context mapping, which are also absolute key in order to achieve the vision.

When we made the first contextual map at NH Hotels they couldn't make heads or tails of what contained. They were already a bit baffled that they had to manage over 40+ segments just for the leisure segment, but the fact that they had to manage context was just mind-blowing.

And the concept is pretty easy to grasp: The same person demands completely different experiences from hospitality depending on if they were on a business trip, on a romantic escapade or in a holiday trip with children. If you don`t have this fact into account, you are going to fall short when trying to be relevant. In order to be relevant, you have to be context-aware.

Problem is that complexity is not the beast to fight, but unnecessary complexity (As I already mentioned in my complexity rant here) and some problems that once had a cookie-cutter approach to its resolution, once closely examined, turned to be a much more thorny issue.

Let's get an example of a concept that was once clear and once faced with a reality check, turned out to be much muddier waters than expected.

Gender.

This is an excerpt from "Queer: A Graphic History" from Meg-John Barker and Julia Scheele

One area of trans that has gained wider attention only relatively recently is genderqueer, or non-binary gender. Both of these words have been used as umbrella terms for an explosion of descriptions of genders that fall between or beyond the gender binary (gender neutral, genderfluid, agender, pangender, and androgynous, to name just a few). This kind of gender explosion was exactly what Sandra Bem proposed was needed to dismantle gender inequality.

It seems likely that there will be similar tensions between queer theory and non-binary people as there are with bi and trans people. They disrupt the gender binary, but some may be seen as returning to identity politics in rights-based campaigns.

Get the picture?

The concept of being out of time

I am pretty conscious as well of the fact that there is a thing which could be at play on the overall issue: The concept that I call "What people can reasonably chew".

In order to make an example of what I am talking about, let's pick an example: Tesla and his remote control ship.

(Text is taken from "Tesla, live and legacy")

Tesla wanted an extraordinary way to demonstrate the potential of his system for wireless transmission of energy [radio]. In 1898, at an electrical exhibition in the recently completed Madison Square Garden, he made a demonstration of the world's first radio-controlled vessel. Everyone expected surprises from Tesla, but few were prepared for the sight of a small, odd-looking, iron-hulled boat scooting across an indoor pond (specially built for the display). The boat was equipped with, as Tesla described, "a borrowed mind."

"When first shown... it created a sensation such as no other invention of mine has ever produced," wrote Tesla. As happened fairly often with his inventions, many of those present were unsure how to react, whether to laugh or take flight. He had cleverly devised a means of putting the audience at ease, encouraging onlookers to ask questions of the boat. For instance, in response to the question "What is the cube root of 64?" lights on the boat flashed four times. In an era when only a handful of people knew about radio waves, some thought that Tesla was controlling the small ship with his mind. In actuality, he was sending signals to the mechanism using a small box with control levers on the side.

Tesla's U.S. patent number 613,809 describes the first device anywhere for wireless remote control. The working model, or "teleautomaton," responded to radio signals and was powered with an internal battery.

Tesla did not limit his method to boats, but generalized the invention's potential to include vehicles of any sort and mechanisms to be actuated for any purpose. He envisioned one operator or several operators simultaneously directing fifty or a hundred vessels or machines through differently tuned radio transmitters and receivers.

When a New York Times writer suggested that Tesla could make the boat submerge and carry dynamite as a weapon of war, the inventor himself exploded. Tesla quickly corrected the reporter: "You do not see there a wireless torpedo, you see there the first of a race of robots, mechanical men which will do the laborious work of the human race."

You can recognize a lot of what I call the "man out of time" pack. The awesome advanced vision, the mind-blowing of the laymen, the irritation for being misunderstood. Tesla's device was literally the birth of robotics, though he is seldom recognized for this accomplishment. The inventor was trained in electrical and mechanical engineering, and these skills merged beautifully in this remote-controlled boat. Unfortunately, the invention was so far ahead of its time that those who observed it could not imagine its practical applications.

I have mentioned a lot of times that The problem with futurism is always ETA (Estimated time of arrival). That's the kind of thing that you cannot foresee and there is a heavy amount of luck involved. Well, there is a couple of times that got nailed, and the results were awesome.

Let's go back to 1983, and to a relatively obscure computer games company in the UK, Called Ashby Computers and Graphics, or better known by the cognoscenti, Ultimate Play The Game.

They release in 1984 two games which were absolutely milestones in the development of the ZX Spectrum computer. First, they released Sabre Wulf, a beautiful adventure set on the jungle, which sold like hotcakes:

Then out of the blue, they launched in November 1984 the bombshell. Probably the single best ever ZX Spectrum game ever made, winner of every single Computer Award in 1984/5 and definitely, probably the single higher technical milestone of the platform (It was later probably surpassed by games like Head Over Heels, but the jump ahead and the jaw-dropping effect was never matched)

Please welcome to the Knight Lore.

Ok, where's the deal? The second game was made BEFORE the first and was consciously kept under wraps because they thought that it would prevent the first one from being sold. You can find the whole story here, and I find it completely fascinating, and a tribute of a completely lost art regarding how to determine the timeline of the state of the art of the industry.

Conclusion

The fact that personalization promise has not been even close to be fulfilled is directly connected with the fact that we have duly ignored all the warnings and set posts that we found in the way.

We decided to take the shortcuts, the easy data gathering, the basic profiling, the lackadaisical context management; the whole lazy student package. We basically tried to skimp on effort and as a result, personalization has been short of achieving anything of significance one time after another.

As we decided, time after time, to avoid entering into strange waters and leaving behind all the needed hard work to achieve truly relevance, the industry basically pulled a Chamberlain, when Industry needed to pull a Churchill.

“You were given the choice between war and dishonour. You chose dishonour, and you will have war.' - To Neville Chamberlain”

Again, we can fully understand that picking the wrong side of the force is the easy way, and water goes where less effort is needed. But, for Christ's sake, do not ride again the personalization horse based on false premises.

Maybe the whole problem is that nobody is really interested in making the most out of personalization because they couldn't find how to really do it with a map, but instead trying to sell expensive hardware and software all over again on the basis of a never reaching the promised land. And they use personalization as a McGuffin. Well, people, Caveat Emptor.

The good news is that the real demand for personalization has never been higher. The time arrow just points in a certain direction, and the direction is that it's relevance or bust. Hope this time we do not spare the effort needed to achieve the real McCoy.