Tech Time by Tim #64

COP 28 started this past week and whilst I expect to be disappointed in the end, I’m hopeful to be proven wrong. Researchers also found the rewind button on ChatGPT, making it throw up all kinds of ingested data and instructions. Not quite the way I expected to see ‘how the sausage was made…’ If you’d prefer a somewhat less messy look behind the scenes, why not visit Apple’s chip lab? After that, we’ve also got some predicted economies and predictable economics, brand identity politics, and last but not least, a showing of Ad Sense and Sensibilities.

At A Glance:

A quick overview of this week’s content.

·?????? The Week That Was: COP(out) 28 Could Be Great, Token Troubles And Word Vomit

·?????? The Business End: Is That Sausage Made To Military Standards?, Predicted Economies And Predictable Economics

·?????? Rules of Engagement: Brand Identity Politics, Ad Sense And Sensibilities

The Week that Was:

A look back at the tech world of the past week.

COP(out) 28 Could Be Great:

The Conference of the Parties of the United Nations Climate Change Conference, mercifully abbreviated as COP 28 (because it’s the 28th year of them accomplishing bugger all), is being held in Dubai this year, and like... Why? Money, obviously, but really? The United Arab Emirates (UAE) is one of the largest fossil fuel economies and suppliers in the world. Cynically speaking, some have referred to COP as more of a fossil fuel trade fair than a climate summit. So yeah, in that regard Dubai makes all the sense in the world as host. But there’s more to it than that, there’s some positive potential here to be examined as well.

The UAE is also a desert nation, and climate change is now coming for the rich too. Herein lies the positive potential for change. That is if we assume the allegations are true of the UAE trying to drum up support for its products amongst poorer and developing nations. These new customers would replace wealthier nations wanting to distance themselves from fossil fuels. In such a scenario those that stand to be most dramatically impacted by climate change would have leverage, far more than they’ve historically had. That’s why COP 28 can be considered a crunch point for those countries on the front lines of climate change.

If used properly, this leverage could be the breakthrough that climate tech needs to truly succeed. Climate tech startups come and go, generally enjoying waves of hype and popular support until they become large enough that fossil fuel executives get nervous and their lobbyists get to work. And here we are once again, climate tech is on the rise and 2024 looks to be a make-or-break year for it in more ways than one. So, once again we find ourselves asking the question of whether or not a breakout climate tech success will occur or not, and even if it does, whether this will be a necessary relief, or a case of ‘too little, too late.’ I expect to be disappointed by COP 28, but I hope to be pleasantly surprised. LINK

Token Troubles And Word Vomit:

Have you ever wondered what would happen if you told ChatGPT to repeat a single word or phrase endlessly? A team of researchers did, and they’ve just published the results. Now, before you try to verify this flaw for yourself, I’m afraid I must disappoint you. The flaw has already been patched and new attempts will result in a warning that such prompts are against the terms of service. But assuming you used this trick before it was patched, you might have been surprised to get some login credentials or confidential information spat out at you.

Text-based generative AI relies on tokens. These are markers of sorts that allow the machine learning model to play a highly complex game of connecting the dots, ultimately giving you the content it thinks you want. Even the most powerful systems have a limit to the number of total tokens they can effectively process and remember. An infinite prompt will break this limit. The result of that was, effectively, ChatGPT getting sick and vomiting up its proverbial stomach contents. Curiously, the original prompt did seem to sometimes affect what kind of information ChatGPT ended up uh… regurgitating.

Now, who would like to guess what kind of implications that has for GPTs, the custom versions of ChatGPT that users can train and release? Those who guessed something in the ballpark of ‘it could give users of a GPT access to the training data and unique prompts the GPT’s maker doesn’t want them to see,’ would be weirdly specific, and also entirely correct.

It feels like every company, even freaking Doritos, is trying to find a way to integrate AI into their products these days. Companies such as Hugging Face are popular partners for such efforts. ?Unfortunately, Hugging Face, apparently has a pretty shit poker face, because some other researchers read it like an open book. Thus the principles behind scalable data extraction feel like they are less of an individual quirk of ChatGPT and more of a systemic flaw with generative AI, the way it's trained, refined, and tuned. I’m curious whether or not these new findings will impact the ongoing generative AI court cases or not. LINK

The Business End:

Coverage of big business moves and events in the tech world.

Is That Sausage Made To Military Standards?

There’s this saying in English that goes a little something like “take a look at how the sausage is made.” This saying refers to the background processes that go into making things that we, as consumers, easily take for granted. This past week we got a look inside Apple’s chip lab, which I found to be interesting. So how is that sausage made? Well, there are two parts actually! The hardware aspects, with all the actual secrets scrubbed out, are what we get to see during the tour. Another unspoken part is the tense partnership between Apple and Arm, the company that licenses the instruction set that Apple uses for its current M-series chips. Despite how critical this instruction set is, Arm only sees about 5% of its total revenue come from the partnership.

I myself have an M series device, and I’ve also got quite a bit of OCD about keeping my devices in ideal condition. I tend to get cases for them that can be a bit overkill. Manufacturers of such cases are particularly fond of claiming that their materials are up to ‘military standards.’ The logic goes that if these materials are good enough for the extreme situations that soldiers find themselves in, surely they are more than a match for mere trouser pockets and bags, right? Except militaries, quite like corporations, tend to seek minimum costs for maximum yields. This can lead to unsanitary situations for personnel, all in pursuit of pruning undesirable characteristics from the so-called ‘boots on the ground.’

However, desires and practical realities tend to differ. And that’s what’s truly interesting about behind-the-scenes looks, they allow you to better gauge just what actually goes into making stuff, or I guess, organizing stuff. Or in the case of logistics, both. When manufacturing and logistics collide, the situation becomes even more complicated. We even have an example of that to look at. As you see, in the same week as the Apple chip lab tour, we also got a look at the present state of Apple’s key manufacturing partner Foxconn. Foxconn is in the process of moving operations out of China, into India. Sausage making, as it turns out, can be quite a messy process indeed. LINK

Predicted Economies And Predictable Economics:

Neither Apple nor Google are particularly keen on naming ChatGPT their app of the year. They don’t want to highlight the fact that their algorithmic pants got pulled down by Microsoft’s chaotic fledgling in front of the entire class. And what a year it’s been. We’re in December now and it feels like the year just flew by. It’s already that time again when everyone starts giving their predictions for the next year, and so too are various analysts weighing in on what Season Two of ‘Captain GPT’s Wild Ride’ might have in store for us. ?

Especially after the dramatic events of the OpenAI coup, many eyes are still on this particular niche of the tech world. All the more reason to zoom out a little, to get a big-picture view of the wider economic circumstances in which the next phase of AI development may or may not happen. That’s important for two reasons:

1)???? Streaming service Spotify’s ongoing monetization struggles show us that even dominant players in a lucrative space aren’t automatically profitable. In 2023 investors began running out of patience with regard to profitability, a trend I only see accelerating over the course of 2024

2)???? Generative AI is extremely resource, capital, and power-hungry, so to truly gauge where it’ll end up in 2024, it is good to examine the economic forecasts for major economic blocks around the world.

A lot of news sources in tech tend to be American, so here, for your convenience, are three calls for China, the Eurozone, and the wider Asian region for the sake of a clearer overall picture. At least at Dutch bank ING, there’s cautious optimism for the overall state of commodities in 2024. Commodities include raw materials such as metals, energy, and agricultural products. All of these greatly influence demand and pricing for everything from critical infrastructure to luxury products. Commodities aren’t nearly as sexy as the products they power, such as generative AI. Pay attention anyway, because without raw materials, no one in tech is accomplishing much of anything. LINK

Rules of Engagement:

Ethics and legal matters regarding tech engagement.

Brand Identity Politics:

Generally speaking, the saying ‘divide and conquer’ refers to splitting up one’s enemies to better pick off the individually weaker opposition one at a time. However, I’ve noticed that several companies in both the social media and gaming sectors are instead looking to divide themselves up to conquer new market segments. Two notable efforts include Microsoft’s preparations for another stab at running a mobile app store, whilst TikTok is continuing its efforts to enter the e-commerce and streaming spaces. Other moves include newsletter platform Substack gearing up to fully support video content, as well as messaging service Discord’s ongoing rollout of premium profile customization features.? Naturally, a lot of User interface (UI) elements will be necessary to facilitate these new features.

Those active in the design-oriented corners of the tech world will be familiar with ‘flat design.’ As apps grow in scope and complexity, we appear to be witnessing a gradual move away from flat design elements and towards more visually distinct design languages. For those who don’t know what flat design is, this is a style that focuses on simple, straightforward icons and menus with minimal visual flourishes. Flat design is usually lighter on device performance and easier for new users to understand because it’s simpler and more basic. Flat design excels at bridging the gap between brands because of how relatively uniform it is across ecosystems. Great for onboarding less tech-savvy new users. However, the notable disadvantage of flat design is that it looks, well… flat, and makes it harder to distinguish yourself from the competition.

Brand identity, or so I feel, is why Valve feels comfortable launching a steam link on Meta’s VR headsets. It is also why Microsoft is now walking back previous claims that it’d try to get Xbox game passes onto rival platforms such as PlayStation and whatever Nintendo ends up calling its next console. Brand identity used to be something worth diminishing for the sake of market share. You wanted to be similar to rivals to make it easier for users to switch over. Now that new battle lines have been drawn, and new moats are being dug, I expect brand identity, and the accompanying design languages to gain prominence again, as can be seen from the change in course by Xbox with regard to game pass availability. LINK

Ad Sense and Sensibility:

Google’s search partner network has come under fire (again) for issues of platform safety. Talk of fuelling sanctioned websites and multi-billion dollar holes in the search partner program certainly isn’t a good look. Yet none of this is happening in a vacuum. The heightened scrutiny comes amidst wider double takes with regard to how ad tech-fuelled platforms influence our lives and societies.

Calls from within the ad tech industry itself are growing louder and vanity metrics should disappear. These metrics were long a driver of ad revenue. There’s also a surge of new research coming out with claims such as how contemporary dating platforms are popularity contests more than they are real match-making tools. Quite like social media toxicity, it would seem. Those making hateful content or comments on the internet appear to feel validated by receiving likes, prompting them to produce more toxicity.

Relationships are complicated in both business and in love, with some just not meant to be. Goldman Sachs, a prominent US bank is finding this out the hard way, as it isn’t just going through a messy divorce with Apple, employees are also choosing to break up with it as poachers from AI companies near and far are out looking to find their match. Again, none of these events are happening in a vacuum.

The history of Ad tech is an important context for understanding the bulk of these seemingly strange and unrelated business decisions. Back in the 1990’s, it was ad tech that first alerted companies to the overwhelming possibilities and advantages that data could yield to them. Ad tech was initially a means to an end, a tool, yet it proved so financially lucrative that everyone from search engine providers to banks got carried away. I believe that what we are seeing now, is a reality check of sorts, during which the limits and shelf-life of a temporary money well are becoming readily apparent. History doesn’t repeat, but it does rhyme, so if you want to be ready for what might come next, perhaps take a look back in time. LINK

A Nice Cup of Serendipity:

Cool bits and bobs from around the web.

Tyrian Purple LINK

Deep Sea Fish LINK

Anthrobots LINK

Scorpions LINK

Android Kill Switch LINK

Lawsuit Diffusion LINK

ASML Inheritance LINK

Q Hallucinations LINK

Geothermal Power LINK

It Got Worse! LINK

DNA Storage LINK

Digital Currency LINK

The Deep End:

A weekly batch of long-form content recommendations.

Invisible Demons:

A rather dramatic title for a newly observed physics phenomenon. LINK

Scalable Extraction Of Training Data From Language Models:

The paper covered in ‘Token Troubles And Word Vomit.’ LINK

Plagiarism and You(Tube):

A video essay about plagiarism with several in-depth case studies. LINK

One More Thing…

Some more housekeeping! As stated in last week’s newsletter, there will be one more normal newsletter after this one. The week after that, you’ll instead be treated to a special edition newsletter called ‘The Year That Was.’ Since this will be my first full year of doing the newsletter, I plan to gather a few of the most important items from previous newsletters throughout the year, with some added commentary and relevant links included. The final week of December will have no newsletter because we’ll all be busy with Christmas and New Year’s celebrations anyway. Speaking of celebrations, Sinterklaas happened in the Netherlands on December 5th, and it was lovely. For those who don’t know, this is a figure based around Saint Nicholas, the patron saint of children. It’s kind of like Christmas, but uniquely Dutch in the finer details.

I had a comfy evening with my family. We didn’t do many gifts, because we already had most of the stuff we wanted. I suppose that in and of itself is a greater gift than anything material could be in the grand scheme of things. I’ve also been trying to read books again in between my newsletter work. It’s hard to find the time since I usually parse around 5000+ articles per week for the newsletter. Yet finding time is quite like eating so-called ‘pepernoten’ which are small, spiced biscuits unique to the Sinterklaas period. Much like pepernoten, you can read a book like a mental snack, then when you’ve had your fill, put it down again for next time. Careful with that food for thought though, if you overeat you might end up feeling sick rather than inspired, all things in moderation friends!