Our take: IAPP AI Governance Conference

Next RegInt Episode

For the next episode we have something very special planned. Let's say it's going to be a special 'Special'. The episode will not go live, but will be published as a recording and will feature a very (you guessed it) special guest. Stay tuned, more info to come!??

Last episode: Our take on the IAPP AI Governance Conference

As always a big thank you to all those who joined us live. For those who didn't, as there is seldom a way to put pictures (or in this case a whole video recording) into words, you can always watch back the last Episode across our channels.??

Tea's AI News Rant - featuring some of the best (or worst?) AI flops of April

• Check out some of the weirdest new 'AI' products, including smart shoes and smart toothpaste dispensers
• New Google Gemini features (available in the US only) have been making sure that we all have plenty of solid nutrition and health advice, as well as have a solid understanding of US political history, make sure to stay informed by checking some of Gemini's greatest hits (those who have seen enough can also find out how to turn off the feature here)

Open AI has been (as always) up to a lot:

• First they dissolved their Superalignement team, which was supposed to be focusing on and using up to 20% of their computing resources for combating long-term AI risks; ten days later they formed a new Safety and Security Committee, that will advise OpenAI on safety issues while they are developing AGI, the Committee will be led by 4 directors, one of them is also Sam Altman
• Their latest model version has a voice feature, which they showcased in this video on Twitter/X (make sure to also watch the spin-off/parody of the video by The Daily Show, that one is much more amusing), this feature also got them into trouble with the Hollywood actress Scarlett Johansson whose voice might have been used to train the model even though she explicitly declined their request to provide her voice. In response, OpenAI published a blog post saying that over 400 individuals submitted their voices in order to be the new voice of OpenAI and that the one disputed by the actress actually comes from one of the submissions.
• OpenAI is now also publicly "exploring whether users should be allowed to create artificial intelligence-generated pornography and other explicit content with its products."
• Considering the previous three bullet points, we have very sad news coming in from Apple, as Apple is apparently partnering up with OpenAI in their efforts to compensate for "the big missing item." That item being a chatbot. This comes despite some of the top executives being "allergic to the idea of Apple going in that direction."

• Meanwhile, Elon plans to build a supercomputer to power the training of the next versions of Grok, which will now need over 100.000 H100 Nvidia GPUs.
• Elon also updated his prognoses for the arrival of AGI, which will apparently reach us (or we it?) already next year.

• Yann LeCun appears to be even more annoyed about it than Tea.

Not to be too depressed about the whole thing, there is still at least one CEO making reasonable public statements. You can read more on the one from Netflix co-CEO Ted Sarandos here. The most important part includes: "AI is not going to take your job. The person who uses AI well might take your job."

This time you again get two of Tea's personal hells, congratulations.

1. Eventbrite promoted illegal opioid sales to people searching for addiction recovery help and that's not all of it. They also advertised the sale of sensitive personal information like social security numbers as well as websites where one can find offers for “wild nights with independent escorts” in India. According to Wired's investigation, further 169 illicit events are being published daily.
2. China's latest response to US Generative AI efforts comes in the form of 'Chat XI PT'. The country’s newest LLM has been trained on “Xi Jinping Thought on Socialism with Chinese Characteristics for a New Era”, as well as other official literature provided by the Cyberspace Administration of China. At the moment, the model is still closed in the hands of CAC.

However, if there is still one person who can save the world that is of course the one and only Sam Altman, who has just signed the Giving Pledge together with his husband. Now, the Giving Pledge is a wonderful promise made by rich people (A.K.A. billioners, so like really rich people), that they will donate 40% of their fortunes to philanthropic purposes. Plot twist: the pledge is (of course) just a moral obligation.

You can read more on this wonderful moral obligation and all the ways it is failing in this critical piece published on the Mother Jones website. Some of the most disturbing bugs in the system include the Pledgers' fortunes massively incrementing despite their promise. The three biggest winners include:

1. Mark Zuckerberg and Priscilla Chan, whose fortune has increased 21.1 times, and
2. Elon Musk, whose fortune has increased by more than 9000% since he signed the Pledge in 2012.

AI Reading Recommendations

• Machine Unlearning in 2024 - get acquainted with the latest stance on machine unlearning methods, different available techniques, their advantages and disadvantages, etc. in this great overview written by Ken Liu from the Stanford AI Lab

• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet - a great research piece from Antropic on extracting various features from state-of-the-art transformers. "Features" refer to the specific pieces of information that the model uses to understand input and generate output. These features can encode any type of information and the researches have discovered that even safety relevant features can be extracted from these types of models. They also appear to be universal accessible and their extractions is possible in multiple languages.
• A Primer on the Inner Workings of Transformer-based Language Models - This new paper by Universitat de Catalunya , University of Groningen and Fundamental AI Research at Meta provides a concise technical introduction focusing on the generative decoder-only architecture. The paper introduces Transformer layer components, including the attention block and feedforward network block. If you want to know how to get from Italy to France via Activation Patching, have a look at figure 6, page 10. But we promise you, the other 54 pages are no less informative.
• What We Learned from a Year of Building with LLMs (Part I) - authors share some "crucial, yet often neglected, lessons and methodologies" that are, in their opinion and based on their experience, essential for developing products based on LLMs. They covered everything from prompting to RAG, and provide a very good read for advanced users. Maybe even a must read for anyone trying to use LLMs for serious.
• Contextual Position Encoding: Learning to Count What's Important - Standard position encoding methods such as Relative Position Encodings (PE) are based on token positions. In contrast, CoPE computes gate values conditioned on the context first, then uses that to assign positions to tokens using a cumulative sum. This allows positions to be contextualized, and represent the count of different units like words, verbs or sentences. CoPE operates on each attention head and so can attend to different position types on each. This is indeed an improvement, albeit still a computationally expensive one.
• As an AI Language Model, "Yes I Would Recommend Calling the Police'': Norm Inconsistency in LLM Decision-Making - Princeton and MIT researchers investigate norm inconsistency, where LLMs apply different norms in similar situations. Specifically, they focus on the high-risk application of deciding whether to call the police in Amazon Ring home surveillance videos. They evaluate the decisions of three state-of-the-art LLMs – GPT-4, Gemini 1.0, and Claude 3 Sonnet – in relation to the activities portrayed in the videos, the subjects’ skin-tone and gender, and the characteristics of the neighborhoods where the videos were recorded. Their analysis reveals significant norm inconsistencies: (1) a discordance between the recommendation to call the police and the actual presence of criminal activity, and (2) biases influenced by the racial demographics of the neighborhoods. These results highlight the arbitrariness of model decisions in the surveillance context and the limitations of current bias detection and mitigation strategies in normative decision-making.
• What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions - An LLM is nothing without its training data. But…how (much) does each data contribute to LLM outputs? In their paper, the researchers develop algorithms, theory, and software for LLM-scale data valuation/attribution influence functions that estimate how the model output would change if they add/remove a specific example from the training dataset. In their words: “Training data has become one of the most sensitive topics in the GenAI era, exemplified by recent lawsuits involving major tech companies. There are lots of data-related sociotechnical problems to be discussed, and we hope our work is a small contribution to it!”
• DE-COP: Detecting Copyrighted Content in Language Models Training Data - The authors propose a new method for detecting textual copyrighted content in LLM training data called DE-COP. A method to determine whether a piece of copyrighted content was included in training, by probing an LLM with multiple-choice questions, including both verbatim text and paraphrases. They also construct BookTection, a benchmark with excerpts from 165 books along with their paraphrases. DE-COP achieves an average accuracy of 72% for detecting suspect books on fully black-box models where prior methods give ≈ 4% accuracy
• Model AI Governance Framework for Generative AI, Fostering a Trusted Ecosystem - This is a very smart and sophisticated AI policy document by the AI Verify Foundation from Singapore. Its not about AGI science fiction or talking about transformative power of LLMs, it rather references real science, recognized technical methods and industry practices (e.g. RLHF, benchmarking, QA). Their approach to AI accountability is also convincing and places all eyes on the Developers (or as we say in Europe, Providers/Manufacturers) because of the amount of manual labour that goes into AI.
• Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? - authors provide multiple insights on fine-tuning dynamics, with the following key findings: (1) Acquiring new knowledge via supervised fine-tuning is significantely slower when not consistent with pre-existing knowledge. (2) LLMs struggle to integrate new knowledge through fine-tuning, this in turn increases their tendency to hallucinate. Taken together, their results highlight the risk of introducing new factual knowledge through fine-tuning, and support the view that large language models mostly acquire factual knowledge through pre-training, whereas finetuning teaches them to use it more efficiently.
• SignLLM: Sign Languages Production Large Language Models - Researchers unveiled SignLLM, a groundbreaking multilingual Sign Language Production (SLP) AI model that creates avatar videos of sign language gestures from prompts in eight different languages. The team began by developing Prompt2Sign, a specialized multilingual dataset fine-tuned for training upper-body movements in sign language. Using this dataset, they trained the SignLLM AI model. The model features two modes that allow it to generate avatar videos in multiple languages from simple prompts.

Our take: IAPP AI Governance Conference

Tea had a lot of fun testing out her hypotheses on the contents of the conference. You can check out her conclusions and a brief personal reflection in her LinkedIn post.