Algorithms of Alienation: Facebook, AI Spam, and the Erosion of Community

?

Recent research [1] from Stanford and Georgetown University reveals a troubling trend on Facebook: the platform's recommendation algorithms are increasingly promoting strange AI-generated images crafted by spammers and scammers. These deceptive visuals captivate an audience that often fails to recognize their artificial origin, leading users to off-platform sites cluttered with ads and even facilitating the sale of low-quality products [2].

?

According to Cory Doctorow this AI slop (a sort of AI-generated spam) is “the laziest, grossest engagement bait” that has turned Facebook into an “anaerobic lagoon of botshit” [3].

?

How Does It Work?

Simply by Facebook’s recommendation algorithm [4] rewarding such AI-generated spam pages that are posting dozens of times a day. And because AI slop works, “increasingly outlandish things are going viral and are then being recommended to the people who interact with them. Some of the pages which originally seemed to have no purpose other than to amass many followers have since pivoted to driving traffic to webpages that are uniformly littered with ads and themselves are sometimes AI-generated, or to sites that are selling cheap products or outright scams” [2].

?

Why Should We Care?

·?????? Quality Degradation: The proliferation of such content further degrades the quality of our feeds, Flooding feeds with meaningless or deceptive posts is degrading the overall user experience.

·?????? A vicious cycle driven by monetization of misinformation: Spammers profit from ad revenues and product sales generated through this engagement bait, incentivizing the creation of more AI slop.

·?????? User Manipulation: Vulnerable users who may not discern that the content is AI-generated are particularly at risk, potentially falling prey to scams or spreading misinformation unknowingly.

?

Now really, why SHOULD we care?

Take for instance this latest AI slop on Facebook exploiting suburban fear: https://www.usermag.co/p/my-neighbors-ridiculous-reason-for

Such exaggerated narratives do a lot more than just driving traffic to ad-laden, low-quality external websites (though widespread visibility achieved by manipulating social media algorithms designed to reward engaging content). They have broader societal implications, beyond algorithmic manipulation and economic exploitation.

?

·???? Societal Impact: This phenomenon feeds into a larger societal issue of atomization, where people increasingly view themselves as isolated individuals rather than part of a broader community. Doctorow argues [3] that this environment is conducive to extreme wealth inequality because it undermines collective action and solidarity, limiting responses to societal issues to individual actions rather than coordinated efforts.

·???? Distrust and Mistrust: The type of promoted content feeds into a cycle of fear and distrust among individuals, amplifying societal divisions and reinforcing a worldview where community and societal structures are devalued. This erosion of trust has significant implications for how people interact with and perceive one another, fostering a climate of suspicion and self-reliance that is opposed to communal support and cooperation.

·???? Ideological Reinforcement: Platforms inadvertently or deliberately reinforce existing societal ideologies driven by material conditions, showing us versions of society that may be skewed or incomplete but resonate with and exacerbate existing fears and tensions.

?

The overall effect of these dynamics have long-term cultural and economic consequences, promoting a culture that is increasingly shaped by commercial interests and individual consumer actions, rather than collective decision-making or public solutions. This affects everything from environmental policies to healthcare and welfare.

?

The real problem of the manipulation of social media platforms for spam and profit is not a technological or economic issue, but a deeply social one as it reflects and reinforces the worst aspects of a fragmented, individualistic society. This atomization poses a significant challenge as it makes our societies more vulnerable to inequality and less capable of collective action, which is often necessary to address large-scale issues such as climate change or social justice.

?

Let's all take responsibility for the content we consume and share!

I encourage you to learn more about the mechanisms behind content recommendation algorithms and to become an advocate for digital literacy in your community.

?·?????? How can we, as informed users, push for greater transparency and accountability from social media platforms? Let's discuss and act!

·?????? Have you ever been misled by sensational content on social media? Share your story in the comments below.

?

----------------------------------------------

Sources:

1.?????? How Spammers, Scammers and Creators Leverage AI-Generated Images on Facebook for Audience Growth: https://cyber.fsi.stanford.edu/io/news/ai-spam-accounts-build-followers

2.?????? A torrent of “shrimp Jesus” and fake chainsaw sculptures: https://www.404media.co/email/1cdf7620-2e2f-4450-9cd9-e041f4f0c27f/

3.?????? Conspiratorialism as a material phenomenon: https://doctorow.medium.com/https-pluralistic-net-2024-10-29-hobbesian-slop-cui-bono-beca9ca3d4e7

4.?????? Behind the curtain of engaging but misleading content on social media lies the power of Artificial Intelligence, particularly through models like Generative Adversarial Networks (GANs). GANs consist of two parts: a 'generator' that creates images and texts, and a 'discriminator' that evaluates their authenticity. The generator produces new data samples, while the discriminator assesses them against real data, teaching the generator to improve its outputs continuously. This iterative process enables AI to produce highly engaging and realistic content that can be difficult to distinguish from content created by humans. The sophistication of these models makes them effective tools for creating viral, eye-catching spam that captures user attention on platforms like Facebook.