"Has Barbie gone to Los Alamos?"

"Has Barbie gone to Los Alamos?"

If you are old like me you may remember the "Sokal affair" where, as an experiment, in 1996 a physicist deliberately managed to get a 100% fake paper past the peer-review system and published. In the wake of this event fake paper generators appeared on the Internet; indeed "SCIgen - An Automatic CS Paper Generator" is still online. However, this pales in comparison to what could be coming thanks to GenAI agents. These agents can take care of the entire end-to-end publication process; as an example see the Systematic Review Automation System which "...handles paper search and selection (up to 3 papers), PDF processing, and generates a complete academic paper with all standard sections (abstract, introduction, methods, results, conclusions, references)". Indeed a recent search of arXiv (which until 2001 was known as the Los Alamos e-Print Server) has strong indications that such GenAI agents are indeed being deployed to create fake papers.

A flood of such papers could well have the potential to completely undermine academic research (see the recent Science piece "‘Systematic reviews’ that aim to extract broad conclusions from many studies are in peril") as researchers will have a priori no idea which papers to trust or not; fake papers eventually forming the majority of manuscripts as there is effectively no upper limit to the Gen-AI agent output rate. ArXiv is just the testbed; what will happen when LLM generated research papers become `photorealistic' and convincing enough get past the already stressed peer-review system?

?Is Barbie at Los Alamos, and has her finger on the button that could nuke all academic research in the (very) near future?

?? Alastair Muir, PhD, BSc, BEd, MBB

Data Science Consultant | @alastairmuir.bsky.social | Risk Analysis and Optimization | Causal Inference

3 个月

My first science project was on “poly water”. ‘Nuff said

Julien Borrel

Founder @ INJEN.IO - Machine Learning Studio

3 个月

A DDOS flooding attack on Arxiv ? As a layman I don’t know how Arxiv really works; can anyone publish any ??paper?? or there is some kind of filter ?

要查看或添加评论,请登录

社区洞察

其他会员也浏览了