🚫 Bye-Bye Junk Reviews? arXiv might react on LLM spam

[root]

It’s 2025, and arXiv, the beloved preprint server of the CS community, is facing a new kind of flood: a torrent of AI-generated review papers. The deluge has become so intense that arXiv moderators are seriously contemplating a controversial policy shift: only accepting review articles that have gone through peer review.

The AI Citation Scheme Invasion

Here’s what’s happening: With the rise of large language models (LLMs), it’s never been easier to churn out a polished-looking review paper. But along with this convenience has come a plague of citation farming — papers that exist primarily to game citation counts, either by over-citing the authors’ own work or linking to random irrelevant papers in what smells suspiciously like “citation-for-hire” schemes.

The result? arXiv moderators are overwhelmed. As they put it:

“We are stuck between a large amount of junk and a small amount of high-quality reviews.”

And because it’s so difficult to distinguish between genuinely helpful review papers and LLM-spam masquerading as scholarship, the editors are thinking of swinging the hammer: ban all review papers unless they’ve been peer-reviewed.

Collateral Damage

That sounds reasonable, right? Until you realize that high-quality, timely reviews, often collaborative efforts across institutions, are now getting caught in the crossfire.

One case that set off this firestorm was a review co-authored by researchers from 20+ top universities. It was “on hold” and potentially rejected, simply because it fell into the new gray area. The authors even contacted arXiv moderators for clarification, only to be told: there’s just too much junk to sort through.

The researchers’ frustration spilled over into Xiaohongshu (China’s Instagram-meets-Reddit), where the author “寒月灼华” summarized the mess:

“There are just too many LLM-generated, citation-spamming review papers. It's hard for moderators to judge what’s good or bad, and good ones are getting mistakenly blocked.”

The Bigger Questions

• Is arXiv still an open-access archive, or has it become a quality filter?
• Should LLMs be banned from generating reviews, or do we just need better detection tools?
• And more philosophically… is the concept of a review paper still relevant in the LLM era?

Here’s what the community had to say:

• "LLM should be used to detect other LLMs."
• "Review papers are just like directories now. Useful, but only if curated carefully."
• "The real solution? Know someone on the moderation team."
• "Deep Research reports are better than most reviews I read."

Ouch!

🧠 Are Review/Survey Papers Still Valuable?

There’s a divide in opinion. Some say review papers are obsolete in the age of ChatGPT and Deep Research. Others argue they’re more important than ever as entry points for cross-disciplinary learning.

“For someone doing LLM work, it's essential to quickly grasp what's happening in NLP, CV, RL, and Sys. A good review serves as a map.”

So, What Now?

The situation remains fluid. If arXiv enforces this new policy, we might see:

• A shift toward submitting review papers to journals like TMLR, Survey & Tutorials, or even Medium (kidding... kinda).
• More reviews being peer-reviewed by design, possibly lowering the bar for journal acceptance.
• New tools for LLM-detection being integrated directly into submission platforms.
• And a lot more whispered “do you know an editor?” backchanneling.

In short, we’re in the middle of an arms race: junk vs. quality, automation vs. moderation, speed vs. scrutiny.

Until then, if you’re planning to post a review on arXiv, maybe wait until you’ve got that Reviewer #2 approval in hand. Otherwise, your magnum opus might just get mistaken for an LLM’s 15-minute side project.

---

🧵 Have thoughts? Should arXiv be stricter with reviews? Or are they throwing out the baby with the bathwater? Let us know on Twitter, Mastodon, or your nearest hallway track.