Edge Rewrite
pun Source code View original
Jump to content

Wikipedia:WikiProject AI Cleanup/Guide and resources

Add links
From Wikipedia, the free encyclopedia

Welcome to the guide on how to do AI cleanup. LLM-generated content (also referred to as "AI-generated content") on Wikipedia often violates several of Wikipedia's core content policies, especially WP:V because of large language models' tendency to hallucinate, and is therefore largely prohibited by WP:NOLLM. AI cleanup is typically done at the AI noticeboard (AINB), but is carried out across the wiki. This guide only consists of advice; if it feels overwhelming, don't worry, you can get by just fine by getting stuck-in and using common sense while learning from observing or asking others along the way.

Determining whether content is LLM-generated

[edit source]

The main method editors use to identify LLM-generated content is to identify stylistic or linguistic signs that are very commonly found in LLM-generated content but much less so in human-written content; these are documented at Wikipedia:Signs of AI writing (if AI cleanup is something you're interested in, this is the page to study). It is possible to determine whether content is LLM-generated or not by the existence of either an obvious sign (such as markdown or chatbot communication), or a combination of less obvious signs.[a] While such methods can produce high degrees of certainty, they are probabilistic, meaning editors should remain wary of false positives and alternative explanations. Detecting LLM-generated content is very much a skill to build; one 2025 study found that human ability to distinguish LLM text from human is generally no better than random chance,[1] while another 2025 study found that heavy users of LLMs can correctly determine whether an article was generated by AI about 90% of the time.[2] As of August 2026, a lot of what is documented at WP:AISIGNS has unfortunately become outdated for the most recent models, making our job harder for the time-being.

Although it is much more time-consuming than looking for AI signs, often the most reliable way to determine whether content is AI-generated is to check whether the sources support the text (WP:V, i.e. whether content has been hallucinated). It can be most efficient to prioritise checking sentences which contain AI signs.

It can sometimes be difficult to determine whether an individual edit used AI. Because LLMs are typically used habitually, it may be easier to look at the author's other edits around the same time (WikiBlame is a useful tool for identifying which user added a particular bit of content). It is usually most efficient to check other large additions, or the first revision of some of their creations (to do this, go to their contributions, scroll to the bottom and click "Articles created" to go to XTools, click the timestamps).

Content added before 30 November 2022 (the launch of ChatGPT) is extremely unlikely to be LLM-generated.

LLM detection software

[edit source]

Many AI detectors, such as GPTZero, are currently on the market. However, they often suffer from false positives, particularly free ones,[3] and should never be used as the sole evidence when accusing someone of AI usage.

These tools can suffer from both false positives and false negatives, and have a lack of transparency in how an "AI" score is calculated. Many sites now offer ways to "humanize" LLM text to attempt to fool these tools. Some human writers may have styles that naturally trigger the tool, and others may use writing plugins that improve their writing but also trigger the tool. The algorithms for such tools also may change, and are usually only free for a small amount of text, or will hide the results behind a paywall modal. However, such tools have improved over time and the best ones (usually non-free) have >95% accuracy rates,[4] and they can help check dozens of suspected edits for further inspection. A consistently high AI score could warrant additional investigation. Mixed scores are lower signal, since other editors may have changed the content substantially, and wiki formatting/markup can throw some tools off.

Useful tools

[edit source]
  • User:Sohom Datta/Link-dispenser – a tool that checks an article's external links for 404 errors (indicating they may have been hallucinated, it is worth doing a site search for the article subject to make sure)
  • Bibtest – a tool that checks whether an article's ISBNs exist
  • Edit filter logs – you can search for the user in edit filter logs to see if they triggered any ones relevant to LLM use (ones to look for include: 893, 1030, 1325, 1346, 1369, see for them all together)
  • GPTZero for Wikipedia – not that reliable, but consistently high AI scores (or a 100% AI score) is worth something
  • Pangram – one of the most reliable automated detectors (despite high false negatives), but it is better used as a sanity check and shouldn't be a substitute for personal judgement

Engaging with editors suspected of LLM use

[edit source]

Many editors who use LLMs do so in good-faith, and are often unaware of WP:NOLLM (User:LWG/These are the LLM users in your neighborhood discusses the different types of editors you may encounter). They are often newcomers who are overwhelmed by the steep learning curve for editing, or people using LLMs to compensate for a low proficiency in English. It's therefore paramount to be wary of WP:BITE. When encountering LLM-generated content, it's worth warning the author, whether with {{uw-ai}} or a 'hand-written' message (again, WikiBlame is useful for finding the author if it isn't clear from the page history). If you're unable to manage the cleanup of the user's edits yourself, consider making a report at the AI noticeboard (abbr. WP:AINB. Unlike at WP:ANI there's no need to notify an editor of a report about them). Warnings may sometimes be better framed as questions about whether they've used LLMs (especially if there's some uncertainty), as they're less combative/more collegial and may solicit some useful information that will help with cleanup.

For good-faith newcomers it is important to be patient and empathetic when explaining things, to point them towards guidance and possibly mentorship, and to obtain a commitment to not use LLMs in future, whether on their talk page or at the AI noticeboard (though often newcomers will stop editing due to this being a very negative early experience). You can also (kindly) ask them to help cleanup their previous edits. For non-native English speakers who have used LLMs, the same applies, but in addition it's worth communicating that content doesn't need to be written in perfect English (WP:IMPERFECT). Questioning someone's proficiency in English can be a sensitive topic; you may want to speak generally, and point them to a Wikipedia in their native language if they don't already edit there (Meta:List of Wikipedias). For editors whose proficiency in English is too low to contribute constructively, there is little you can do other than communicate clearly that they shouldn't edit wikis whose languages they don't speak. This'll usually become obvious because they continue to make AI-generated responses despite being told not to do so (see WP:AITALKSIGNS).

People generally don't respond well to their edits being reverted and accusations of LLM use. They may respond angrily, be evasive about whether they used LLMs, respond with AI-generated comments, attempt to wikilawyer their way around NOLLM, or straight-up lie (some users may be unaware they've been using LLMs, as is commonly the case with Grammarly or MS Copilot). In these instances, rather than spend time and energy arguing, it's usually best to explain things to them, and wait to see if they make further edits using LLMs (keep WP:ROPE in mind). If they continue to use LLMs after receiving thorough explanations, they can be blocked (whether at ANI or by using {{@AINBA}} at AINB). They may also go inactive. In both cases, you can peacefully proceed with cleanup. Some people may come from cultures that place emphasis on saving face, with this causing evasiveness; in these instances it's best to seek a commitment to not use LLMs in future, rather than seek a humiliating admission of past use (inline with WP:PREVENTATIVE).

It's usually best to be vague about how the content was determined to be LLM-generated; telling the editor about WP:AISIGNS may result in them attempting to mask or superficially fix the content (i.e. WP:BEANS).

Useful essays

[edit source]

Doing the cleanup

[edit source]

AI cleanup is typically done on a per-user basis, rather than on individual articles, because if someone uses LLMs once they're likely to use them habitually. Reviewing LLM-generated content and attempting to preserve it while ensuring WP:PAG compliance is time-consuming, and it's usually more realistic to remove such content by either stubbifying or rewriting from scratch. Cleanup is sometimes as simple as reverting the offending edit, or restoring an earlier version, though it becomes harder if the edit is old and there have been significant edits by other editors after it (even sometimes to it). Here you can manually remove content added by the user which is still live in the article, and replace it with that of an earlier version if pre-existing content was overwritten. Who Wrote That? is useful for highlighting content added by the user.[b] For some additions, or for when the prior version of the article is of poor quality, it's worth considering salvaging basic facts from the content.

Any pages (not just articles) that meet the criteria laid out at WP:G15 can be tagged for speedy deletion. LLM-generated creations can also be dealt with by draftifying and tagging with {{AI-generated}},[c] taking them to WP:AfD, or the presumptive removal process outlined below.

Presumptive removal

[edit source]

If a user has a history of LLM use, any editor can presumptively remove their edits (i.e. without reviewing each one) within a determined timeframe, inline with Wikipedia:Presumptive removal of AI-generated content. This is because of the asymmetry of effort between publishing LLM-generated content and cleaning it up; it would be extremely time-consuming to review each edit and creation so as to preserve as much as possible, compared to how little time it takes to generate them.

Presumptive reverting of edits (sometimes referred to as "LLMPRV") is the same process as discussed above, except you don't need to review the content set to be removed. For creations where the LLM-using editor is the only significant contributor and meets certain criteria, you can tag them for a special type of proposed deletion called "LLMPROD" wherein they are deleted by an admin after 5 or more days if no-one removes the tag. There is guidance on the process at WP:LLMPROD#Process, but it basically involves checking the page history for "significant contributions" from other editors and copy-pasting the tag and an edit summary (you may want to have a page like User:Kowal2701/LLMPROD to tab to, LLMPRODs haven't been added to Twinkle yet). What counts as a "significant contribution" is subjective, but this typically excludes things like copyediting or gnoming. Articles that do have significant contributions from others should be tagged with {{AI-generated}} instead. If you've LLMPRODed a large number of articles on a single topic (eg. 20+, you can see this in the WikiProject table on XTools, although not every article will have been tagged for WikiProjects), it's usually best practice to notify the relevant WikiProject(s) about them and ask if anyone wants to 'save' them from deletion. Often the hard part of presumptive removal is determining the timeframe during which the editor used LLMs.

If an editor is clearly good-faith and promises not to use LLMs in future, you can draftify and tag their creations instead of deleting so that they can work on them if they wish and go through AfC if necessary.

Places and resources for cleanup

[edit source]

Patrolling the wiki

[edit source]

There are many ways to search the wiki for LLM-generated content; see User:Gnomingstuff/Guide to finding AI-generated text for an in-depth guide.

Useful tools

[edit source]
[edit source]

Maintenance:

Discussions:

User warnings:

Articles for Creation:

Speedy deletion:

Proposed deletion:

Edit notice:

Administrative:

WikiProject AI Cleanup:

See also

[edit source]

Footnotes

[edit source]

Notes

[edit source]
  1. ↑ Although an AI sign such as the use of emdashes (WP:AIDASH) often appears in human-written content as well, the co-occurrence of this with several other AI signs rapidly decreases the likelihood the content was human-written. For example, if the likelihood of emdashes appearing in human-written content was, say, 20%, and that of, say, the words "pivotal" and "interplay" (WP:AIVOCAB) and over-attribution (WP:AIATTR) was 15% and 25% respectively, the likelihood of the content being human-written if these three signs appear simultaneously drops to only 0.75%.
  2. ↑ DiffUndo can be used to restore the pre-existing text (if any had been overwritten) while keeping edits after it intact. To use DiffUndo, go to the revision before the offending edit(s) by clicking its timestamp, click "Edit", click "Show changes", then undo all lines except those added by the user before clicking "Publish".
  3. ↑ User:MPGuy2824/MoveToDraft is useful because it automatically tags the mainspace redirect for speedy deletion (WP:R2), rather than you having to do it manually.

References

[edit source]
  1. ↑ Cheng, Adam; Lin, Yiqun; Reedy, Gabriel; Joseph, Christine; Wirkowski, Samantha; Mallette, Viviane; Nagesh, Vikhashni; Krieser, David; Calhoun, Aaron (2025). "Ability of AI detection tools and humans to accurately identify different forms of AI-generated written content". Advances in Simulation. 10 (1) 66. doi:10.1186/s41077-025-00396-6. PMC 12752165. PMID 41272826.
  2. ↑ Russell, Jenna; Karpinska, Marzena; Iyyer, Mohit (2025). People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Vienna, Austria: Association for Computational Linguistics. pp. 5342–5373. arXiv:2501.15654. doi:10.18653/v1/2025.acl-long.267. Archived from the original on 2025-08-29. Retrieved 2025-09-05 – via ACL Anthology.
  3. ↑ Fowler, Geoffrey (August 14, 2023). "What to do when you're accused of AI cheating". The Washington Post. Retrieved October 20, 2023.
  4. ↑ "People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text". arxiv.org. Retrieved 2025-11-28.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Wikipedia:WikiProject_AI_Cleanup/Guide_and_resources&oldid=1368421036"