How to Tell if a Paper Was Written by AI

From Claude, ChatGPT, and Gemini (Good document to download and save)

Provided by Prof Joseph Barnes, Albers School of Business & Economics, Seattle University

Who(?) better than AI to ask how to tell if a document was written by AI, so I asked Claude, ChatGPT, and Gemini for insights.

It's important to understand that AI detection is an evolving field, and no single keyword or phrase guarantees a piece of writing was generated by AI. However, certain patterns and tendencies can raise red flags.

Here's a breakdown of indicators, focusing on keywords and broader stylistic characteristics:

Check for Consistent Tone and Style: AI writing often has a?polished but somewhat generic feel, with:

• Sentences that are grammatically correct but lack a strong personal voice.
• Repetition of certain phrases like "It is important to note that..." or "In conclusion..."
• A formal, neutral tone — even in papers that might benefit from creativity or strong personal perspective.

Analyze Depth of Insight: AI can summarize facts well but often struggles with:

• Original thought?or?deep critical analysis.
• Personal reflection, lived experience, or creative analogies.
• Human papers tend to have more?nuance, personal touches, and insights that come from curiosity or independent thinking.

Keywords and Phrases:

• Overuse of "sophisticated" vocabulary: Words like "intricate," "nuanced," "delve," "tapestry," "multifaceted," and "pivotal" are often overused by AI. This can create a sense of overly formal or unnecessarily complex language.
• Formulaic transitions: Excessive use of phrases like "furthermore," "in conclusion," "moreover," and "therefore" can indicate a lack of natural flow.
• Repetitive phrasing: AI models sometimes repeat certain phrases or sentence structures, even when it sounds unnatural.
• "Academic" sounding filler: Phrases that sound like they are used to pad out a paper, without adding substance.

Stylistic Characteristics:

• Lack of personality and emotion: AI-generated text often lacks the subtle nuances of human expression, such as humor, sarcasm, or personal anecdotes.
• Inconsistencies in tone: The tone of the writing might shift abruptly, indicating that different sections were generated independently.
• Logical inconsistencies or factual errors: AI models can sometimes generate plausible sounding but inaccurate information.
• Predictable sentence structure: AI-generated text can sometimes have a predictable and repetitive sentence structure, lacking the variations found in human writing.
• Overly formal or generic language: AI can sometimes produce text that is very generalized and lacks a specific voice.
• Odd use of bolding: AI sometimes bolds random key words, as if to emphasize search engine optimization.

Pay Attention to Over-Explanation

AI writing often over-explains simple points, filling space with unnecessary elaboration (because AI is trained to meet word counts rather than write concisely). Example:

"The Industrial Revolution was a time of great change, affecting society, the economy, and technology. This period of transformation significantly altered how people lived and worked in ways that are still relevant today." A human might say: "The Industrial Revolution transformed work and daily life."

Check Revision History (if available)

If you have access to a Google Doc history or similar, you might notice:

• Human papers evolve with?edits, revisions, and changes.
• AI papers are often pasted all at once, with?minimal editing.

Assess Alignment with the Writer’s Past Work

• If the writing quality or style is?radically different?from previous work, that’s a flag.
• Example: Someone who struggled with grammar before suddenly turns in a flawlessly written, highly polished research paper. I usually compare someone’s writing on a paper to their writing on emails.

Look for Surface-Level Sources or Citations: AI can generate citations, but often they are:

• Fake (hallucinated)?or slightly altered versions of real sources. Check the sources to see if they are valid.
• Overly dependent on?general knowledge?instead of up-to-date or niche academic sources.
• Missing proper formatting (APA, MLA, etc.).

Cover Letters and Resumes: Look for:

Generic Phrases

• "I am writing to express my strong interest in..."
• "I am confident that my skills and experience make me the perfect candidate..."
• "I am excited about the opportunity to bring my skills to your esteemed organization..."
• "I am a highly motivated, detail-oriented professional with excellent communication skills..."

Overused Descriptors

• Excessive use of adjectives like "passionate," "dedicated," "innovative," and "strategic" without specific examples
• Strings of buzzwords ("synergy," "leverage," "optimize," "facilitate")
• Generic skill lists without context or examples

Awkward Formulations

• Unnaturally formal language throughout ("I would be most appreciative if you would consider my application")
• Inconsistent use of industry jargon (too much in some places, missing in others)
• Oddly sophisticated vocabulary mixed with simpler language

Structural Giveaways

• Perfect paragraph lengths that are too similar throughout
• Robotic transitions between sections ("Furthermore," "In addition," "Moreover")
• Identical formatting of job achievements ("Increased X by Y%," "Managed Z resulting in...")

Content Issues

• Vague accomplishments without specific metrics or context
• Claims that don't align with the timeline of employment
• Experiences that seem disconnected from the career path
• Suspiciously perfect alignment with job requirements without nuance

The strongest indicator is usually the overall impression of generic content that could apply to almost any job rather than showing genuine personality and specific, relevant experience.

In summary:

Here are key indicators that can help you spot AI-generated writing:

Content Patterns

• Repetitive phrasing?- AI often reuses similar sentence structures or transitions
• Generic language?- Overuse of common phrases rather than unique expressions
• Inconsistent expertise?- May sound knowledgeable in one paragraph but superficial in the next
• Lack of nuance?- Simplified treatment of complex topics without meaningful depth

Structural Elements

• Formulaic structure?- Follows predictable patterns in paragraph organization
• Perfect formatting?- Unnaturally consistent adherence to requested formats
• Awkward transitions?- Logical leaps between sections without proper connections
• Excessive hedging?- Overuse of phrases like "it may be the case that" to avoid definitive claims

Technical Indicators

• Citation issues?- Made-up references or citations that don't exist when checked
• Factual errors?- Confident statements of incorrect information ("hallucinations")
• Timestamp inconsistencies?- References to events that don't align with supposed writing date
• Unusual word choices?- Occasionally uses rare words that seem out of place

Have an Oral Conversation About the Paper

Genuine writers can usually discuss their process comfortably. If answers feel vague or overly generic, that’s another clue.

Ask the writer directly:

• "What was your process for writing this?"
• "Which part of the paper was the hardest to write?"
• "What sources did you find most helpful?"

Detection Tools

Some tools can scan papers and give you a?probability score?that the text was AI-generated. These include:

• GPTZero?(popular for educators)
• Copyleaks
• Turnitin (AI detection built-in)

Remember that detection methods aren't perfect, and sophisticated AI can sometimes evade basic detection. A combination of these indicators usually provides stronger evidence than any single sign.

Important Considerations:

• AI detection tools are not foolproof. They can produce false positives and false negatives.
• Human writers can also use these keywords and phrases, so context is crucial.
• AI technology is constantly evolving, making it increasingly difficult to detect AI-generated content.

By combining careful analysis of keywords, stylistic characteristics, and the overall context of the writing, you can increase your chances of identifying AI-generated content.

?