New AI detection tool measures how “surprising” word choices are (www.freethink.com)
TLDR; Summary
- A team of researchers at the University of Maryland has developed an AI detection tool called Binoculars that can accurately identify over 90% of writing samples generated by AI.
- Binoculars uses two "lenses" to analyze text: an "Observer" LLM that measures the unpredictability of the text, and a "Performer" LLM that predicts the next word based on the previous words.
- If there is little difference in perplexity scores between the two LLMs, Binoculars predicts that the text was likely written by AI.
- The tool has a false-positive rate of just 0.01%, meaning it incorrectly flags one out of every 10,000 samples as AI-generated.
- However, schools may be hesitant to use the tool due to the risk of any false positives, even though the risk is lower than with other AI detection tools.
- The researchers believe Binoculars could be valuable for detecting AI-written content on websites and social media platforms.
- Teachers may