Unfated.AI Logo

New AI detection tool measures how “surprising” word choices are (www.freethink.com)

TLDR; Summary

  • A team of researchers at the University of Maryland has developed an AI detection tool called Binoculars that can accurately identify over 90% of writing samples generated by AI.
  • Binoculars uses two "lenses" to analyze text: an "Observer" LLM that measures the unpredictability of the text, and a "Performer" LLM that predicts the next word based on the previous words.
  • If there is little difference in perplexity scores between the two LLMs, Binoculars predicts that the text was likely written by AI.
  • The tool has a false-positive rate of just 0.01%, meaning it incorrectly flags one out of every 10,000 samples as AI-generated.
  • However, schools may be hesitant to use the tool due to the risk of any false positives, even though the risk is lower than with other AI detection tools.
  • The researchers believe Binoculars could be valuable for detecting AI-written content on websites and social media platforms.
  • Teachers may