RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors
Date:
Abstract
In this pre-recorded talk I summarize the motivation, design decisions, and findings of our ACL 2024 long paper “RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors”. In our paper we release RAID–the largest and most comprehensive benchmark dataset for machine generated text detectors. Using RAID, we show that detectors struggle to operate well at safe false positive rates, regularly struggle to generalize to unseen generators and decoding strategies, and are susceptible to adversarial attacks. We provide an evaluation suite and leaderboard to promote further shared evaluation of machine-generated text detectors.
Location
This talk was recorded on August 10th 2024 at the Centara Grand Hotel in Bangkok, Thailand as part of the ACL 2024 conference.