In an analysis comparing college admission essays generated by artificial intelligence to 30,000 human-written essays from before ChatGPT was released, Cornell University researchers found that AI essays are highly generic and easy to distinguish from human writing.
The large language models struggled to create unique narratives, even when researchers provided them with specific characteristics of the essay writer. In fact, providing those specific characteristics often made the essays sound even more robotic, as the AI would force keywords about the author’s identity into the essay, the researchers found.
“The admissions essay is an opportunity for applicants to offer a glimpse into who they are, beyond all the structured information on the application form,” said one of the authors, Rene Kizilcec, an associate professor of information science at Cornell. “Tools like ChatGPT can give solid feedback on writing and are likely a good idea for weak writers. But asking for a full draft will yield a generic essay that just does not sound like any real applicant.”
The researchers also trained an AI tool to differentiate between the AI- and human-written essays, which worked with near-perfect accuracy.
