What happens to the culture of science when researchers increasingly rely on language models? During this online lecture, Maria Antoniak (University of Colorado Boulder) presents a series of studies on how researchers use language models. She argues that to make policy and build tools around these language systems, we need evidence from the naturally-occurring data that represents real-world AI usage by researchers.

Researchers are already frequent users of AI tools, with 81% of 816 surveyed authors reporting LLM use in their workflow in early 2024, though usage and concern vary by demographic group and field. But studying this usage in the wild is hard. Realistic logs of how people use these tools are difficult to obtain, and chat data turns out to be full of personal disclosures.

Language models can both reflect and reshape the norms of the communities that use them. Research communities differ in their structural, stylistic, rhetorical, and citational conventions. Across 81,000 papers from eleven research communities, and using a framework built from interviews with interdisciplinary researchers, Antoniak’s work has shown that models asked to adapt writing between fields move most metrics in one direction regardless of the target, homogenizing the writing style of research papers.

Institutions urgently need to address challenges arising with the use of language models for paper writing, but developing constructive policies is difficult without data and evidence. For example, the arXiv banned unpublished computer science survey and position papers in 2025, citing LLM-generated content, but measuring that data directly reveals that while survey and position papers do have higher predicted rates of generated content,  generated non-survey and non-position papers outnumber them roughly six to one, and the ban would cut around half of submissions in Computers and Society against three percent in Computer Vision. These unintended side effects reflect cultural differences between disciplines that need to be taken into account when considering policies around AI usage.

About the speaker

Maria Antoniak is a natural language processing (NLP) and cultural analytics researcher working at the intersection of computing and the humanities. She’s an Assistant Professor of Computer Science at the University of Colorado Boulder, affiliated with Information Science and the Boulder NLP Group, where she directs the Culture, Language, and Systems (CLS) Lab.

Registration

To attend, please complete the registration form below. You will receive a confirmation email with the Teams link included.