David Powers
Researcher in Computer and Cognitive Science and oversees a wide range of projects in artificial intelligence, robotics and assistive technology
Agents4Science 2025 is an interesting experiment. The whole conference restricted to AI written papers, and reviewed by AIs.
Many authors are now routinely using AI to write or rewrite their papers, including finding missed references. Conversely conferences and publishers are now exploring how AI can be used to referee papers - and establish which work is genuine and which is AI-hallucinated. I myself have found AIs have hallucinated several papers my colleagues and I might have (and possible should have) written. In one case, this was in a grant application and it turned out the applicant had asked the AI for further relevant papers from our group.
AI researchers are still trying to get a grip on this, and the Association for the Advancement of Artificial Intelligence (AAAI) this year introduced AI reviewing as a supplement to human reviewing (with authors seeing both anonymous human and AI reviews, as well as an AI-generated summary). AAAI-26 saw another massive increase in submissions, and this review system tested in practice. But recognizing AI-authored papers, distinguishing AI-hallucinated 'research' from real work, and assuring the ongoing quality of publication venues remain daunting challenges.
Agents4Science 2025 will provide an opportunity to see papers that are openly AI-written and openly AI-reviewed, and analyse this data to inform the community’s efforts to ensuring research integrity and optimized processing in our new AI-driven age. This doesn’t mean just identifying AI-generated papers, but exploring the scope for active human-AI teaming in solving important research problems, and deploying AI help systems, advisors and chatbots. I’ll look forward to seeing the data.
The acceptance rate of ~16% (24 out of 300+) is comparable to many journals and lower than most conferences. This looks like being an interesting and useful dataset for analysis to help us understand the use of AI in the research world.