Overview
The SHROOM-CAP shared task is set to take place at the CHOMPS-2025 workshop, co-located with IJCNLP-AACL 2025. This initiative focuses on advancing the state-of-the-art in hallucination detection within scientific content generated by large language models (LLMs). Participants will engage in identifying hallucinated content across multiple languages, contributing to a significant area of research in AI and machine learning.
Background & Relevance
Hallucination detection is a critical area of study in the field of natural language processing, particularly as LLMs become more prevalent in generating scientific texts. These models often produce outputs that, while seemingly plausible, can contain inaccuracies or fabrications—referred to as hallucinations. The SHROOM-CAP task aims to address this issue by providing a platform for researchers to develop and evaluate methods for detecting such inaccuracies in cross-lingual contexts. This is essential not only for improving the reliability of AI-generated content but also for ensuring the integrity of scientific communication.
Key Details
- Train set available by: 31.07.2025
- Validation set available by: 05.09.2025
- Test set available by: 05.10.2025
- Test phase ends: 12.10.2025
- Leaderboard release: 15.10.2025
- System description papers due: 25.10.2025
- Notification of acceptance: 05.11.2025
- Camera-ready due: 11.11.2025
- Proceedings due: 01.12.2025
- CHOMPS workshop dates: 23/24th December 2025
- Location: Mumbai, India
Eligibility & Participation
The SHROOM-CAP task is open to researchers and practitioners interested in the detection of hallucinations in scientific content produced by LLMs. Participants can engage in any of the available languages, which include both high-resource and low-resource Indic languages. This task is particularly suited for those working in the fields of natural language processing, machine learning, and AI research.
Submission or Application Guidelines
To participate in the SHROOM-CAP shared task, follow these steps:
1. Register your team: Complete the registration form here.
2. Join the Google group: Stay updated by joining the mailing list here.
3. Submit results: Use the submission platform available at this link before the test phase ends on 12.10.2025.
4. Submit system description papers: These should be submitted by 25.10.2025.
Additional Context / Real-World Relevance
The SHROOM-CAP shared task is part of a broader effort to enhance the reliability of AI-generated content, especially in scientific domains. As LLMs are increasingly utilized for research and publication, ensuring the accuracy of their outputs is paramount. This task not only contributes to the academic community but also has implications for the public’s trust in AI-generated information.
Conclusion
The SHROOM-CAP shared task presents a valuable opportunity for researchers to contribute to the critical area of hallucination detection in scientific publications. Participants are encouraged to engage with this challenge, develop innovative solutions, and share their findings at the upcoming CHOMPS workshop. Explore this exciting opportunity and help advance the field of AI research.
Category: CFP & Deadlines
Tags: hallucination detection, cross-lingual, scientific content, LLMs, AI research, CHOMPS, IJCNLP, AACL, NLP, machine learning
