A team of graduate students in CS and ECE took the top prize in a competition held as part of NeurIPS, a top conference for artificial intelligence (AI). The competition required teams to curate training data for a multimodal AI reasoning (understanding images and language). The team worked on the challenge as part of professor Igor Molybog’s graduate course on Large-Scale AI.
The NeurIPS 2025 Data Curation for Vision–Language Reasoning (DCVLR) challenge was designed to evaluate how data curation choices—rather than model architecture or scale—affect multimodal reasoning performance under a fixed training and evaluation pipeline. In this setting, participants were allowed to curate up to 10,000 multimodal reasoning examples to fine-tune a shared base model and were evaluated on a benchmark suite comprising both known and held-out tasks spanning academic question answering, mathematics, physics, and general visual reasoning.
Team 3 from ECE 605 presented a data curation strategy based on difficulty-aware filtering, which achieved the highest overall performance in the competition. A task example was judged difficult if the base model failed to solve it at least half the times it had attempted (for specific inference hyperparameters). Starting from the organizer-provided Walton Multimodal Cold Start dataset, examples were filtered according to this criterion, yielding a compact training set that achieved 49.1% overall accuracy on the official DCVLR benchmark, substantially outperforming the base model and models trained on much larger datasets.
The team is working on a publication that discusses the main and auxiliary results in more detail.
In Spring 2026, members of the team and Prof. Molybog organize a follow-up paper-reading seminar for volunteers that will follow the format of the ECE 605 course but focus on Visual-Language-Action models and other applications of multimodal reasoning in robotics. Any interested students are welcome to contact Prof. Molybog directly.
Read more via UH News article: https://www.hawaii.edu/news/2025/11/20/ai-competition-winner/