Author: aizawa

Hiroo Takizawa (SOKENDAI/NII LLMC) received the Encouragement Award (奨励賞) at the 20th Young Researchers Symposium on Natural Language Processing (YANS2025) for his presentation “Evaluating Context-Following Capabilities of LLMs in Japanese.”

Our website has moved to a new server. Some content is still under construction.

Seminar talk was given by Dr. Frieder Simon.  The talks was titled “The AIMO (AI Math Olympiad) and LLMs’ mathematical abilities.” Thank you for interesting talks.

NTCIR-19 Kickoff Event was held. We introduced a new pilot task

For more details on sciclaimeval, click here.　

Multiple-Choice Questions (MCQs) are frequently used to evaluate Large Language Models (LLMs). We have released MCQFormatBench, an evaluation dataset designed to assess a model’s robustness to the MCQ format. We categorized the answer process for MCQs into four types and designed eight tasks. By converting 600 questions sampled from the MMLU dataset for each task,… Continue reading (2025.9.1) MCQFormatBench dataset released