A team of software engineering researchers from the University of Waterloo has received the ACM SIGSOFT Distinguished Paper Award at an international conference held in Rio de Janeiro, Brazil. The award was presented at FORGE 2026, which took place April 12–13, 2026, as part of the broader ICSE 2026 event.
The research team, which includes lead author and recent master’s graduate Evelien Riddell, alongside MMath student James Riddell, PhD student Gengyi Sun, research engineer Michał Antkiewicz, and Professor Krzysztof Czarnecki, was recognized for their paper titled “Stalled, Biased, and Confused: Uncovering Reasoning Failures in LLMs for Cloud-based Root Cause Analysis.” The work examines how effectively artificial intelligence systems can perform root cause analysis, which is the process of identifying the source of failures within complex cloud computing environments.
To evaluate these systems, the researchers developed a controlled framework that removed confounding factors such as simplified agent architectures, deterministic tools, and structured knowledge graphs to isolate how the models reason. The team tested six open-source large language models across three different workflows, analyzing 48,000 scenarios over a period of 228 days. Beyond checking for accuracy, the study established a list of 16 types of reasoning failures and utilized an automated judge to review over 3,000 inference traces.
The findings indicate that while current artificial intelligence models are often capable of providing plausible-sounding diagnoses, their underlying reasoning is frequently flawed. The study further notes that as the complexity of the task increases, the accuracy of the models tends to decline.
The research was conducted under the supervision of Professor Czarnecki, who leads the Waterloo Intelligent Systems Engineering Lab. Evelien Riddell, who holds a Bachelor of Science in physics and computer science from the University of British Columbia in addition to her recent Waterloo degree, conducted this award-winning research as the foundation of her master’s thesis.
