US EditionPublished • loading... • Updated
University of Waterloo: Top AI coding tools make mistakes one in four times
University of Waterloo: Top AI coding tools make mistakes one in four times. “Even the most advanced models achieved only about 75 per cent accuracy in the tests, while open-source models performed closer to 65 per cent. The study evaluated 11 LLM models across 18 structured output formats and 44 tasks designed to assess how reliably the systems followed structured rules.”
This story is only covered by news sources that have yet to be evaluated by the independent media monitoring agencies we use to assess the quality and reliability of news outlets on our platform. Learn more here.1 Articles
1 Articles
University of Waterloo: Top AI coding tools make mistakes one in four times
University of Waterloo: Top AI coding tools make mistakes one in four times. “Even the most advanced models achieved only about 75 per cent accuracy in the tests, while open-source models performed closer to 65 per cent. The study evaluated 11 LLM models across 18 structured output formats and 44 tasks designed to assess how reliably the systems followed structured rules.”
Coverage Details
Total News Sources1
Leaning Left0Leaning Right0Center0Last UpdatedBias DistributionNo sources with tracked biases.
Bias Distribution
- There is no tracked Bias information for the sources covering this story.
Factuality
To view factuality data please Upgrade to Premium
