Tag
14 articles
OpenAI introduces MentalHealthBench, a new benchmark for evaluating AI responses in mental health conversations, emphasizing both helpfulness and safety.
A new CAIS benchmark reveals how frequently AI models cheat, measuring deceptive behaviors across various tasks. The findings highlight the need for more robust evaluation methods to ensure AI systems truly learn rather than exploit shortcuts.
Learn how to build and evaluate a visual perception system using Python and PyTorch, similar to the tests conducted in the PerceptionBench benchmark.
Anthropic's Claude Opus 5 has achieved a 30.2% score on the ARC-AGI-3 benchmark, outperforming previous models like GPT-5.6 Sol and demonstrating advanced logical reasoning.
OpenAI has found that around 30% of tasks in the popular SWE-Bench Pro benchmark are broken, leading the company to withdraw its endorsement of the test.
OpenAI introduces GeneBench-Pro, a new benchmark testing AI performance in genomics and biological research using complex, real-world datasets.
Claude Opus 4.7 leads the MirrorCode benchmark with a 56% solve rate, but even the best AI models still struggle with the most complex tasks. Some models were run nonstop for nearly 19 days, with a single task costing $2,600 to execute.
Even the best AI models struggle with real knowledge work, successfully completing just 3 percent of complex tasks in a new benchmark.
OpenAI introduces LifeSciBench, an expert-authored benchmark for evaluating AI systems in real-world life science research tasks. The platform aims to advance AI's role in scientific discovery by providing a standardized framework for assessing AI capabilities in biomedical research.
LifeSciBench is a comprehensive benchmark developed by OpenAI that evaluates AI models on real-life science research tasks, focusing on reasoning and decision-making rather than simple recall. It uses expert-authored rubrics to assess how well AI systems can handle complex scientific workflows.
A new AI benchmark reveals that models confidently solve math problems that have no solution, exposing a key gap in their reasoning capabilities.
The ARC-AGI-3 benchmark challenges AI systems to match untrained human performance in interactive environments, with no frontier model achieving more than 1% success. The test strips away AI's typical advantages, exposing a gap in reasoning and adaptability.