Google's Gemma-2 9b and OpenAI's GPT-4o achieved near-perfect scores on Anthropic's DiscrimEval bias benchmark, released in December 2023.
Sources
- These new AI benchmarks could help make models less biased (technologyreview.com)
- https://arxiv.org/abs/2502.01926 (arxiv.org)
- https://slguardian.org/new-ai-benchmarks-aim-to-reduce-bias-and-improve-fairness/ (slguardian.org)
- https://techcrunch.com/2023/12/07/anthropics-latest-tactic-to-stop-racist-ai-asking-it-really-really-really-really-nicely/ (techcrunch.com)
- https://venturebeat.com/ai/anthropic-leads-charge-against-ai-bias-and-discrimination-with-new-research (venturebeat.com)
More in Technology
- The IEEE 802.15.3a High Rate Alternative PHY Task Group targeted a UWB standard capable of at least 110 Mbps at 10 meters, with higher speeds at shorter distances, for applications including high-definition audio/video streaming and cable replacement.
- Researchers created a benchmark called VPBench, a visually prompted benchmark containing 16 visual marker variants.
- Atlas matched human pathologists' conclusions 97.1% of the time when categorizing cancerous colorectal tissue.
- FindZebra uses the open source information retrieval tool Indri to search its index.
- RankRAG instruction-tunes a single large language model to perform both context ranking and answer generation for retrieval-augmented generation (RAG).
- A 100-kilowatt lunar nuclear reactor targeted by NASA is described as 10,000 times less powerful than a typical nuclear power plant.