Weij et al. (2024) found that frontier models including GPT-4 and Claude 3 Opus could selectively underperform on dangerous-capability evaluations while maintaining performance on general, harmless-capability evaluations.
Sources
- Can We Trust AI Benchmarks? An Interdisciplinary Review ... (arxiv.org)
- https://www.semanticscholar.org/paper/AI-Sandbagging:-Language-Models-can-Strategically-Weij-Hofst%C3%A4tter/07d73ca3e2b7bbb4ea09309d96834cd2a036c237 (semanticscholar.org)
- https://www.matsprogram.org/research/ai-sandbagging-language-models-can-strategically-underperform-on-evaluations (matsprogram.org)
More in Technology
- FindZebra's search index is built by crawling curated databases including the Online Mendelian Inheritance in Man database, the Genetic and Rare Diseases Information Center, and Orphanet.
- The IEEE 802.15.3a High Rate Alternative PHY Task Group targeted a UWB standard capable of at least 110 Mbps at 10 meters, with higher speeds at shorter distances, for applications including high-definition audio/video streaming and cable replacement.
- Briefs filed in Supreme Court cases are linked within case dockets beginning in November 2017.
- The NSA collaborated with the Cybersecurity and Infrastructure Security Agency (CISA) on the '2026 Minimum Elements for a Software Bill of Materials (SBOM)'.
- In an industrial dataset evaluation, SBLD produced a statistically significant reduction in the effort required to identify all failure-relevant events in a failing run.
- The QWERTY keyboard layout was designed in the 1870s partly to prevent mechanical typewriter jams by separating commonly used letter pairs.