When GPT-4 was tested in 2023 on easy Codeforces problems, Narayanan and Kapoor (2023b) found that it could regularly solve problems added before 5 September 2021 but got no question right among problems added afterward.
Sources
- Can We Trust AI Benchmarks? An Interdisciplinary Review ... (arxiv.org)
- https://www.normaltech.ai/p/gpt-4-and-professional-benchmarks (normaltech.ai)
- https://shana.codes/posts/week-notes-4-03-23.html (shana.codes)
- https://www.cs.princeton.edu/~sayashk/papers/crcl-kapoor-henderson-narayanan.pdf (cs.princeton.edu)
More in Technology
- The global automotive industry was projected to lose more than $110 billion in 2021 because of the microchip shortage.
- AI models evaluated against FrontierMath include OpenAI's GPT-4o and o1-preview.
- The U.S. Naval Research Laboratory's SWELL experiment achieved continuous in-orbit laser power transmission for more than 100 days in 2023, delivering about 1.5 W over 1.45 meters with an end-to-end efficiency of approximately 11%.
- Agreement between AI models and physician majority vote on NEJM Image Challenge cases was low, with Cohen's kappa of 0.08 for both GPT-4o and Claude 3.7, and 0.24 for Doubao.
- Aerial reconnaissance can be used for the collection of imagery intelligence.
- The article 'IC Failure Analysis: Magic, Mystery, and Science' has the DOI 10.1109/54.605998.