OpenAI researchers Yuri Burda and Harri Edwards accidentally discovered the 'grokking' phenomenon after arithmetic-training experiments ran for days instead of hours, finding that models which initially memorized example sums eventually learned to add previously unseen numbers.
Sources
- Large language models can do jaw-dropping things. But nobody ... (technologyreview.com)
- https://www.quantamagazine.org/how-do-machines-grok-data-20240412/ (quantamagazine.org)
- https://arxiv.org/abs/2201.02177 (arxiv.org)
More in Technology
- An advanced persistent threat gains unauthorized access to a computer network and remains undetected for an extended period.
- The primary-source files on the American Memory website were made available for classroom use through the Library of Congress's Teaching with Primary Sources Program.
- Smartphone or tablet-based time use surveys can passively collect data such as geolocation for activities and periods when the device was in use.
- Canyon SIGINT satellites were in use during the same period as Aquacade satellites with an apparently different set of capabilities.
- A software defect in the Chinook helicopter was described as "positively dangerous" because deficiencies meant the pilot's full control of the engines "could not be assured."
- In the PacifAIst benchmark evaluation, Google's Gemini 2.5 Flash achieved the highest overall Pacifism Score among eight evaluated models, scoring 90.31%.