A 2019 paper coauthored by Harvard's Boaz Barak showed that double descent can occur not only as models gain parameters, but also when they receive more training data or are trained for longer.
Sources
- Large language models can do jaw-dropping things. But nobody ... (technologyreview.com)
- https://arxiv.org/abs/1912.02292 (arxiv.org)
- https://openai.com/index/deep-double-descent/ (openai.com)
- https://windowsontheory.org/2019/12/05/deep-double-descent/ (windowsontheory.org)
- https://quantum.harvard.edu/boaz-barak (quantum.harvard.edu)
More in Science
- The Elevation Science Institute conducts paleontological field work in the Bighorn Basin of Montana and Wyoming.
- At a 2009 FDA advisory-panel meeting, members voted 36-1 to recommend that children's acetaminophen products be made in a single concentration, but the FDA never made that requirement mandatory.
- The 'information bottleneck' theory has been suggested as a possible explanation for how human brains learn.
- The fault slip on the southern San Andreas Fault triggered by the 2017 Chiapas earthquake began minutes after that earthquake and continued for more than a year.
- In a two-year Duke University study, participants assigned to reduce calories by 25% achieved an average reduction of 12%, and their biological age increased by 0.11 years per year, compared with 0.71 years per year among people eating their usual diets.
- Physicists discovered in 2013 the world's first naturally occurring topological insulator, which can be mined from the earth's crust.