Koch et al. (2021) found that more than 70% of benchmark datasets used in prominent computer-vision papers had been reused from other domains.
Sources
- Can We Trust AI Benchmarks? An Interdisciplinary Review ... (arxiv.org)
- https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/file/3b8a614226a953a8cd9526fca6fe9ba5-Paper-round2.pdf (datasets-benchmarks-proceedings.neurips.cc)
- https://aihub.org/2022/02/17/the-life-of-a-dataset-in-machine-learning-research-interview-with-bernard-koch/ (aihub.org)
More in Technology
- A facility for manufacturing leading-edge chips costs between $5 billion and $20 billion.
- Briefs filed in Supreme Court cases are linked within case dockets beginning in November 2017.
- Apple's M1 chip, manufactured by TSMC, contains 16 billion transistors.
- Anthropic launched an AI tool whose agents can autonomously operate microscopes, lasers, and robotic arms, which the company described as its first tool designed for the physical world.
- The average smartphone has more computing power than all of NASA had when it sent astronauts to the moon in 1969.
- The article 'New Theory Cracks Open the Black Box of Deep Neural Networks' was originally published by Quanta Magazine.