JPEG compression levels used in API calls can change the ranking lineup of models on visually prompted benchmarks.
Sources
- Visually Prompted Benchmarks Are Surprisingly Fragile (arxiv.org)
- https://www.researchgate.net/publication/398937366_Visually_Prompted_Benchmarks_Are_Surprisingly_Fragile (researchgate.net)
- https://lisadunlap.github.io/vpbench/ (lisadunlap.github.io)
- https://github.com/TonyLianLong/VPBench (github.com)
More in Technology
- A 100-kilowatt lunar nuclear reactor targeted by NASA is described as 10,000 times less powerful than a typical nuclear power plant.
- In NEJM Image Challenge cases where fewer than 40% of physicians selected the correct diagnosis, Claude 3.7 achieved 86.5% accuracy and GPT-4o achieved 78.8% accuracy.
- Researchers used artificial intelligence and X-ray scans to decipher parts of a 2,000-year-old papyrus scroll burned in the A.D. 79 eruption of Mount Vesuvius.
- The paper 'The unusual suspects' by N. Fatma, M. Chinnakotla, M. Shrivastava, S. Singh, and S. Markovitch appeared in the Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence on 4 February 2017, spanning pages 1107–1113.
- Under the draft EU AI Regulation, conformance with harmonised standards creates a presumption of conformity for high-risk AI applications and services.
- The paper 'Large-scale network motif analysis using compression' by P. Bloem and S. de Rooij was published online on 23 June 2020 in Data Mining and Knowledge Discovery, volume 34, issue 5, pages 1421–1453.