Insights from the Use of Previously Unseen Neural Architecture Search Datasets

Geada, Rob; Towers, David; Forshaw, Matthew; Atapour-Abarghouei, Amir; Mcgough, A Stephen

Abstract

The boundless possibility of neural networks which can be used to solve a problem-each with different performance leads to a situation where a Deep Learning expert is required to identify the best neural network. This goes against the hope of removing the need for experts. Neu-ral Architecture Search (NAS) offers a solution to this by automatically identifying the best architecture. However, to date, NAS work has focused on a small set of datasets which we argue are not representative of real-world problems. We introduce eight new datasets created for a series of NAS Challenges: AddNIST, Language, MultNIST, CIFAR-Tile, Gutenberg, Isabella, GeoClassing, and Chesseract. These datasets and challenges are developed to direct attention to issues in NAS development and to encourage authors to consider how their models will perform on datasets unknown to them at development time. We present experimentation using standard Deep Learning methods as well as the best results from challenge participants.

Citation

Geada, R., Towers, D., Forshaw, M., Atapour-Abarghouei, A., & Mcgough, A. S. (2024, June). Insights from the Use of Previously Unseen Neural Architecture Search Datasets. Presented at IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), Seattle, WA

• Presentation Conference Type: Conference Paper (published)
• Conference Name: IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR)
• Start Date: Jun 17, 2024
• End Date: Jun 21, 2024
• Acceptance Date: Feb 26, 2024
• Online Publication Date: Sep 16, 2024
• Publication Date: Sep 16, 2024
• Deposit Date: Apr 19, 2024
• Publicly Available Date: Sep 23, 2024
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Pages: 22541-22550
• Series ISSN: 1063-6919
• Book Title: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• ISBN: 9798350353013
• DOI: https://doi.org/10.1109/CVPR52733.2024.02127
• Public URL: https://durham-repository.worktribe.com/output/2389724