Resource-efficient convolutional networks: a survey on model, arithmetic, and implementation-level techniques

JunKyu Lee, Lev Mukhanov, Amir Sabbagh Molahosseini, Umar Minhas, Yang Hua, Jesus Martinez-del-Rincon, Kiril Dichev, Cheol-Ho Hong*, Hans Vandierendonck

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

55 Downloads (Pure)

Abstract

The Convolutional Neural Networks (CNNs) are used in our daily life, including self-driving cars, virtual assistants, social network services, healthcare services, face recognition, etc. However, deep CNNs demand substantial compute resources during training and inference. The machine learning community has mainly focused on model-level optimizations such as architectural compression of CNNs, while the system community has focused on implementation-level optimization. In between, various arithmetic-level optimization techniques have been proposed in the arithmetic community. This article provides a survey on resource-efficient CNN techniques in terms of model-, arithmetic-, and implementation-level techniques and identifies the research gaps for resource-efficient CNN techniques across the three different level techniques. Our survey clarifies the influence from higher to lower-level techniques based on our resource-efficiency metric definition and discusses the future trend for resource-efficient CNN research.

Original languageEnglish
JournalACM Computing Surveys
Early online date14 Mar 2023
DOIs
Publication statusEarly online date - 14 Mar 2023

Fingerprint

Dive into the research topics of 'Resource-efficient convolutional networks: a survey on model, arithmetic, and implementation-level techniques'. Together they form a unique fingerprint.

Cite this