Merging Neurons for Structure Compression of Deep Networks

Guoqiang Zhong, Hui Yao, Huiyu Zhou

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Deep neural networks are increasingly used in many fields, such as pattern recognition, computer vision, and natural language processing. However, how to apply deep neural networks in mobile settings has become an urgent issue, as mobile devices are getting more and more popularity. This is mainly due to the fact that mobile devices usually have very limited computation and storage resources, which prevents from running a large-scale deep network. This paper proposes a novel method for structure compression of deep neural networks. The main idea is to merge the neurons and connections of the original network using clustering methods. To the end, the new network after compression possesses much less parameters, which leads to reduced requirements for computation and storage resources. Experiments on benchmark data sets demonstrate that the proposed method can greatly improve the efficiency of deep neural networks, while retain their learning capability.

Original languageEnglish
Title of host publication2018 24th International Conference on Pattern Recognition, ICPR 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1462-1467
Number of pages6
ISBN (Electronic)9781538637883
DOIs
Publication statusPublished - 29 Nov 2018
Externally publishedYes
Event24th International Conference on Pattern Recognition, ICPR 2018 - Beijing, China
Duration: 20 Aug 201824 Aug 2018

Publication series

NameProceedings - International Conference on Pattern Recognition
Volume2018-August
ISSN (Print)1051-4651

Conference

Conference24th International Conference on Pattern Recognition, ICPR 2018
CountryChina
CityBeijing
Period20/08/201824/08/2018

Bibliographical note

Funding Information:
This work was supported by the National Key R&D Program of China under Grant 2016YFC1401004, the Science and Technology Program of Qingdao under Grant No. 17-3-3-20-nsh, the CERNET Innovation Project under Grant No. NGI-I20170416, the CCF-Tencent Open Fund, the UK EPSRC under Grants EP/N508664/1, EP/R007187/1 and EP/N011074/1, the Royal Society-Newton Advanced Fellowship under Grant NA160342, and the Fundamental Research Funds for the Central Universities of China.

Publisher Copyright:
© 2018 IEEE.

Copyright:
Copyright 2019 Elsevier B.V., All rights reserved.

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition

Fingerprint

Dive into the research topics of 'Merging Neurons for Structure Compression of Deep Networks'. Together they form a unique fingerprint.

Cite this