Towards Ubiquitous Intelligent Computing: Heterogeneous Distributed Deep Neural Networks

Zongpu Zhang, Tao Song, Liwei Lin, Yang Hua, Xufeng He, Zhengui Xue, Ruhui Ma, Haibing Guan

Research output: Contribution to journalArticlepeer-review

463 Downloads (Pure)


For the pursuit of ubiquitous computing, distributed computing systems containing the cloud, edge devices, and Internet-of-Things devices are highly demanded. However, existing distributed frameworks do not tailor for the fast development of Deep Neural Network (DNN), which is the key technique behind many intelligent applications nowadays. Based on prior exploration on distributed deep neural networks (DDNN), we propose Heterogeneous Distributed Deep Neural Network (HDDNN) over the distributed hierarchy, targeting at ubiquitous intelligent computing. While being able to support basic functionalities of DNNs, our framework is optimized for various types of heterogeneity, including heterogeneous computing nodes, heterogeneous neural networks, and heterogeneous system tasks. Besides, our framework features parallel computing, privacy protection and robustness, with other consideration for the combination of heterogeneous distributed system and DNN. Extensive experiments demonstrate that our framework is capable of utilizing hierarchical distributed system better for DNN and tailoring DNN for real-world distributed system properly, which is with low response time, high performance, and better user experience.
Original languageEnglish
Number of pages15
Journal IEEE Transactions on Big Data
Early online date26 Nov 2018
Publication statusEarly online date - 26 Nov 2018


Dive into the research topics of 'Towards Ubiquitous Intelligent Computing: Heterogeneous Distributed Deep Neural Networks'. Together they form a unique fingerprint.

Cite this