Distributed Deep Learning in IoT: Splitting Neural Networks in Inference and Training
Loading...
Files
Authors
Guo, Jia
Date
Type
thesis
Language
eng
Keyword
distributed deep learning , IoT , Split Learning
Alternative Title
Abstract
In this thesis, we first propose a scheme to solve the inference problem of Network of Neural Networks (NoNN) under changing channel conditions. The proposed scheme is composed of two methods: two devices work in parallel and one device works with the server. Error Correction Codes (ECC) are also applied to deal with the errors during data transmitting. Our scheme is to find the optimal method and code rate of ECC to maximize the accuracy within a specific delay threshold. Experiment results show that the proposed scheme performs much better than the other schemes for inference of NoNN under changing channel environment compared with other schemes with a single method or fixed code rate.
We then propose a scheme applying Online Knowledge Distillation via Collaboratively Learning (KDCL) to Split Learning to get a distilled model of server-part model in SplitNN after training. The distilled model could be designed to fit the memory-limited Internet of Things (IoT) devices so that clients could get prediction results independently after receiving the distilled model from the server. In this way, high communication latency and costs between client and server can be avoided. Experiments show that the proposed scheme could successfully get a distilled model after training. Besides, the accuracies of both the original model and distilled model could get a significant improvement compared with baseline models.
Description
Citation
Publisher
License
Queen's University's Thesis/Dissertation Non-Exclusive License for Deposit to QSpace and Library and Archives Canada
ProQuest PhD and Master's Theses International Dissemination Agreement
Intellectual Property Guidelines at Queen's University
Copying and Preserving Your Thesis
This publication is made available by the authority of the copyright owner solely for the purpose of private study and research and may not be copied or reproduced except as permitted by the copyright laws without written authority from the copyright owner.
ProQuest PhD and Master's Theses International Dissemination Agreement
Intellectual Property Guidelines at Queen's University
Copying and Preserving Your Thesis
This publication is made available by the authority of the copyright owner solely for the purpose of private study and research and may not be copied or reproduced except as permitted by the copyright laws without written authority from the copyright owner.