| Title |
Holarchic structures for decentralized deep learning: a performance analysis |
| ID_Doc |
41296 |
| Authors |
Pournaras, E; Yadhunathan, S; Diaconescu, A |
| Title |
Holarchic structures for decentralized deep learning: a performance analysis |
| Year |
2020 |
| Published |
Cluster Computing-The Journal Of Networks Software Tools And Applications, 23, 1 |
| DOI |
10.1007/s10586-019-02906-4 |
| Abstract |
Structure plays a key role in learning performance. In centralized computational systems, hyperparameter optimization and regularization techniques such as dropout are computational means to enhance learning performance by adjusting the deep hierarchical structure. However, in decentralized deep learning by the Internet of Things, the structure is an actual network of autonomous interconnected devices such as smart phones that interact via complex network protocols. Self-adaptation of the learning structure is a challenge. Uncertainties such as network latency, node and link failures or even bottlenecks by limited processing capacity and energy availability can significantly downgrade learning performance. Network self-organization and self-management is complex, while it requires additional computational and network resources that hinder the feasibility of decentralized deep learning. In contrast, this paper introduces a self-adaptive learning approach based on holarchic learning structures for exploring, mitigating and boosting learning performance in distributed environments with uncertainties. A large-scale performance analysis with 864,000 experiments fed with synthetic and real-world data from smart grid and smart city pilot projects confirm the cost-effectiveness of holarchic structures for decentralized deep learning. |
| Author Keywords |
Deep learning; Optimization; Holarchy; Multi-agent system; Resilience; Smart city |
| Index Keywords |
Index Keywords |
| Document Type |
Other |
| Open Access |
Open Access |
| Source |
Science Citation Index Expanded (SCI-EXPANDED) |
| EID |
WOS:000512937700015 |
| WoS Category |
Computer Science, Information Systems; Computer Science, Theory & Methods |
| Research Area |
Computer Science |
| PDF |
https://link.springer.com/content/pdf/10.1007/s10586-019-02906-4.pdf
|