See veebileht kasutab küpsiseid kasutaja sessiooni andmete hoidmiseks. Veebilehe kasutamisega nõustute ETISe kasutustingimustega. Loe rohkem
Olen nõus

TOT-Net: An Endeavor Toward Optimizing Ternary Neural Networks

Nazari, N.; Loni, M.; Salehi, M.E.; Daneshtalab, M.; Sjodin, M. (2019). TOT-Net: An Endeavor Toward Optimizing Ternary Neural Networks. Proceedings - Euromicro Conference on Digital System Design, DSD 2019. 305−312.10.1109/DSD.2019.00052.
publitseeritud konverentsiettekanne
Nazari, N.; Loni, M.; Salehi, M.E.; Daneshtalab, M.; Sjodin, M.
  • Inglise
Proceedings - Euromicro Conference on Digital System Design, DSD 2019
Journal of Functional Foods
9781728128610
2019
305312
Ilmunud
1.1. Teadusartiklid, mis on kajastatud Web of Science andmebaasides Science Citation Index Expanded, Social Sciences Citation Index, Arts & Humanities Citation Index, Emerging Sources Citation Index ja/või andmebaasis Scopus (v.a. kogumikud)
Teadmata
SCOPUS

Viited terviktekstile

dx.doi.org/10.1109/DSD.2019.00052

Seotud asutused

University of Tehran; Mälardalens högskola

Lisainfo

© 2019 IEEE. High computation demands and big memory resources are the major implementation challenges of Convolutional Neural Networks (CNNs) especially for low-power and resource-limited embedded devices. Many binarized neural networks are recently proposed to address these issues. Although they have significantly decreased computation and memory footprint, they have suffered from accuracy loss especially for large datasets. In this paper, we propose TOT-Net, a ternarized neural network with [-1, 0, 1] values for both weights and activation functions that has simultaneously achieved a higher level of accuracy and less computational load. In fact, first, TOT-Net introduces a simple bitwise logic for convolution computations to reduce the cost of multiply operations. To improve the accuracy, selecting proper activation function and learning rate are influential, but also difficult. As the second contribution, we propose a novel piece-wise activation function, and optimized learning rate for different datasets. Our findings first reveal that 0.01 is a preferable learning rate for the studied datasets. Third, by using an evolutionary optimization approach, we found novel piece-wise activation functions customized for TOT-Net. According to the experimental results, TOT-Net achieves 2.15%, 8.77%, and 5.7/5.52% better accuracy compared to XNOR-Net on CIFAR-10, CIFAR-100, and ImageNet top-5/top-1 datasets, respectively.
activation function | convolutional neural networks | optimization | ternary neural network