Jump to : Download | Abstract | Contact | BibTex reference | EndNote reference |

cheng2015fast

Yu Cheng, Felix X Yu, Rogerio S Feris, Sanjiv Kumar, Alok Choudhary, Shih-Fu Chang. Fast Neural Networks with Circulant Projections. Research Report arXiv preprint arXiv:1502.03436, 2015.

Download [help]

Download paper: (link)

Copyright notice:This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract

The basic computation of a fully-connected neural network layer is a linear projection of the input signal followed by a non-linear transformation. The linear projection step consumes the bulk of the processing time and memory footprint. In this work, we propose to replace the conventional linear projection with the circulant projection. The circulant structure enables the use of the Fast Fourier Transform to speed up the computation. Considering a neural network layer with d input nodes, and d output nodes, this method improves the time complexity from \mathcal{O}(d^2) to \mathcal{O}(d\log{d}) and space complexity from \mathcal{O}(d^2) to \mathcal{O}(d). We further show that the gradient computation and optimization of the circulant projections can be performed very efficiently. Our experiments on three standard datasets show that the proposed approach achieves this significant gain in efficiency and storage with minimal loss of accuracy compared to neural networks with unstructured projections

Contact

FelixX. Yu
Shih-Fu Chang

BibTex Reference

@TechReport{cheng2015fast,
   Author = {Cheng, Yu and Yu, Felix X and Feris, Rogerio S and Kumar, Sanjiv and Choudhary, Alok and Chang, Shih-Fu},
   Title = {Fast Neural Networks with Circulant Projections},
   Institution = {arXiv preprint arXiv:1502.03436},
   Year = {2015}
}

EndNote Reference [help]

Get EndNote Reference (.ref)

 
bar

For problems or questions regarding this web site contact The Web Master.

This document was translated automatically from BibTEX by bib2html (Copyright 2003 © Eric Marchand, INRIA, Vista Project).