Compact and Computationally Efficient Representation of Deep Neural Networks

By Simon Wiedemann, K. Müller, Wojciech Samek. 2018

77 citations recorded by the source corpus.

At the core of any inference procedure, deep neural networks are dot product operations, which are the component that requires the highest computational resources. For instance, deep neural networks, such as VGG-16, require up to 15-G operations in order to perform the dot products present in a single forward pass, which results in significant energy consumption and thus limits their use in resource-limited environments, e.g., on embedded devices or smartphones. One common approach to reduce the complexity of the inference is to prune and quantize the weight matrices of the neural network. Usually, this results in matrices whose entropy values are low, as measured relative to the empirical probability mass distribution of its elements. In order to efficiently exploit such matrices, one usually relies on, inter alia, sparse matrix representations. However, most of these common matrix storage formats make strong statistical assumptions about the distribution of the elements; therefore, cannot efficiently represent the entire set of matrices that exhibit low-entropy statistics (thus, the entire set of compressed neural network weight matrices). In this paper, we address this issue and present new efficient representations for matrices with low-entropy statistics. Alike sparse matrix data structures, these formats exploit the statistical properties of the data in order to reduce the size and execution complexity. Moreover, we show that the proposed data structures can not only be regarded as a generalization of sparse formats but are also more energy and time efficient under practically relevant assumptions. Finally, we test the storage requirements and execution performance of the proposed formats on compressed neural networks and compare them to dense and sparse representations. We experimentally show that we are able to attain up to $\times 42$ compression ratios, $\times 5$ speed ups, and $\times 90$ energy savings when we lossless convert the state-of-the-art networks, such as AlexNet, VGG-16, ResNet152, and DenseNet, into the new data structures and benchmark their respective dot product.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data (2019, background): “Many of these devices are equipped with various sensors and increasingly potent hardware that allow them to collect and process data at unprecedented scales [1][2][3].”
Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communication (2018, background): “[39]) and to investigate the learning process and the changes in representation with explanation methods [40], [41].”
Entropy-Constrained Training of Deep Neural Networks (2018, background): “We set their respective initial cardinalities, K̄, as follows: (i) LeNet-300-100, K̄ = [3, 3, 33], (ii) LeNet-5, K̄ = [5, 5, 33], and (iii) VGG-Cifar-10, K̄ = [33, 17, 15, 11, 7, 7, 7, 5, 5, 5, 3, 3, 3, 17, 33].”