Convolutional neural network compression for natural language processing

By Krzysztof Wróbel, Marcin Pietron, Maciej Wielgosz, Michał Karwatowski, Kazimierz Wiatr. 2018

9 citations recorded by the source corpus.

Convolutional neural networks are modern models that are very efficient in many classification tasks. They were originally created for image processing purposes. Then some trials were performed to use them in different domains like natural language processing. The artificial intelligence systems (like humanoid robots) are very often based on embedded systems with constraints on memory, power consumption etc. Therefore convolutional neural network because of its memory capacity should be reduced to be mapped to given hardware. In this paper, results are presented of compressing the efficient convolutional neural networks for sentiment analysis. The main steps are quantization and pruning processes. The method responsible for mapping compressed network to FPGA and results of this implementation are presented. The described simulations showed that 5-bit width is enough to have no drop in accuracy from floating point version of the network. Additionally, significant memory footprint reduction was achieved (from 85% up to 93%).

Subfields: Artificial Intelligence, Natural Language Processing, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Efficiency & Compression

Citations in the corpus

The Right Tool for the Job: Matching Model and Instance Complexities (2020, methodology): “both speeds up numerical operations and reduces model size (Wróbel et al., 2018; Shen et al., 2019; Zafrir et al., 2019).”