An Exploration of Approaches to Integrating Neural Reranking Models in Multi-Stage Ranking Architectures

By Zhucheng Tu, Matt Crane, R. Sequiera, Juncheng Zhang, Jimmy J. Lin. 2017

7 citations recorded by the source corpus.

We explore different approaches to integrating a simple convolutional neural network (CNN) with the Lucene search engine in a multi-stage ranking architecture. Our models are trained using the PyTorch deep learning toolkit, which is implemented in C/C++ with a Python frontend. One obvious integration strategy is to expose the neural network directly as a service. For this, we use Apache Thrift, a software framework for building scalable cross-language services. In exploring alternative architectures, we observe that once trained, the feedforward evaluation of neural networks is quite straightforward. Therefore, we can extract the parameters of a trained CNN from PyTorch and import the model into Java, taking advantage of the Java Deeplearning4J library for feedforward evaluation. This has the advantage that the entire end-to-end system can be implemented in Java. As a third approach, we can extract the neural network from PyTorch and "compile" it into a C++ program that exposes a Thrift service. We evaluate these alternatives in terms of performance (latency and throughput) as well as ease of integration. Experiments show that feedforward evaluation of the convolutional neural network is significantly slower in Java, while the performance of the compiled C++ network does not consistently beat the PyTorch implementation.

Subfields: Computer Networks and Communications, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

Jaewoo Kang

In Information Retrieval & Search

Citations in the corpus

Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering (2018, methodology): “We use bidirectional long short term memory (Bi-LSTM) as our encoding network, and apply various ranking functions proposed by previous works (Sev-eryn and Moschitti, 2015; Tu et al., 2017).”
An anatomy for neural search engines (2019, methodology): “, by incorporating a neural network into a Lucene search engine [Tu et al., 2017], or by mixing the carefully”