Leveraging BERT for Extractive Text Summarization on Lectures

By Derek Miller. 2019

275 citations recorded by the source corpus.

In the last two decades, automatic extractive text summarization on lectures has demonstrated to be a useful tool for collecting key phrases and sentences that best represent the content. However, many current approaches utilize dated approaches, producing sub-par outputs or requiring several hours of manual tuning to produce meaningful results. Recently, new machine learning architectures have provided mechanisms for extractive summarization through the clustering of output embeddings from deep learning models. This paper reports on the project called Lecture Summarization Service, a python based RESTful service that utilizes the BERT model for text embeddings and KMeans clustering to identify sentences closes to the centroid for summary selection. The purpose of the service was to provide students a utility that could summarize lecture content, based on their desired number of sentences. On top of the summary work, the service also includes lecture and summary management, storing content on the cloud which can be used for collaboration. While the results of utilizing BERT for extractive summarization were promising, there were still areas where the model struggled, providing feature research opportunities for further improvement.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Big Bird: Transformers for Longer Sequences (2020, background): “The model is easily adapted for a diverse range of sequence based tasks – as a seq2seq model for translation [92], summarization [66], generation [15], etc. or as a standalone encoders for sentiment analysis [84], POS tagging [65], machine reading comprehension [94], etc. – and it is known to…”
MathBERT: A Pre-Trained Model for Mathematical Formula Understanding (2021, methodology): “These pre-trained models performed well in general NLP tasks like text classiﬁcation [Devlin et al. , 2018], machine translation [Zhu et al. , 2020; Sundararaman et al. , 2019] and machine summarization [Miller, 2019; Xenouleas et al. , 2019].”
BERT: A Review of Applications in Natural Language Processing and Understanding (2021, background): “Since the nesting produces a vector representation of the text, this representation can be clustered with an arbitrary size K, which allows building a dynamic presentation of the text [53].”
Dialect Diversity in Text Summarization on Twitter (2020, methodology): “Other unsupervised algorithms, such as LexRank [24], TextRank [50], centroid-based approaches [64, 52, 19, 60] quantify the importance of a sentence based on how well it represents the collection.”