The task of summarization can be categorized into two methods, extractive and abstractive summarization. Extractive approach selects highly meaningful sentences to form a summary while the abstractive approach interprets the original document and generates the summary in its own words. The task of generating a summary, whether extractive or abstractive, has been studied with different approaches such as statistical-based, graph-based, and deep-learning based approaches. Deep learning has achieved promising performance in comparison with the classical approaches and with the evolution of neural networks such as the attention network or commonly known as the Transformer architecture, there are potential areas for improving the summarization approach. The introduction of transformers and its encoder model BERT, has created advancement in the performance of many downstream tasks in NLP, including the summarization task. The objective of this thesis is to study the performance of deep learning-based models on text summarization through a series of experiments, and propose “SqueezeBERTSum”, a trained summarization model fine-tuned with the SqueezeBERT encoder which achieved competitive ROUGE scores retaining original BERT model’s performance by 98% with ~49% fewer trainable parameters.
School of Sciences and Engineering
Computer Science & Engineering Department
MS in Computer Science
Committee Member 1
Committee Member 2
Institutional Review Board (IRB) Approval
Not necessary for this item
Mohamed, S. M.
(2022).Extractive Text Summarization on Single Documents Using Deep Learning [Master's Thesis, the American University in Cairo]. AUC Knowledge Fountain.
Mohamed, Shehab Mostafa Abdel-Salam. Extractive Text Summarization on Single Documents Using Deep Learning. 2022. American University in Cairo, Master's Thesis. AUC Knowledge Fountain.
Available for download on Monday, July 25, 2022