Transformers Based Text Summarization and Keyword Extraction

Abstract

The exponential growth of digital text data from online articles, academic publications, social media, and enterprise documents has made automated information extraction a critical research priority. This paper proposes an integrated Transformer-based framework for simultaneous text summarization and keyword extraction. The proposed system employs a fine-tuned BERT model for keyword extraction through masked language modeling and attention weight analysis, combined with a fine-tuned T5 transformer for abstractive summarization. A multi-task learning architecture jointly optimizes both tasks through shared encoder representations. The system achieves a ROUGE-1 score of 44.8 for summarization and an F1 score of 52.3% for keyword extraction on standard benchmarks, outperforming existing single-task and statistical baseline methods. The system is implemented as a web application supporting real-time document analysis for academic, enterprise, and content management applications.