BERT BiLSTM-Attention Similarity Model
American University in Cairo
Second Author's Department
Computer Science & Engineering Department
Find in your Library
2021 IEEE International Conference on Artificial Intelligence and Computer Applications, ICAICA 2021
Semantic similarity models are a core part of many of the applications of natural language processing (NLP) that we may be encountering daily, which makes them an important research topic. In particular, Question Answering Systems are one of the important applications that utilize semantic similarity models. This paper aims to propose a new architecture that improves the accuracy of calculating the similarity between questions. We are proposing the BERT BiLSTM-Attention Similarity Model. The model uses BERT as an embedding layer to convert the questions to their respective embeddings, and uses BiLSTM-Attention for feature extraction, giving more weight to important parts in the embeddings. The function of one over the exponential function of the Manhattan distance is used to calculate the semantic similarity score. The model achieves an accuracy of 84.45% in determining whether two questions from the Quora duplicate dataset are similar or not.
(2021). BERT BiLSTM-Attention Similarity Model. 2021 IEEE International Conference on Artificial Intelligence and Computer Applications, ICAICA 2021, 366–371.
Aboutaleb, Ahmed, et al.
"BERT BiLSTM-Attention Similarity Model." 2021 IEEE International Conference on Artificial Intelligence and Computer Applications, ICAICA 2021, 2021, pp. 366–371.