Pre-trained model with numerical encoding and extension vocabulary for question answering in biomedical domain

Abstract

Extracting and translating information from vast amounts of biomedical literature takes considerable time and energy. In particular, biomedical literature includes various types of terminology and numerical facts, which makes it difficult for humans to interpret. Accordingly, we propose a deep learning-based QA model in which numerical encoding and extension vocabulary are added to solve these problems. The proposed QA model shows excellent performance even with further pre-training using a small amount of biomedical dataset.