Drs. T. Glatard, C. Poullis (Chair)
Title: Neural Network Approaches to Medical Toponym Recognition
Date: Friday, April 3, 2020
Place: Online via Zoom teleconference (more details to be accounced)
Toponym identification, or place name recognition, within epidemiology articles is a crucial task for phylogeographers, as it allows them to analyze the development, spread, and migration of viruses. Although public databases, such as GenBank, contain the geographical information, this information is typically restricted to country and state levels. In order to identify more fine-grained localization information, epidemiologists need to read relevant scientific articles and manually extract place name mentions.
In this thesis, we investigate the use of various neural network architectures and language representations to automatically segment and label toponyms within biomedical texts. We demonstrate how our language model based toponym recognizer relying on transformer architecture can achieve state-of-the-art performance. This model uses pre-trained BERT as the backbone and fine-tunes on two domains of datasets (general articles and medical articles) in order to measure the generalizability of the approach and cross-domain transfer learning.
Using BERT as the backbone of the model, resulted in a large highly parameterized model (340M parameters). In order to obtain a light model architecture we experimented with parameter pruning techniques, specifically we experimented with Lottery Ticket Hypothesis (LTH), however since this pruning technique does not scale well to highly parametrized models and loses stability, we proposed a novel technique to augment LTH in order to increase the scalability and stability of this technique and tested our technique on toponym identification task.
The evaluation of the model was performed using a collection of 105 epidemiology articles from PubMed Central. Our proposed model significantly improves the state-of- the-art model by achieving an F-measure of 90.85% compared to 89.13%.