Master Thesis Defense: Hayda Marcia Soares Almeida

Speaker: Hayda Marcia Soares Almeida

Supervisors: Drs. L. Kosseim Marie-Jean Meurs

Examining Committee: Drs. G. Butler, B. Jaumard, C. Constantinides (Chair)

Title: A Supervised Learning Approach for Imbalanced Text Classification of Biomedical Literature Triage

Date: Tuesday, April 7, 2015

Time: 11:00 a.m.

Place: EV 3.309

ABSTRACT

This thesis presents the development of a machine learning system, called mycoSORT , for supporting the first step of the biological literature manual curation process, called triage. The manual triage of documents is very demanding, as researchers usually face the time-consuming and error-prone task of screening a large amount of data to identify relevant information. After querying scientific databases for keywords related to a specific subject, researchers generally find a long list of retrieved results that has to be carefully analyzed to identify only a few documents that show a potential of being relevant to the topic. Such an analysis represents a severe bottleneck in the knowledge discovery and decision-making processes in scientific research. Hence, biocurators could greatly benefit from an automatic support when performing the triage task. In order to support the triage of scientific documents, we have used a corpus of document instances manually labeled by biocurators as "selected" or "rejected", with regards to their potential to indicate relevant information about fungal enzymes. This document collection is characterized by being large, since many results are retrieved and analysed to finally identify potential candidate documents; and also highly imbalanced, concerning the distribution of instances per relevance: the great majority of documents are labeled as rejected, while only a very small portion are labeled as selected. Using this dataset, we studied the design of a classification model to identify the most discriminative features to automate the triage of scientific literature and to tackle the imbalance between the two classes of documents. To identify the most suitable model, we performed a study of 324 classification models, which demonstrated the results of using 9 different data under sampling factors, 4 sets of features and the evaluation of 2 feature selection methods as well as 3 machine learning algorithms. Our results demonstrated that the use of an under sampling technique is effective to handle imbalanced datasets and also help manage large document collections. We also found that the combination of under sampling and feature selection using Odds Ratio can improve the performance of our classification model. Finally, our results demonstrated that the best fitting model to support the triage of scientific documents is composed by domain relevant features, filtered by Odds Ratio scores, the use of dataset under sampling and the Logistic Model Trees algorithm.