Skip to main content
Thesis defences

PhD Oral Exam - Ahmad Abdellatif, Software Engineering

Towards Understanding and Improving the Value of Chatbots in Software Engineering


Date & time
Friday, December 17, 2021 (all day)
Cost

This event is free

Organization

School of Graduate Studies

Contact

Dolly Grewal

Where

Online

When studying for a doctoral degree (PhD), candidates submit a thesis that provides a critical review of the current state of knowledge of the thesis subject as well as the student’s own contributions to the subject. The distinguishing criterion of doctoral graduate research is a significant and original contribution to knowledge.

Once accepted, the candidate presents the thesis orally. This oral exam is open to the public.

Abstract

Software chatbots have been around since 1966, where the first computer interacted with a human. Recently, advances in artificial intelligence and natural language processing and understanding led to the rise and widely use of chatbots in a variety of services (e.g., healthcare, e-commerce, customer service). Nowadays, chatbots have become the main conduit between humans and services. Through natural language, chatbots enable users to communicate with different services intuitively. Another reason for the increasing popularity of chatbots in several domains is their benefits in saving time, effort, and cost. These benefits and wide adoption of chatbots attract practitioners to implement chatbots that support software engineering tasks. Chatbots play an important role in various software development tasks from answering development questions to running tests and controlling services. While there are numerous chatbots and their capability of supporting software practitioners is encouraging, little is known about the development challenges and usage benefits of software engineering chatbots.

This thesis presents series of empirical studies that aim to understand the challenges of developing chatbots for the software engineering domain, highlight the value of using chatbots in software development, and proposes novel approaches to support developers at developing more efficient software engineering chatbots. More specifically, we tackle three aspects of chatbots in software engineering. First, we present an empirical study to explore the chatbot development challenges. We find that chatbot developers face several challenges that are related to chatbot integration, development, natural language understanding platforms (NLUs), user interaction, and user input.

Second, we propose a chatbot that answers software project related questions to showcase the potential of chatbots in the software development. We find that practitioners are able to complete their tasks more accurately (65.6% more completed tasks) and in less time (83.3% faster) when using chatbots compared to using conventional tools. During this work we find two critical challenges in chatbot development which are selecting an NLU model for the chatbot implementation and curating a high-quality dataset to train the NLU model.

Third, we propose guidelines and approaches to improve chatbots in the software engineering domain. First, we assess the performance of multiple widely used NLUs using representative software engineering tasks to guide chatbot developers in designing more efficient chatbots. We report a guideline for chatbot developers on the best performing NLUs for intents classification and entity extraction. Finally, we investigate an approach that combines synonyms replacement and paraphrasing techniques to augment the training dataset of SE chatbots, which helps chatbot developers create high-quality datasets for training the NLU models. We find that augmenting the dataset using the combined approach does not improve the NLU's performance for intents classification. Also, the results show that using the combined approach has a negligible effect on the NLU's confidence in its classification.

Back to top

© Concordia University