Seminar by Weiyi Shang (Queen's University)

Speaker: Weiyi Shang
                Queen's University

Title:  Log Engineering: Towards Systematic Mining of Logs to Support the
           Development of Ultra-large Scale Systems

Date: Thursday, February 12th, 2015

Time: 10:30AM - 12PM

Place: EV2.184

ABSTRACT

Logs are generated from statements inserted into the code during development to draw the attention of system operators and developers to important run--time events. Such statements reflect the rich experience of system experts. The rich content of logs has led to a new market for log management applications that assist in storing, querying and analyzing logs. Moreover, recent research has demonstrated the importance of logs in understanding and improving software systems. However, developers often
treat logs as textual data. We believe that logs have much more potential in assisting developers. Therefore, we propose Log Engineering to systematically leverage logs in order to support the development of ultra--large-scale systems.

To better understand the current practice of leveraging logs, we study the challenge of understanding logs and study the evolution of logs. We find that knowledge derived from development repositories, such as issue reports, can assist in understanding logs. We also find that logs co--evolve with the code, and that changes to logs are often made without considering the needs of Log Processing Apps that surround the software system. These findings highlight the need for better documentation and tracking approaches for logs.

We then propose log mining approaches to assist the development of systems. We first find that logging characteristics provide strong indicators of defect--prone source code files. Hence, efforts should focus on the code with large amounts of logging statements or their churn. Finally, we present a log mining approach to assist in verifying the deployment of Big Data Analytics applications.

BIO

Weiyi Shang is a Post Doctoral Fellow in the Software Analysis and Intelligence Lab at Queen's University (Canada). He has received his Ph.D. and M.Sc. degrees from Queen's University (Canada) and he obtained B.Eng. from Harbin Institute of Technology. His research interests include big data software engineering, software engineering for ultra--large--scale systems, software log mining, empirical software engineering, and software performance engineering. His work has been published at premier venues such as ICSE, ASE, ICSME , MSR and WCRE, as well as in major journals such as EMSE, JSS, JSEP and SCP. His work has won premium awards, such as SIGSOFT Distinguished paper award at ICSE 2013 and best paper award at WCRE 2011. His industrial experience includes helping improve quality and performance of ultra--large-scale systems in BlackBerry. Early tools and techniques developed by him are already integrated into products used by millions of users worldwide.