Keynote: Dr. Fabiano Dalpiaz
On the quest for more credible results in ML4SE research
Automated classifiers are increasingly used in software engineering for labeling previously unseen SE data. In the NLPRE community, researchers have proposed automated classifiers that predict if a requirement pertains to a certain quality characteristic, whether an app review represents a bug or a feature request, and more. We are accustomed to reading papers that present impressive results, with precision and recall often above 90%. But are those results a credible representation of the classifier's performance in operational environments? In this talk, I will describe the ECSER (Evaluating Classifiers in Software Engineering Research) pipeline, which assists researchers by suggesting a series of steps for the rigorous execution and evaluation of automated classification research in SE. Through excerpts from conducted replication studies, I will show how ECSER allows to draw more nuanced conclusions, sometimes contradicting the claims of the original studies. ECSER can be seen as a call to action for increasing the maturity of the NLP4RE community. Link to presentation slides is here
Fabiano Dalpiaz is an associate professor in the Department of Information and Computing Sciences at Utrecht University in the Netherlands. He is principal investigator in Utrecht University's Requirements Engineering lab. In his research, Fabiano blends artificial intelligence with information visualization to increase the quality of the requirements engineering process and artifacts, with the aim of delivering higher-quality software. His research is often validated in-vivo through collaborations with the software industry. He acts and acted as program co-chair of RE 2023, REFSQ 2021, RCIS 2020, and the RE@Next! track of RE 2021. He was the organization chair of the REFSQ 2018 conference, and he is an associate editor for the Requirements Engineering Journal and the Business & Information Systems Engineering Journal. He often serves on the program committee of conferences such as RE, CAiSE, REFSQ, ICSE, and AAMAS. You can find more about Fabiano at his website.
