Thank you!
NLP4RE 2024 is over! We thank all participants, authors, and all others who contributed to the success of the workshop. Looking forward to meeting you all again at NLP4RE 2025!
Workshop Overview
Natural language processing (NLP) plays an essential role in several areas of software engineering (SE), and requirements engineering (RE) is no exception. Requirements are generally authored and communicated in textual form and different levels of formality, from structured (e.g., user stories) to unstructured natural language. In the last few years, the advent of massive and heterogeneous sources, such as tweets and app reviews, has attracted even more interest from the RE community, and the recent developments in large language models (LLMs) and generative AI have opened new opportunities for RE. LLMs will likely be the enabling technology for solving long-standing RE problems, such as traceability, classification, and compliance.
The main goal of NLP4RE is to represent a community- building venue for researchers who apply NLP technologies to solve RE problems and automatically support RE activities.
The NLP4RE Workshop is co-located with REFSQ'24. Check out the REFSQ'24 Conference here: https://2024.refsq.org
Contributions
The workshop welcomes contributions regarding both theory and application of NLP technologies in RE. We also encourage contributions that highlight challenges faced by industrial practitioners when dealing with requirements expressed in NL, and the experiences of academics in technology transfer.
We are interested in Tool Papers (see the Call for Papers), in which the authors provide a brief description of an NLP tool for RE, and a plan for a tool demo at the workshop.
We are also interested in Report Papers (see the Call for Papers), in which the authors provide an overview on the current and past research of their teams. These contributions do not require novelty with respect to previous work, because the main goal of the workshop is to foster discussion and networking.
Moreover, this year we encourage submissions discussing the following topics:
Large Language Models (LLMs) and RE
How traditional and recent RE tasks (e.g., issue classification, prompt engineering) can be solved with the new developments in LLMs and chat-based NLP solutions (e.g., ChatGPT, BARD, Bing Chat), and how RE can help the development of LLMs.Traditional vs. advanced NLP
Comparative analysis of automated solutions that solve RE tasks using traditional versus advanced NLP solutions (e.g., LLMs).Education
The integration of NLP4RE in the general RE educational programs, and how LLMs could change how we teach programming and other SE tasks.Ethical challenges in NLP4RE research
Studies on the impact of NLP4RE tools and techniques on humans and related ethical challenges.Replicability of NLP4RE
Studies that replicate NLP4RE tools as well as studies that survey the replicability of the NLP4RE research.Evaluation in NLP4RE
Studies on the evaluation of NLP4RE tools, including but not limited to analysis of the automation against human performance and collaborative modes combining human performance with automated solutions.
Specific topics of interest include but are not limited to:
- LLMs for RE and vice versa
- App Review analysis and classification
- Social media mining and analysis for RE
- Bug report mining and analysis for RE
- Requirements quality assurance and ambiguity
- Requirements tracing
- Requirements retrieval
- Model generation
- Test generation
- Ethics (e.g., bias, fairness, sustainability) in NLP4RE
- Bias/Fairness in NLP4RE
- NLP4RE education
- Information extraction from legal and policy documents
- Information extraction from requirements
- Dependency and relation extraction
- Domain-specific NLP for RE
- Automated requirements management
- Multi-modal requirements analysis
- Functional / Non-functional requirements categorization
- Formalization of informal requirements
- Question-answering for RE
- Summarization of requirements documents
- Speech-to-text and speech analysis in RE
- Requirements datasets
Call for Papers
Technical Papers
8 pages (plus 2 pages for references), describing novel technical solutions for the application of NLP technologies to RE-relevant problems in the topics of the workshop, or evaluation of existing solutions and NLP4RE-relevant phenomena using, e.g., experiments, and other empirically sound research strategies. This type of paper also includes case studies, interview/observational studies, and systematic literature reviews.
Experience Papers
5 to 8 pages (plus 2 pages for references), describing practical experiences in the application of NLP technologies to RE-relevant artifacts. These papers should focus on retrospective reporting and discussion of lessons learned.
Project Report Papers
5 pages (plus 1 page for references), in which the authors provide an overview of their team's past, current or upcoming projects and research. These contributions include summaries of ongoing projects (EU, national, regional), which include an NLP4RE dimension, and preliminary project ideas for which the presenters are looking for collaborators. These contributions do not require novelty with respect to previous work, and are oriented to foster discussion and networking.
Our recommended template for Report Papers can be dowloaded here.
Vision Papers
5 pages (plus 1 page for references), outlining a roadmap for research in the workshop’s topics, including industrial and research challenges based on currently available knowledge. We also encourage contributions highlighting challenges practitioners face when dealing with requirements expressed in NL, and challenges faced by academics in technology transfer studies or when applying/evaluating NLP4RE technologies in practice.
Tool Papers
5 pages (including screenshots and references), in which the authors provide a short description of an NLP-based tool for RE with screenshots and a clear plan for a demo at the workshop. These contributions do not require novelty with respect to previous work, and the authors can also showcase tools presented in past conferences and workshops. These papers will be evaluated based on the potential interest raised by the tool, and based on the clarity of the plan for the demo.
Conference-first Papers
1 page (including references), in which the authors present a paper previously presented at RE, REFSQ or ICSE. The contribution should briefly summarize the content of the original paper. These contributions are oriented to foster dissemination, and will not appear in the proceedings.
Format and requirements
Submissions should be written in English and submitted in PDF format (page size A4, single column) formatted according to the CEUR Proceedings Style:
- Overleaf CEURArt Template (please use 1-column style): https://www.overleaf.com/read/gwhxnqcghhdt
- CEUR Template Download: http://ceur-ws.org/Vol-XXX/CEURART.zip
It is required for at least one author of each accepted submission to register, attend the workshop and present their research to the workshop participants.
All papers will undergo a traditional single-blind peer-review process (3 reviews per paper) to check scientific soundness, adequacy to the workshop topics, and compliance with the required template. We plan to publish the accepted papers in the CEUR Proceedings, with ISBN number.
Keynote: Prof. Andreas Vogelsang
Prompting the Future: Integrating Decoder-Only LLMs and Requirements Engineering
Decoder-only LLMs, such as GPT, have revolutionized how we interact with artificial intelligence. Their ability to understand, generate, and manipulate language presents unprecedented opportunities and challenges across various disciplines, including Requirements Engineering (RE). This talk introduces a pioneering approach to integrating decoder-only LLMs into RE, poised to redefine the landscape of requirements elicitation, specification, and validation.
The presentation is structured into two primary segments. The first part delves into the application of decoder-only models in automating RE tasks. It explores how these models can assist in accurately capturing and specifying requirements, generating requirement documents, and automating the verification of requirements consistency and completeness. By examining case studies and current research, this section will highlight the transformative potential of decoder-only LLMs in enhancing efficiency, accuracy, and comprehensiveness in RE.
The second segment of the talk positions RE as a critical discipline for developing well-crafted prompts essential for interacting with decoder-only LLMs. It underscores the importance of precise, unambiguous, and comprehensive requirements in formulating prompts that elicit accurate and relevant responses from the models. This part will also discuss the art and science of crafting effective prompts, drawing parallels between requirements specification techniques and prompt engineering strategies.
Link to presentation slides is here.
Andreas Vogelsang is full professor for Software and Systems Engineering at the University of Cologne. He received a PhD from the Technical University of Munich. His research is at the intersection between requirements engineering, empirical software engineering, and machine learning. He has published over 100 papers in international journals and conferences such as TSE, JSS, IEEE Software, RE, and ICSE. In 2018, he was appointed as Junior-Fellow of the German Society for Informatics (GI). Further information can be obtained from https://cs.uni-koeln.de/sse.
Program: Monday, April 8th, 2024
Session 1 (9:00 - 10:35)
-
9:00 - 9:05 - Introduction
-
9:05 - 10:05 - Keynote – Prompting the Future: Integrating Decoder-Only LLMs and Requirements Engineering by Andreas Vogelsang (slides)
-
10:05 - 10:35 - T-FREX: A Transformer-based Feature Extraction Method from Mobile App Reviews by Quim Motger, Alessio Miaschi, Felice Dell'Orletta, Xavier Franch and Jordi Marco.
Break (10:35 - 11:00)
Session 2 (11:00 - 12:30)
-
11:00 - 11:30 - Requirements Classification for Smart Allocation: A Case Study in the Railway Industry by Sarmad Bashir, Muhammad Abbas and Mehrdad Saadatmand.
-
11:30 - 12:00 - Which AI Technique Is Better to Classify Requirements? An Experiment with SVM, LSTM, and ChatGPT by Abdelkarim El-Hajjami, Nicolas Fafin and Camille Salinesi.
-
12:00 - 12:30 - Automated Requirements Demarcation using Large Language Models: An Empirical Study by Kaishuo Wang, Feier Zhang and Mehrdad Sabetzadeh. (video)
Lunch (12:30 - 14:00)
Session 3 (RE4AI 2024) (14:00 - 15:30)
-
14:00 - 14:30 - Documentation of non-functional requirements for systems with machine learning components by Elma Bajraktari, Thomas Krause and Christian Kücherer
-
14:30 - 15:00 - Peeking Outside the Black-Box: AI Explainability Requirements beyond Interpretability by Jakob Droste, Hannah Deters, Ronja Fuchs and Kurt Schneider
-
15:00 - 15:30 - RE4AI Panel by Workshop Participants
Session 4 (16:00 - 17:30)
-
16:00 - 16:30 - Leveraging Knowledge Graphs for Goal Model Generation by Shahin Abdoul Soukour, William Aboucaya and Nikolaos Georgantas
-
16:30 - 17:00 - Automating Data Flow Diagram Generation from User Stories Using Large Language Models by Guntur Budi Herwanto
-
17:00 - 17:30 - Wrap-up and take away messages by Workshop Participants
Organizing Committee
For questions about the workshop, reach us via e-mail.
Sallam
Abualhaija
University of Luxembourg
(Luxembourg)
Chetan
Arora
Monash
University
(Australia)
Davide Dell'Anna
Utrecht University
(The Netherlands)
Alessio Ferrari
CNR-ISTI
(Italy)
Sepideh Ghanavati
University of Maine
(United States)
Program Committee
- Muhammad Abbas, RISE Research Institute, Sweden
- Fatma Başak Aydemir, Boğaziçi University, Turkey
- Anna Barcomb, University of Calgary, Canada
- Dan Berry, University of Waterloo, Canada
- Mitra Bokaei Hosseini, University of Texas - San Antonio, USA
- Fabiano Dalpiaz, Utrecht University, The Netherlands
- Henning Femmer, Qualicen GmbH, Germany
- Jannik Fischbach, Netlight Consulting and fortiss, Germany
- Xavier Franch, Universitat Politècnica de Catalunya, Spain
- Julian Frattini, Blekinge Institute of Technology, Sweden
- Davide Fucci, Blekinge Institute of Technology, Sweden
- Smita Ghaisas, TCS Research, India
- Eduard Groen, Fraunhofer IESE, Germany
- Jin Guo, McGill University, Canada
- Emitzá Guzmán, Vrije Universiteit Amsterdam, The Netherlands
- Frank Houdek, Daimler Ag, Germany
- Vijayanta Jain, University of Maine, USA
- Sylwia Kopczyńska, Poznan University of Technology, Poland
- Clara Lüders, University of Hamburg, Germany
- Luisa Mich, University of Trento, Italy
- Lloyd Montgomery, University of Hamburg, Germany
- Mohammad Moshirpour, University of Calgary, Canada
- Nan Niu, University of Cincinnati, USA
- Barbara Paech, University of Heidelberg, Germany
- Mehrdad Sabetzadeh, University of Ottawa, Canada
- Nicolas Sannier, University of Luxembourg, Luxembourg
- Laura Semini, University of Pisa, Italy
- Michael Unterkalmsteiner, Blekinge Institute of Technology, Sweden
- Han van der Aa, University of Mannheim, Germany
- Andreas Vogelsang, University of Cologne, Germany
- Liping Zhao, University of Manchester, UK
Steering Committee
The continuity of the workshop is guaranteed by a steering committee, which currently consists of the following people:
- Sallam Abualhaija, University of Luxembourg, Luxembourg
- Fabiano Dalpiaz, Utrecht University, The Netherlands
- Alessio Ferrari, CNR-ISTI, Italy
- Xavier Franch, Universitat Politècnica de Catalunya, Spain