Automatic Prompt Engineering: the Case of Requirements ClassificationPreview Paper
This program is tentative and subject to change.
Context and motivation: Large language models (LLMs) are increasingly used to address requirements engineering (RE) tasks, including trace-link recovery, legal compliance, model generation, and others. Question/problem: Most of the existing studies rely on static, non-adaptive prompting strategies that do not fully harness the models’ capabilities. Specifically, these studies overlook the potential of automatic prompting engineering (APE), a technique that allows LLMs to self-generate and fine-tune prompts to improve task performance. Principal ideas/results: This research preview aims to study the effectiveness of APE techniques in LLM-powered RE tasks. As a preliminary step, we perform a benchmarking study in which we compare APE techniques with more traditional prompting solutions for the task of requirements classification. Our results show that, on average and with some exceptions, APE outperforms the baselines. We outline research avenues, including the evaluation and tailoring of APE for other RE tasks, and considering the human-in-the-loop. Contribution: To the best of our knowledge, this is the first study to introduce APE in RE, paving the way for a deeper exploration of LLMs’ potential in this field.
This program is tentative and subject to change.
Tue 8 AprDisplayed time zone: Brussels, Copenhagen, Madrid, Paris change
16:00 - 17:10 | Research Track - Session R6 - Requirements Elicitation and Analysis IResearch Track at B3 - Teleensenyament | ||
16:00 20mTalk | Automatic Prompt Engineering: the Case of Requirements ClassificationPreview Paper Research Track Mohammad Amin Zadenoori CNR-ISTI, Liping Zhao The University of Manchester, Waad Alhoshan Al-Imam Mohammed Ibn Saud Islamic University, Alessio Ferrari CNR-ISTI | ||
16:20 20mTalk | Exploring Generative Pretrained Transformers to support Sustainability Effect IdentificationPreview Paper Research Track Barbara Paech Heidelberg University, Peter Kaiser University of Applied Science Mannheim, Peter Bambazek Johannes Kepler University Linz, Iris Groher Johannes Kepler University Linz, Norbert Seyff University of Applied Sciences and Arts Northwestern Switzerland FHNW | ||
16:40 30mTalk | ReqRAG: Enhancing Software Release Management through Retrieval-Augmented LLMs: An Industrial StudyEvaluation Paper Research Track Md Saleh Ibtasham Alstom, Sarmad Bashir RISE Research Institutes of Sweden, Muhammad Abbas Khan RISE Research Institutes of Sweden, Zulqarnain Haider Alstom, Mehrdad Saadatmand RISE Research Institutes of Sweden, Antonio Cicchetti Mälardalen University Pre-print |