Options
Managing the Algorithm: Prompt Engineering for AI-based Systems as an Emerging Business Skill
Type
fundamental research project
Start Date
April 2024
End Date
March 2025
Description
Today, artificial intelligence (AI) is pervasive as it influences a lot of different areas in our private and working lives, for instance through AI-based conversational agents, which are applied in various usage scenarios. Enabled through the advancements of natural language processing (NLP) achieved by large language models (LLMs), AI mimics our human natural intelligence. In this context, it can lead an intelligent conversation with a human counterpart as recent examples such as ChatGPT show very impactful. Interacting with these intelligent AI-based agents through conversational dialogs is nowadays an omnipresent foundation of everyday life. LLMs offer a unique opportunity for individuals to develop and enhance their modern competencies, such as communication, reasoning, creativity, and problem-solving, by interacting with these AI systems through natural language dialogs. On top of that, creating new value with AI is a transformative competency for the future (OECD 2018). Nonetheless, generative AI and LLMs also face difficulties, such as the need for technical skills, the risk of producing wrong or meaningless outputs (oftentimes called hallucination), and the lack of reasoning needed for approaching complex problems. Thus, the improvement of communicating with AI-based systems is a critical endeavor. By using prompts, which are textual instructions and examples of the desired interactions, users can enhance the quality and relevance of the AI outputs. However, engineering prompts comes with several challenges. First, prompting in itself is a challenging task that involves active research on the direct and indirect effects of prompts on generative AI models, and lacks established workflows for prompt engineering, which requires extensive experimentation and evaluation by NLP experts to avoid undesirable AI outputs. Second, understanding the process of how AI-based systems produce outcomes based on user inputs is critical for generative AI outputs but not well understood. Third, although managing algorithms through effective prompts is an important skill, there is a critical lack of what defines prompt engineering as a business skill. Prompt engineering not only relates to understanding the technical aspects of AI-based systems but also involves the ability to apply this skill effectively and creatively in a variety of situations such as analytical thinking, strategic planning, decision-making, and negotiation in collaboration with an algorithm. The goal of this project departs from prior techno-centric research and introduces a skill-based approach to prompt engineering as the critical key for enabling individuals to manage algorithms. Thus, the main research question (RQ) is based on the question “How can prompt engineering for business skills in generative AI-based tool environments be conceptualized, designed, and evaluated?” Guided by this RQ, the project is built upon three work packages, all aiming at publications in top-tier journals and conferences.Conceptually, we draw upon a skill development perspective to overcome the aforementioned challenges and answer the main RQ. We argue that effective prompt engineering strategies allow for improved skills in the business domain. Understanding how to conceptualize prompt engineering from a skill perspective and how to design effective learning processes is a time-critical endeavor for research to provide scientific evidence in this novel research area. The project has three overall deliverables: First, we provide a conceptualization of prompt engineering skills in the business domain. Second, we design a learning process based on worked examples to embed prompt engineering strategies into higher education business teaching. Third, the project provides empirical insights into the effects of prompt strategies and contributes to research related to worked examples when considering the interaction with emerging generative AI technologies.
Leader contributor(s)
Funder(s)
Topic(s)
digital learning
generative artificial intelligence
management education
business skills
2 results
Now showing
1 - 2 of 2
-
PublicationWorked Examples to Facilitate the Development of Prompt Engineering Skills( 2024)
;Antonia Tolzin ;Nils KnothThis paper explores the evolving field of prompt engineering in Artificial Intelligence (AI), with a focus on Large Language Models (LLMs). As LLMs exhibit remarkable potential in various educational domains, their effective use requires adept prompt engineering skills. We introduce a skill-based approach to prompt engineering and explicitly investigate the impact of using worked examples to facilitate prompt engineering skills among students interacting with LLMs. We propose hypotheses linking prompt engineering, worked examples, and perceived anthropomorphism to the quality of LLM output. Our initial findings support the critical relationship between proficient prompt engineering and the resulting output quality of LLMs. Subsequent phases will further explore the role of worked examples in prompt engineering, aiming to provide practical recommendations for educational improvement and industry application. Additionally, this research aims to shed light on the responsible utilization of LLMs in education and contribute insights to educational practice, research, and organizational development.Type: conference paperJournal: Thirty-Second European Conference on Information Systems (ECIS 2024) -
PublicationAI literacy and its implications for prompt engineering strategies( 2024)
;Nils Knoth ;Antonia TolzinJan Marco LeimeisterArtificial intelligence technologies are rapidly advancing. As part of this development, large language models (LLMs) are increasingly being used when humans interact with systems based on artificial intelligence (AI), posing both new opportunities and challenges. When interacting with LLM-based AI system in a goal-directed manner, prompt engineering has evolved as a skill of formulating precise and well-structured instructions to elicit desired responses or information from the LLM, optimizing the effectiveness of the interaction. However, research on the perspectives of non-experts using LLM-based AI systems through prompt engineering and on how AI literacy affects prompting behavior is lacking. This aspect is particularly important when considering the implications of LLMs in the context of higher education. In this present study, we address this issue, introduce a skill-based approach to prompt engineering, and explicitly consider the role of non-experts' AI literacy (students) in their prompt engineering skills. We also provide qualitative insights into students’ intuitive behaviors towards LLM-based AI systems. The results show that higher-quality prompt engineering skills predict the quality of LLM output, suggesting that prompt engineering is indeed a required skill for the goal-directed use of generative AI tools. In addition, the results show that certain aspects of AI literacy can play a role in higher quality prompt engineering and targeted adaptation of LLMs within education. We, therefore, argue for the integration of AI educational content into current curricula to enable a hybrid intelligent society in which students can effectively use generative AI tools such as ChatGPT.Type: journal-articleJournal: Computers and Education: Artificial IntelligenceVolume: 6Issue: JuneScopus© Citations 5