Managing the Algorithm: Prompt Engineering for AI-based Systems as an Emerging Business Skill

Today, artificial intelligence (AI) is pervasive as it influences a lot of different areas in our private and working lives, for instance through AI-based conversational agents, which are applied in various usage scenarios. Enabled through the advancements of natural language processing (NLP) achieved by large language models (LLMs), AI mimics our human natural intelligence. In this context, it can lead an intelligent conversation with a human counterpart as recent examples such as ChatGPT show very impactful. Interacting with these intelligent AI-based agents through conversational dialogs is nowadays an omnipresent foundation of everyday life. LLMs offer a unique opportunity for individuals to develop and enhance their modern competencies, such as communication, reasoning, creativity, and problem-solving, by interacting with these AI systems through natural language dialogs. On top of that, creating new value with AI is a transformative competency for the future (OECD 2018). Nonetheless, generative AI and LLMs also face difficulties, such as the need for technical skills, the risk of producing wrong or meaningless outputs (oftentimes called hallucination), and the lack of reasoning needed for approaching complex problems. Thus, the improvement of communicating with AI-based systems is a critical endeavor. By using prompts, which are textual instructions and examples of the desired interactions, users can enhance the quality and relevance of the AI outputs. However, engineering prompts comes with several challenges. First, prompting in itself is a challenging task that involves active research on the direct and indirect effects of prompts on generative AI models, and lacks established workflows for prompt engineering, which requires extensive experimentation and evaluation by NLP experts to avoid undesirable AI outputs. Second, understanding the process of how AI-based systems produce outcomes based on user inputs is critical for generative AI outputs but not well understood. Third, although managing algorithms through effective prompts is an important skill, there is a critical lack of what defines prompt engineering as a business skill. Prompt engineering not only relates to understanding the technical aspects of AI-based systems but also involves the ability to apply this skill effectively and creatively in a variety of situations such as analytical thinking, strategic planning, decision-making, and negotiation in collaboration with an algorithm. The goal of this project departs from prior techno-centric research and introduces a skill-based approach to prompt engineering as the critical key for enabling individuals to manage algorithms. Thus, the main research question (RQ) is based on the question “How can prompt engineering for business skills in generative AI-based tool environments be conceptualized, designed, and evaluated?” Guided by this RQ, the project is built upon three work packages, all aiming at publications in top-tier journals and conferences.Conceptually, we draw upon a skill development perspective to overcome the aforementioned challenges and answer the main RQ. We argue that effective prompt engineering strategies allow for improved skills in the business domain. Understanding how to conceptualize prompt engineering from a skill perspective and how to design effective learning processes is a time-critical endeavor for research to provide scientific evidence in this novel research area. The project has three overall deliverables: First, we provide a conceptualization of prompt engineering skills in the business domain. Second, we design a learning process based on worked examples to embed prompt engineering strategies into higher education business teaching. Third, the project provides empirical insights into the effects of prompt strategies and contributes to research related to worked examples when considering the interaction with emerging generative AI technologies.