Why Incorporating a Platform-Intermediary can Increase Crowdsourcees’ Engagement

2.1 Crowdsourcing

The fundamental idea of crowdsourcing is that a crowdsourcer (e.g., a company) proposes to an undefined group of contributors (i.e., individuals), henceforth called crowdsourcees, the voluntary undertaking of a task presented in an open call (Blohm et al. 2013). The ensuing interaction process unfolds over IT-based crowdsourcing platforms, owned and managed by the crowdsourcer himself or provided by an intermediary. There are several forms of crowdsourcing and platform types, which can be categorized according to their specific crowdsourcing function. Vuković (2009) differentiates between four types of functions, representing the product or service lifecycle-part that is crowdsourced by the project: design and innovation, development and testing, marketing and sales, or support. While enterprises that crowdsource design processes rather benefit from the crowd’s innovation power (e.g., Threadless.com), those who conduct marketing-related projects rely on its predictive power (e.g., Predictify). Other forms of crowdsourcing make use of the simple mass and diversity of people that can be reached. In the case of crowdtesting (or crowdsourced software testing), either experts (e.g., for complex tasks) or potential end-users (for micro tasks) are approached to test, e.g., applications or webpages regarding their functions, usability, or interface-design (e.g., Testbirds.com).

Independent of the type and ultimate output objective of a crowdsourcing project, crowdsourcer and crowdsourcees engage in a participative, IT-mediated interaction process to create mutual benefit (Estellés-Arolas and González-Ladrón-De-Guevara 2012). Thus, in a broader context, this process relates to the macro-construct of value co-creation (Storbacka et al. 2016), by which organizations open themselves up to the co-creation efforts of external individuals (Zwass 2010). For the crowdsourcing enterprise, value and project success is multidimensional (Blohm et al. 2013). First, it may involve solving a crowdsourcer’s problem that cannot be satisfactorily solved in-house (Blohm et al. 2016). Yet, the generated value may go beyond problem solving, and indirect benefits like enhanced brand visibility and reputation are desired side-effects of a successful campaign (Ye and Kankanhalli 2015). Similarly, for crowdsourcees the benefit of participation can be of economic nature (i.e., a reward or remuneration) or may satisfy other needs, like social recognition, self-esteem, skill development, or entertainment (Estellés-Arolas and González-Ladrón-De-Guevara 2012). In the case of financial remuneration, the initial motivation to participate is rather extrinsic, and further factors, experienced throughout the interaction process, may influence contributions and commitment towards the crowdsourcer.

Hence, it can be argued that crowdsourcees take on several roles throughout the value co-creation process, ranging from a platform-mediated worker (e.g., solving a problem and obtaining a reward in return), to a community member (e.g., fostering interaction and enjoying social exchange) and to becoming a (potential) consumer and influencer (e.g., learning about offerings and spreading the word). Accordingly, this paper argues that value goes beyond simple transactions of resources, and success needs to be defined more holistically. In this context, Storbacka et al. (2016) illustrate that engagement is the micro-foundation of value co-creation. Without engagement, no resource integration can occur and no value can be co-created. It is argued that the conceptual and physical context determines why, when and how an individual engages. Hence, we assume that crowdsourcees’ engagement is inseparably linked to the perceived co-creation experience within the interaction process. Some authors have emphasized the need for researching crowdsourcing from an experience-based perspective (Vuković 2009; Füller et al. 2009; Pedersen et al. 2013) and studying the topic of engagement (Zwass 2010). De Vreede et al. (2013) explain initial engagement, suggesting personal interest, goal clarity, and motivation as antecedents. Sun et al. (2012) found that task-complexity and self-efficacy are drivers of sustained participation. Moreover, a participant engagement index for crowdsourcing has been proposed, based on the characteristics of contributions (Nguyen et al. 2015). Riedl et al. (2013) found a positive impact of platform-design choices on the crowdsourcee’s attitude, while process satisfaction and a sense of virtual community was found to impact affective commitment (Schulten and Schaefer 2015). Lastly, it was observed that crowdsourcing participation is perceived as a hedonic experience, enhancing brand image (Djelassi and Decoopman 2013). While interest is growing and first research attempts offer insights into specific types of stimuli and potential measures for engagement, no study provides a holistic engagement definition for the context of crowdsourcing and systematically examines the underlying mechanisms of the engagement process throughout the IT-mediated journey from a crowdsourcee’s perspective.

2.2 Crowdsourcing Intermediaries

As already mentioned, crowdsourcers can set up their own crowdsourcing platform and processes (e.g., My Starbucks Idea) or they can refer to intermediaries (e.g., Innocentive or Testbirds) that provide a technical infrastructure and access to a crowd. In this sense, they either serve as market places, offering a virtual platform where crowdsourcer and crowdsourcees simply interact for the purpose of value co-creation, or they even act as mediators who offer additional services such as task specification, crowd acquisition, and evaluation of results to support the end-to-end crowdsourcing process (Zogaj et al. 2014).

On the one hand, these types of crowdsourcing intermediaries can be considered as brokers, insuring that crowdsourcing enterprises do not only connect with a suitable crowd by providing the necessary skills and resources, but also shift risks, efforts and overhead related to crowd and process management (Zogaj et al. 2014). On the other hand, if the crowd is considered a form of valuable resource and social capital which often consists of actual and potential customers or end-users of the crowdsourcing enterprise, handing over full power to an intermediary may also bear some risks. As outlined by Zogaj et al. (2014), crowdsourcing intermediaries may face three main challenges, depending on their services: managing the process, the crowd, and the technology. In all three areas, mistakes can have major impact on the crowdsourcing experience of participants and their engagement throughout the interaction process. Hence, the crowdsourcing enterprise may not only risk to lose valuable contributors during or after the interaction due to perceived negative experiences out of their control, but also their reputation if undesired interactions are transferred to the brand’s image (Gebauer et al. 2013). Additionally, the crowdsourcing enterprise may miss a promising opportunity to directly interact and connect with the crowd, thereby stimulating overall engagement, which could create extra value, e.g., in form of positive word of mouth, further knowledge contributions, or repeated participation (Nambisan and Nambisan 2008). By deploying a mediator, one may assume that he absorbs all the crowd’s attention and commitment, comparable to the role of an employer, while the crowdsourcer is only perceived as an ordering party, defining the task and gathering the contributions. Positive impressions may be attributed to the intermediary, rather than to the crowdsourcer as the initiating party.

First studies dealt with the management challenges from an intermediary perspective (Zogaj et al. 2014), especially an intermediary’s ability to solve problems (Terwiesch and Xu 2008) and support the innovation process (Feller et al. 2012) or the assessment of the architectural structure (Colombo et al. 2013) and platform-typification (Kaganer et al. 2013). However the issue of how and why crowdsourcing intermediaries influence crowdsourcees’ experiences and associated psychological and behavioral responses to the crowdsourcer within and after the interaction process has not been addressed by research yet. Hence, this paper studies the intermediary’s impact on the engagement process of crowdsourcees towards the crowdsourcer. In the following section an initial overview of the engagement concept is provided.

2.3 The Concept of Engagement

Engagement is a broad field, which is discussed, e.g., in the Organizational Behavior, Marketing, and Information Systems (IS) literature. Due to the interdisciplinary character of crowdsourcing and the diverse roles of crowdsourcees (e.g., a platform-mediated worker, community member, or consumer and influencer), different perspectives of engagement seem suitable.

First, from an IS-perspective, user engagement is defined as a situational or enduring emotional, cognitive and behavioral connection between a user and a (technological) resource (Attfield et al. 2011), based on a user experience that extends beyond pure usability (O’Brien and Toms 2008). A vague description of the user engagement process is offered, consisting of a point of engagement, a period of sustained engagement, disengagement, and (possibly) reengagement (O’Brien and Toms 2008). Behavioral responses (e.g., technology use, length, return) can be observed through interaction patterns (Attfield et al. 2011). Second, employee or work engagement commonly refers to a psychological state that is above and beyond simple satisfaction, as well as a behavioral response that includes, e.g., innovative behaviors, proactive contribution, and over-fulfillment of task (Macey and Schneider 2008). Third, community engagement discusses the identification and interaction of community members within the group (Algesheimer et al. 2005). Brodie et al. (2013) identified learning, sharing, advocating, socializing and co-developing as relevant behavioral sub-processes. Lastly, the concept of consumer engagement is defined as a psychological state that occurs by virtue of interactive, co-creative customer experiences with a focal agent (Brodie et al. 2011), usually followed by behavioral responses in form of referral or consumption (Kumar et al. 2010; Van Doorn et al. 2010). In this context, customer experience is related to the internal, subjective perception of interactions throughout the customer journey (Johnston and Kong 2011).

Although those definitions differ in terms of the engagement object (i.e., a resource/technology, an employer, a community or a company) and resulting behavior (i.e., use, contribute, interact or consume) the underlying understanding of engagement is very similar. For the purposes of this study, engagement is defined as a dynamic, iterative process by which a specific type of psychological state, desired by the engagement object (e.g., an enterprise), develops among engagement subjects (i.e., an individual), resulting in value-contributions for both parties. Despite of its potentially dynamic nature, the psychological end-state is regarded as a relatively pervasive and persistent (Wefald and Downey 2009), positive affective-cognitive (Hollebeek 2011b) state of mind. Active participation in the creation of an offering is widely assumed to be a central antecedent of engagement formation (Brodie et al. 2011, 2013; Vivek et al. 2012; Kumar et al. 2010). In the subsequent section each part of the here defined engagement process is assessed in detail.

3.1 Conceptualizing the Process of Engagement

Based on the above mentioned summary of the engagement concept and the provided working definition of the engagement process, a guiding model is developed for the purpose of this study (see Fig. 1). It can be summarized as a four-step process model from a subject’s perspective. In the first step, perceived interaction points, so-called stimuli, and prior experiences serve as input factors (A). In a second step, these input factors stimulate a subject’s cognitive, emotional and behavioral experience dimension, initiating an experience evaluation process and resulting in several intermediate satisfaction responses (B). The sum of all intermediate experience evaluations result in a final commitment state (C) and related behavioral consequences (D) as process outcomes. Each step is elaborated in more detail and grounded in substantial theory.

Open image in new window

3.1.1 Process Step A

First of all, based on the logic of the model of Kano et al. (1984), we assume that perceived stimuli throughout the interaction process between subject and entity can take on different roles within the subsequent process of engagement, depending on their categorization from an attribute-perspective (see Fig. 2 for an illustration). The model, routed in the fields of marketing and product lifecycle management, is used to explain in how far product or service attributes (later referred to as stimuli) may lead to different satisfaction levels, depending on a subject experience perception and related expectations (Chen and Chuang 2008). Firstly, Kano et al. (1984) suggest that some attributes, often called differentiating attributes, are explicitly demanded by the subject, and that satisfaction is assumed to be proportional to the level of fulfillment. That means the higher the level of fulfillment (i.e., beyond expectations), the higher the subject’s satisfaction and vice versa. Secondly, Kano et al. (1984) advocate that so-called attractive attributes have the potential to lead to very high satisfaction as they are neither explicitly expressed nor expected by the subject. Lastly, Kano et al. (1984) introduce the basic requirements that are simply needed for a product or service to perform. Those basic attributes have the potential to only foster a state of fulfillment or dissatisfaction (i.e., no over-fulfillment is possible), as their performance is simply taken for granted and clear expectations exist. However, no over-fulfillment of those attributes can be expected. Consequently, from a high level view, all perceived stimuli that serve as potential input factors in the process of engagement can be related to one of those attribute-categories, initiating a specific level of satisfaction-generation. Besides, it is also assumed that prior experiences with those (or similar) stimuli influence the experience evaluations, as familiarity is strongly related to the expectations a subject has towards a specific attribute (Bowden 2009). Hence, prior experience as a relevant input factor is also added to the process model. Nevertheless, initial involvement (i.e., a subject’s personal interest, relevance, or value of something), originally supposed to enhance positive judgements, was not found to be an influencing factor for successive experience evaluations and satisfaction responses in prior studies (Mano and Oliver 1993; Oliver and De Sarbo 1989). Hence, it is intentionally omitted as an input variable for now. For the purpose of this study, in the subsequent process step satisfaction generation as a substantial part of the engagement process is assessed and outlined in more detail.

Open image in new window

3.1.2 Process Step B

Within the multidimensional perspective of engagement, researchers agree on the observation that the processing of stimuli has a cognitive, emotional and behavioral dimension (Brodie et al. 2013; Hollebeek 2011a; Kahn 1990). The cognitive dimension can be interpreted as a more passive state of immersion and absorption (e.g., being focused and stimulated) (Hollebeek 2011a; Hollebeek et al. 2014) or a more active state of cognitive processing to expedite comprehension (e.g., reasoning, learning or decision making) (Mollen and Wilson 2010). The emotional dimension relates to the feelings, activated by an experience (e.g., happiness). Based on the cognitive and emotional perception, a behavioral response towards a specific stimulus (e.g., continue interaction) may be expressed within the process. Addressed dimensions regarding a perceived stimulus are assumed to be evaluated by the subject, and an intermediate state is generated, happening unconsciously (Bowden 2009; Sashi 2012). As already suggested by Kano et al. (1984), such an intermediate state can be defined as a specific satisfaction level, which may change with each subsequent stimulus experience (Verhoef 2003; Oliver 1993). From the consumer and organizational behavior literature, satisfaction is generally known as a factor influencing loyalty behavior and thus creates additional value for the firm (Hallowell 1996; Abraham 2012). While several researchers agree on its relevance within the engagement process, like Bowden (2009), Hollebeek (2011b), Sashi (2012), and Wefald and Downey (2009), they question its sufficiency and assume that other, stronger mechanisms are operating. However, this depends on the choice of the satisfaction definition that is used, as several have appeared in the past years. Some of them are unidimensional, based on a solely cognitive evaluation as proposed by the famous expectancy disconfirmation model (Oliver 1980), on which also the Kano-model builds. The concept of satisfaction applied in this paper is a more comprehensive one that allows for differentiating between more levels of satisfaction which are relevant for truly understanding the engagement process. It is a function of both, cognition and affect, as suggested by the two-appraisal model of Oliver and De Sarbo (1989). Based on their model, cognitively perceived disconfirmation between an expectation and perception may lead to positive or negative emotional arousal as a satisfaction response to the experience. Generally, three categories of events can be differentiated: (1) perceptions in a confirmation region, in which deviations from expectations are considered normal, without any emotional arousal; (2) plausible but infrequent disconfirming perceptions that are noted as unusual, arousing some emotions; (3) highly unexpected deviations, evoking disconfirmation and high emotional arousal due to surprise (Oliver et al. 1997; Oliver and De Sarbo 1989). However, even without expectation disconfirmation, low to high emotional arousal may occur, e.g., if an experience is novel and no clear expectations exist (i.e., an expected unexpectedness) (Oliver and De Sarbo 1989). In this context, Briggs et al. (2008) offer a finer grained definition of the satisfaction response in an IS-context. They describe it as a valenced affective arousal continuum, reaching from not-aroused to aroused, and the valence characterizes the level of arousal as positive or negative, while not-aroused describes a neutral, rather cognitive state. A switch of valence from positive to negative (or vice versa) may occur without passing the neutral state (Briggs et al. 2008). This is a relevant insight for the process perspective of engagement, in which several stimuli throughout an interaction journey are perceived and evaluated consecutively. Thus, individual satisfaction responses may not be explained independently but relate to preceding ones, while high emotional arousal may have a dominant impact on the overall experience evaluation and the end-state. An illustration of the emotional arousal continuum is presented in Fig. 3, based on the descriptions of Briggs et al. (2008), extended by means of the specific satisfaction levels and terms from Mano and Oliver (1993) and Oliver and Swan (1989). Those five ascending types of positively (i.e., contentment, pleasure, delight, elation, ecstasy) and negatively (i.e., boredom, displeasure, disappointment, frustration, outrage) valanced satisfaction responses are used for the detailed assessment in later sections, for which reason they are numbered here. Consequently, combining the multi-dimensional logic of the satisfaction concept with the emotional arousal continuum offers a suitable tool for assessing the satisfaction responses within the engagement process and for identifying the role of specific stimuli.

Open image in new window

3.2 Application to the Topic of Crowdsourcing

Independent of the crowdsourcer’s original intention, performing a crowdsourcing initiative creates an experience that may foster engagement among crowdsourcees. The crowdsourcing experience in this paper is defined as a crowdsourcee’s internal and subjective perception of the end-to-end, IT-mediated interaction process, resulting in a psychological state. It is an online experience, in which perceived stimuli can be found in the pre-participation- (e.g., invitation receipt), participation- (e.g., task solving), and post-participation phase (e.g., reward receipt). Due to its participative character, the underlying assumption is that crowdsourcing generally has the potential to generate high levels of engagement towards the crowdsourcing enterprise among participants. Depending on the set up, crowdsourcees (i.e., the engagement subject) may engage with the crowdsourcer directly or via an intermediary (i.e., the engagement objects). They can have varying degrees of familiarity concerning the objects (e.g., prior crowdsourcing or customer experiences), influencing their expectations and experience evaluation. Henceforth, the crowdsourcee’s engagement process is conceptualized as a psychological process that models satisfaction response levels as generative mechanisms through which a crowdsourcee develops calculative and affective commitment based on perceived stimuli and prior experiences, resulting in diverse value contributions.

First, referring to step A and B in the process model, we assume that the cognitive and emotional dimension can be addressed due to a diverse range of potentially attractive and differentiating attributes, leading to different satisfaction response levels. From an IS-perspective, a system’s attributes like novelty, variety, aesthetics (affective or sensory appeal), and fun have the potential to arouse medium to high levels of emotions (Attfield et al. 2011; O’Brien and Toms 2008). In the crowdsourcing context this could be referred to an attractive and fun-providing crowdsourcing platform. In addition, the organizational behavior literature discusses some task attributes (e.g., entertaining or challenging), a subject’s identification with it (e.g., good skill-task fit), and specific rewards (e.g., unexpected) as potential generators of positive affect (Macey and Schneider 2008). This may relate to a fun-providing task or a surprise benefit (e.g., a gift, status upgrade) in the crowdsourcing context. Moreover, according to community research, interaction with the crowd may stimulate a sense of group belonging that is known to foster intense positive feelings (Algesheimer et al. 2005). Lastly, from consumer behavior research we know that personal and close interactions between buyers and sellers (e.g., in co-creative set ups) can have a positive effect on demand and word of mouth, due to the buyers’ desire for recognition and appreciation (Mustak et al. 2013). This may be transferred to crowdsourcer-to-crowd interaction throughout the task-solving process (e.g., compliment, query, or support), resulting in positive arousal or relief.

Subsequently, referring to step C in the process model, a state of calculative commitment may develop if a clear utility of participation is seen. Additionally, a state of affective commitment may develop when emotional arousal has been generated. Resulting direct and indirect behavioral value contributions (step D) towards the crowdsourcer and intermediary may lead to repeated participation, virtual or direct word of mouth, referral behavior, further voluntary knowledge or feedback contributions (exceeding the scope of the original task), as well as consumption activities (buying/using something from the crowdsourcer).

Moreover, when including a crowdsourcing intermediary, specific stimuli like the platform, communication with the crowd, and reward transaction may be outsourced to the intermediary, depending on the service agreement. It needs to be investigated (1) which stimuli-related experiences play a major role in the engagement process, and (2) how crowdsourcees relate them to the crowdsourcer or intermediary. This will help to identify the risks and values of incorporating an intermediary.

4.1 Case Selection

The first case (A) is initiated by a leading insurance company from Switzerland (hereafter called InsureCorp) together with a leading Swiss crowdsourcing intermediary. In order to apply a user-centered approach for developing its new website, in 2015 the company decided to use crowdsourcing with potential end-users. Pre-selected crowdsourcees were invited via email to individually test and give feedback regarding the website’s interface as well as to report on functional bugs, usability, and provide ideas for improvement and additional features. They had to go through realistic test scenarios to explore the whole page. In return, they were offered a fixed monetary reward, transferred at the end of the project. InsureCorp chose to cooperate with an intermediary, responsible for acquiring the crowd, providing the platform, evaluating contributions, and handling the payment process. The professional crowdsourcing platform integrates all necessary functions to support the task-solving process, like feedback space, discussion forum, information-wiki, etc. InsureCorp could follow the submission process and communicate with crowdsourcees for questions via a chat function. It conducted three independent crowdsourcing projects with around 20 (potential) end-users per iteration, each with a duration of five days and involving a new crowd. The last project was analyzed in detail for the purpose of the study, presented in this research paper (May 2016).

In comparison, the second case (B), initiated by one of Switzerland’s largest retail companies (hereafter called RetailCorp), is fully managed by the initiator itself. RetailCorp conducts regular crowdsourcing projects with customers to improve its web shop, as this channel is increasingly growing in importance for them. For the investigated project (June, 2016), around 300 (potential) consumers of the web shop were invited per mail to participate individually in a crowdsourcing initiative via a link to an improvised crowdsourcing platform, consisting of a registration and landing page with access to files (e.g., design suggestions), a survey tool and a collaboration space for discussion. RetailCorp could communicate with crowdsourcees via mail (used as a chat function) throughout the process. Crowdsourcees were asked to evaluate several design suggestions for a new website interface (A/B-Testing) by answering a structured questionnaire. In return, they received a gift voucher for the shop.

Both projects illustrate common cases and incorporate all characteristics of crowdsourcing being a concrete task proposed via an open call, a virtual platform for feedback submission and interaction as well as a specified reward. Both cases can be placed in the field of crowdtesting (i.e., crowdsourced software-testing) with end-users, which is a relatively new and growing area and known for the use of intermediaries (Leicht et al. 2017). As both companies were already familiar with those projects, it is expected that exceptional problems, unusually influencing the crowdsourcing experience, could be reduced.

4.2 Data Collection

For the case assessment, first, to understand the intended experience, five semi-structured interviews were conducted with two managers from each crowdsourcing enterprise as well as one manager from the intermediary and one focus group discussion (including all). Based on that, a general blueprint of the interaction process with all its potential stimuli from a crowdsourcee’s perspective could be visualized for both cases. This supported the subsequent interview process and ensured that collected data on stimuli were comparable.

Second, semi-structured in-depth interviews (60–90 min) with a total of fourteen crowdsourcees (seven for each case) were conducted to decipher the crowdsourcing experience and underlying engagement processes. A slightly adapted version of the novel approach from consumer behavior, called “Sequential Incident Laddering Technique” (SILT), was used (Jüttner et al. 2013). Respondents were first asked to recall all stimuli (“incidents”) from the interaction process (step A). The process blueprint supported the interviewer in guiding the discussion. Subsequently, simple “what”, “why”, “how” questions were asked (“laddering”) to establish the link between a stimulus and the crowdsourcee’s cognitive, emotional, satisfaction and behavioral response within the process (step B). Lastly, the crowdsourcee’s final commitment and (planned) behavioral contribution (step C and D) were captured. Interviewees were asked to describe their emotional and rational disposition towards the crowdsourcer and intermediary. To avoid a recall bias (Koenig-Lewis and Palmer 2008), interviews took place two to seven days after participation. For reasons of better comparability, crowdsourcees with the same cultural background (Swiss) and comparable income as well as some prior crowdsourcing familiarity (2–7 projects with different crowdsourcers) were selected to avoid interviewing overly excited or bored individuals. The interviews were transcribed and assessed, together with the other data sources, by means of qualitative content analysis (Mayring 2015; Gläser and Laudel 2010). A category system, based on the theoretical framework of the engagement process, was developed and collected data was coded along stimuli: (a) perceived experience dimensions (emotional, cognitive or behavioral), (b) satisfaction response levels according to the negative (boredom, displeasure, disappointment, frustration, outrage) and positive (contentment, pleasure, delight, elation, ecstasy) emotional arousal states illustrated in Fig. 1, (c) related engagement object (crowdsourcer or intermediary), commitment state (affective, calculative or none), and (planned) behavior. Three researchers independently coded the data by allocating direct and indirect statements to the categories (interpretive approach) and subsequently discussed and aligned findings.

Third, to extend information and validate statements on crowdsourcees’ behavioral responses within the participation process, data concerning the time spent on the platform and with the website as well as demographic information and amount of previous activities were tracked. Contributions were analyzed in terms of their length (word count) and level of detail (i.e., under-/over-fulfillment of task).

5.1 Insure Corp: Assessment of the Crowdsourcing Experience and Underlying Engagement Process

Figure 5 gives an overview of perceived stimuli at the bottom of the illustration (process step A) and presents vertically the cognitive, emotional, related satisfaction and behavioral responses (process step B) from the seven interviewed crowdsourcees of InsureCorp. Captured responses regarding the final commitment states are not shown in the figure (out of space issues) but are elaborated in detail in the upcoming sections. Additionally, coding examples of satisfaction response types and commitment states can be found in the Appendix (available online via http://link.springer.com). All previously identified interaction points were perceived and mostly experienced by interviewees, shaping their crowdsourcing experience. As expected, crowdsourcees related the stimuli task, test object and (partly) the support function to the crowdsourcer and the rest to the intermediary. Around half (53%) of all perceived stimuli were evaluated to be emotionally arousing, dominantly positive in terms of pleasure, delight or elation (58%), of which the major part (66%) can be accounted to the crowdsourcer-managed stimuli. In comparison, 85 percent of negative-emotional evaluations (displeased, disappointed, frustrated) relate to intermediary-managed stimuli.

Open image in new window

More specifically, the invitation mail led only to positive or neutral states, while positive-emotional evaluations were related to feelings of happiness regarding the crowdsourcing enterprise (pleasant) or even task-excitement and surprise about the personal direct contact via mail (delight). Contrarily, the registration- and platform interface only led to neutral or negative statements, resulting in displeasure or even disappointment states due to irritations from requested type of data, language issues or negative surprises concerning the interface design. The test object (i.e., the website) led mainly to positive emotions due to its maturity and coverage (pleasant) as well as its relevance (core online service page) and surprisingly novel design, functions and features (delight). The behavioral data collected from the platform showed that emotionally aroused crowdsourcees spent also more time with the test object (more than 40 min) in comparison to the less aroused ones (less than 40 min). Interestingly, the task (i.e., website exploration with realistic case scenarios and preparation of think-aloud videos) covered the whole range of emotions. Feelings of surprise regarding the novelty and explorative type of task as well as stimulation due to the level of challenge and realistic cases led to either states of delight or even elation, or to disappointment and frustration, depending on the valence of arousal. Contribution analysis revealed that more emotionally aroused crowdsourcees did more than was expected in the task (over-fulfillment) and gave more detailed feedback in terms of word count (1.25 to 1.6 time as much), while negatively aroused ones contributed much less. Nevertheless, the support chat with the crowdsourcer only aroused pleasure, relief or simple contentment, depending on the speed of response and perceived friendliness of staff; while the discussion forum for interaction with other crowdsourcees was more cognitively evaluated (content or bored) or aroused negative emotions (displeased). The closing mail (received after submission of feedback) also resulted in negative (displeased), positive (pleased) or neutral (contentment) states, depending on the perceived tonality, fairness of treatment throughout the process and understandability of payment instructions. Lastly, the reward (fixed monetary amount) led mostly to a neutral state (content) as expectations were clear (amount was known before participation) and effort perceived as reasonable for all apart from one disappointed interviewee.

Collected data on ensuing commitment and behavior (process step C and D) towards InsureCorp demonstrates that four out of the seven interviewees described an attitude change and showed signs of calculative and affective commitment. They perceived InsureCorp to be more innovative, modern, open-minded, collaborative, customer-centric or supportive after participation. They mentioned an improved brand image and a strengthened relationship to the crowdsourcer. Those crowdsourcees described diverse value contributions, like return-intentions for further projects, referral and word of mouth to colleagues regarding the participation, a desire for additional contributions without monetary reward, willingness to use the test object in future, consumption-intentions as well as an interest in observing the further development of the website. One interviewee described himself as only calculatively committed due to perceived utility concerning learning potentials in the area of website design, resulting only in a willingness to return, although other crowdsourcing projects were of interest too. Lastly, two interviewees stated to be not committed at all and perceived no change in attitude. They were not sure if they would participate in another project due to their negative experiences. In comparison, crowdsourcees described their attitude towards the intermediary mostly rational by emphasizing the latter’s utility and perceived role as means to an end. They used terms as responsive, fair, reliable, effective and well-organized, referring to a more calculative form of commitment. Only those that felt familiar with the intermediary due to repeated interaction seemed to have developed a form of more affective commitment.

5.2 Retail Corp: Assessment of the Crowdsourcing Experience and Underlying Engagement Process

All previously identified interaction points were generally perceived and related to the crowdsourcer only, as no other party was mentioned (Fig. 6). Fifty percent of all perceived stimuli were evaluated to be emotionally arousing, while even 68 percent were evaluated to be positive in terms of pleasure, delight or elation of which all can be accounted to the crowdsourcer-managed stimuli.

Open image in new window

Taking a closer look at the experience evaluations (process step A and B), the invitation mail led to mainly positive statements, due to feelings of happiness regarding the crowdsourcing enterprise and a personal relevance of the test object (pleasant). The registration and platform interface also led to only neutral states and displeasure or even disappointment, due to an annoying amount of requested data, limited and bad integrations of functions. The test object also led mainly to positive emotions, due to its maturity (pleasant), surprisingly modern design and features (delight) and stimulation of personal need recognition (elation). Behavioral data showed that emotionally aroused crowdsourcees spent also more time with the test object (more than 10 min) in comparison to the less aroused ones (less than 10 min). The task (i.e., A/B-testing and filling out structured UX-surveys) led to neutral (bored), positive (pleased) and negative (disappointed) emotions, depending on the perceived happiness about the type of task and negative surprise about the low level of challenge. All crowdsourcees filled out the questionnaires and evaluated the test object as requested by the task. The support chat with the crowdsourcer aroused simple contentment or even pleasure when responses were perceived as detailed and displeasure when the contact channel (mail instead of platform) was disliked. In contrast, the discussion forum only led to positive evaluations, while emotions were aroused when the chat was perceived as vivid and responses as helpful (pleased/relieved). The closing mail (including information on reward, planned changes based on feedback, other crowdsourcing projects) also resulted in a broad range of emotional states, depending on the aroused happiness, due to perceived impact and valence of surprise about changes (delight/disappointment). Lastly, the reward (shopping voucher send via mail) led to a neutral state (content) as expectations were clear and the effort perceived as reasonable.

Data on the ensuing commitment and behavior shows that only three out of the seven interviewees described an attitude change and signs of calculative and affective commitment towards RetailCorp. They described RetailCorp as a lovable and caring company that has become an important part of their life. Some even felt like a member of the company, much closer than before participation. Those crowdsourcees also mentioned return approaches and referrals regarding the project and the company itself, as well as the use of the test object. Two interviewees described themselves as at least calculatively committed, due to their interest in the reward and perceived utility through support of a product that is of use to them. They mentioned its future use and a general willingness to return for other crowdsourcing projects but would also join projects of other companies. Only one interviewee perceived no change in attitude and felt not committed to the crowdsourcer at all, unwilling to participate again as he missed the sense and incentive of this activity.

6.1 Similarities Across Cases: Role of Stimuli Within the Engagement Process

In both cases an equal amount of emotionally evaluated stimuli (half/half) could be observed as well as similar patterns of levels of negative and positive emotional arousals along the interaction process. First, both cases showed that the invitation mail led to only low to medium high levels of emotional arousal (referring to response types 1–2 in Table 1) or at least to a simple contentment without arousal (0), while the test object led to a broader range of positive emotions (1–3). But obviously, both stimuli did not arouse any negative feelings. Second, the registration and platform interface as well as the reward-receipt mainly led to neutral states (0) or even low to medium high levels of negative arousal, ranging from − 1 to − 2. But no positive emotions were expressed. Third, the task, support chat, discussion board, and closing mail covered the whole range of states, including positive, negative and neutral ones (− 3 − 3). Based on this we can cluster those stimuli into four group types, inspired by the attribute-based perspective proposed by Kano et al. (1984) and enriched with the multidimensional logic of the satisfaction response described by Oliver and De Sarbo (1989) and the more detailed emotional arousal continuum provided by Briggs et al. (2008). The four stimuli groups can be described as follows:

6.2 Differences Across Cases: Role of Intermediary within the Engagement Process

One explanation for those observations is that intermediary-managed stimuli mainly belong to the category of risk factors (registration, platform, reward), with high potential of negative arousal, while the stimuli managed by the crowdsourcer are game changers (task and support chat) and value adders (test object), both with the potential of arousing high levels of positive emotions. Thus, if crowdsourcees evaluate crowdsourcer-managed stimuli positively, this relates directly to the overall experience evaluation regarding the crowdsourcer (Oliver 1993) and chances are high that an overall positive and emotional commitment state develops as well (Verhoef 2003).

Consequently, we propose that in a mediated setting in which crowdsourcees can differentiate stimuli-related experiences, they are generally able to draw separate conclusions regarding their engagement towards the different parties. In such a setting, negative evaluations, which are especially probable with stimuli like risk factors, can be absorbed by an intermediary when outsourced to them. Hence, we propose that crowdsourcing enterprises incorporating intermediaries theoretically have the potential to generate a desired commitment state, when recognized and perceived stimuli under their control rather belong to the group of game changers (e.g., task) and value adders (e.g., test object) and the intermediary mainly controls risk factors (e.g., platform interface). In this context, crowdsourcing enterprises may minimalize their effort while maximizing the engagement potential by proactively designing and managing game changers and value adders under their control, with the goal of arousing positive medium to high emotions. Thus, by involving an intermediary, crowdsourcers are not only able to outsource part of the work of managing the initiative, but also to shift risk regarding potential threats to the intermediary, while increasing their chance of fostering engagement towards the enterprise, when focusing on a targeted design and execution of their high-potential stimuli. Yet, this requires the ownership of stimuli to be recognizable for crowdsourcees.

Finally, we want to emphasize that these propositions were formulated in a way that they are also generalizable to other crowdsourcing contexts, in order to explore and test them further with different types of projects and intermediary set-ups. Yet, they need to be treated with care as empirical observations are based on only two crowdsourcing cases with a total of fourteen interviews. Case study research is not sufficient to prove causal relationships (Yin 2013). Hence, here described observations can be seen as a kind of pilot study for testing a promising methodology and making initial propositions (Leonardi 2011). For future research, it is recommended to verify proposed relationships with quantitative research. A survey approach may be applied, which tests satisfaction response levels related to specific stimuli and their impact on commitment and behavior. Also, experiments with manipulated stimuli may be used to explain effects in controlled settings.