Idea Evaluation Mechanisms for Collective Intelligence in Open Innovation Communities: Do Traders outperform Raters?
ISBN
978-1-61839-472-9
Type
conference paper
Date Issued
2011-12-04
Author(s)
Research Team
Crowdsourcing, CCC, IWI6
Abstract
The increasing popularity of open innovation approaches has led to the rise of various open innovation communities on the Internet which might contain several thousand user-generated ideas. However, a company’s absorptive capacity is limited regarding such an amount of ideas so that there is a strong need for mechanisms supporting the evaluation of these ideas. In this paper, we focus on the evaluation of such mechanisms for collective idea evaluation. Applying a multi-method approach, we compare six different configurations of a prediction market with a multi-criteria rating scale that performed best in previous research. We combine a web-based experiment with 448 participants, data from a participant survey, and an independent expert jury. Based on cognitive load theory, we explain why a multi-criteria rating scale outperforms prediction markets in terms of evaluation accuracy and evaluation satisfaction. This study contributes to theory building in the emerging field of collective intelligence.
Language
English
Keywords
Cognitive Load Theory
Collective Intelligence
Communities
Crowdsourcing
Information aggregation
Open innovation
Prediction Marktets
Rating Scales
Web 2.0
HSG Classification
contribution to scientific community
HSG Profile Area
SoM - Business Innovation
Refereed
Yes
Book title
ICIS 2011 Proceedings
Publisher
AIS Association for Information Systems
Event Title
32. International Conference on Information Systems (ICIS)
Event Location
Shanghai
Event Date
04.-07.12.2011
Subject(s)
Division(s)
Eprints ID
219905
File(s)![Thumbnail Image]()
open.access
Name
JML_304.pdf
Size
682.89 KB
Format
Adobe PDF
Checksum (MD5)
68e4de231a96ee931affbfe68886114c