The Effect of Rating Scales on Decision Quality and User Attitudes in Online Innovation Communities
Journal
International Journal of Electronic Commerce (IJEC)
ISSN
1086-4415
ISSN-Digital
1557-9301
Type
journal article
Date Issued
2013-03
Author(s)
Research Team
CCC, Crowdsourcing, IWI6
Abstract
Given the rise of the Internet, consumers increasingly engage in co-creating products and services. Whereas most co-creation research deals with various aspects of generating user-generated content, this study addresses designing ratings scales for evaluating such content. In detail, we analyze functional and perceptional aspects of two frequently used rating scales in online innovation communities. Using a multimethod approach, our experiments show that a multicriteria scale leads to higher decision quality of users than a single-criterion scale, that idea elaboration (i.e., idea length) negatively moderates this effect such that the single-criterion rating scale outperforms the multicriteria scale for long ideas, and finally that the multicriteria scale leads to more favorable user attitudes toward the Web site. To ensure robustness of our results, we applied a bootstrap-based Monte Carlo simulation based on our experimental data. We found that around 20 user ratings per idea are sufficient for creating stable idea rankings and that a combination of both rating scales leads to a 63 percent performance improvement over the single-criterion rating scale and 16 percent over the multicriteria rating scale. Our work contributes to co-creation research by offering insights as to how the interaction of the technology being used (i.e., rating scale) and the attributes of the rating object affects two central outcome measures: the effectiveness of the rating in terms of decision quality of its users and the perception of the scale by its users as a predictor of future use.
Funding(s)
Language
English
Keywords
Rating Scales
Decision Quality
User Attitudes
Online Innovation Communities
HSG Classification
contribution to scientific community
HSG Profile Area
SoM - Business Innovation
Refereed
Yes
Publisher
Routledge, Taylor & Francis Group
Publisher place
Philadelphia, Pa.
Volume
17
Number
3
Start page
7
End page
36
Pages
30
Subject(s)
Division(s)
Eprints ID
220400
File(s)![Thumbnail Image]()
Name
JML_378.pdf
Size
881.14 KB
Format
Adobe PDF
Checksum (MD5)
db9f1d9db9707604db82148119e3f14a