Abstract
This paper introduces a comparative analysis between rating and pairwise self-reporting via questionnaires in user survey experiments. Two dissimilar game user survey experiments are employed in which the two questionnaire schemes are tested and compared for reliable affect annotation. The statistical analysis followed to test our hypotheses shows that even though the two self-reporting schemes are consistent there are significant order of reporting effects when subjects report via a rating questionnaire. The paper concludes with a discussion of the appropriateness of each self-reporting scheme under conditions drawn from the experimental results obtained.
An Erratum for this chapter can be found at http://dx.doi.org/10.1007/978-3-642-24600-5_65 .
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Agresti, A.: Analysis of ordinal paired comparison data. Journal of the Royal Statistical Society. Series C (Applied Statistics) 41(2), 287–297 (1992)
Chan, J.C.: Response-order effects in Likert-type scales. Educational and Psychological Measurement 51(3), 531–540 (1991)
Cohen, J.: A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20, 37–46 (1960)
Cronbach, J.L.: Coefficient alpha and the internal structure of tests. Psychometrika 16(3), 297–334 (1951)
Likert, R.: A technique for the measurement of attitudes. Archives of Psychology 140, 1–55 (1932)
Linn, R., Gronlund, N.: Measurement and assessment in teaching. Prentice-Hall, Englewood Cliffs (2000)
Lund, H.H., Klitbo, T., Jessen, C.: Playware technology for physically activating play. Artifical Life and Robotics Journal 9(4), 165–174 (2005)
Poels, K., IJsselsteijn, W.: Development and validation of the game experience questionnaire. In: FUGA Workshop Mini-Symposium, Helsinki, Finland (2008)
Read, J., MacFarlane, S., Cassey, C.: Endurability, engagement and expectations. In: Proceedings of International Conference for Interaction Design and Children (2002)
Scheffe, H.: An analysis of variance for paired comparisons. Journal of the American Statistical Association 47(259), 381–400 (1952)
Stevens, S.S.: On the Theory of Scales of Measurement. Science 103(2684), 677–680 (1946)
Tognetti, S., Garbarino, M., Bonarini, A., Matteucci, M.: Modeling enjoyment preference from physiological responses in a car racing game. In: Proceedings of the IEEE Conference on Computational Intelligence and Games, Copenhagen, Denmark, August 18-21, pp. 321–328 (2010)
Viswanathan, M.: Measurement of individual differences in preference for numerical information. Journal of Applied Psychology 78(5), 741–752
Yannakakis, G.N.: Preference Learning for Affective Modeling. In: Proceedings of the Int. Conf. on Affective Computing and Intelligent Interaction, pp. 126–131. IEEE, Amsterdam (2009)
Yannakakis, G.N., Hallam, J.: Towards Optimizing Entertainment in Computer Games. Applied Artificial Intelligence 21, 933–971 (2007)
Yannakakis, G.N., Hallam, J., Lund, H.H.: Entertainment Capture through Heart Rate Activity in Physical Interactive Playgrounds. User Modeling and User-Adapted Interaction, Special Issue: Affective Modeling and Adaptation 18(1-2), 207–243 (2008)
Yannakakis, G.N., Maragoudakis, M., Hallam, J.: Preference Learning for Cognitive Modeling: A Case Study on Entertainment Preferences. IEEE Systems, Man and Cybernetics; Part A: Systems and Humans 39(6), 1165–1175 (2009)
Yannakakis, G.N., MartÃnez, H.P., Jhala, A.: Towards Affective Camera Control in Games. User Modeling and User-Adapted Interaction 20(4), 313–340 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yannakakis, G.N., Hallam, J. (2011). Ranking vs. Preference: A Comparative Study of Self-reporting. In: D’Mello, S., Graesser, A., Schuller, B., Martin, JC. (eds) Affective Computing and Intelligent Interaction. ACII 2011. Lecture Notes in Computer Science, vol 6974. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24600-5_47
Download citation
DOI: https://doi.org/10.1007/978-3-642-24600-5_47
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24599-2
Online ISBN: 978-3-642-24600-5
eBook Packages: Computer ScienceComputer Science (R0)