Abstract
The paper deals with the recording and the evaluation of a multi modal (audio/video) database of spontaneous emotions. Firstly, motivation for this work is given and different recording strategies used are described. Special attention is given to the process of evaluating the emotional database. Different kappa statistics normally used in measuring the agreement between annotators are discussed. Following the problems of standard kappa coefficients, when used in emotional database assessment, a new time-weighted free-marginal kappa is presented. It differs from the other kappa statistics in that it weights each utterance’s particular score of agreement based on the duration of the utterance. The new method is evaluated and the superiority over the standard kappa, when dealing with a database of spontaneous emotions, is demonstrated.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
LDC: SUSAS (Speech Under Simulated and Actual Stress). Language Data Consortium (1999), http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC99S78
LDC: Emotional Prosody Speech and Transcripts. Language Data Consortium (2002), http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2002S28
Martin, O., Kotsia, I., Macq, B., Pitas, I.: The eNTERFACE 2005 Audio-Visual Emotion Database. In: Proc. of the 22nd International Conference on Data Engineering Workshops (ICDEW 2006), p. 8 (2006)
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W., Weiss, B.: A Database of German Emotional Speech. In: Interspeech 2005, pp. 1517–1520 (2005)
Battocchi, A., Pianesi, F.: DAFEX: Un Database Di Espressioni Facciali Dinamiche. In: Proceedings of the SLI-GSCP Workshop ”Comunicazione Parlata e Manifestazione delle Emozioni” (2004)
Randolph, J.J.: Free-marginal multirater kappa: An alternative to Fleiss’ fixed-marginal multirater kappa. In: Joensuu University Learning and Instruction Symposium 2005, Joensuu, Finland (2005)
Fleiss, J.L.: Measuring nominal scale agreement among many raters. Psychological Bulletin 76(5), 378–382 (1971)
Cohen, J.: A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20(1), 37–46 (1960)
Randolph, J.J.: Free-marginal multirater kappa: An alternative to Fleiss’ fixed-marginal multirater kappa. In: Joensuu University Learning and Instruction Symposium 2005, Joensuu, Finland (2005)
Callejas, Z., López-Cózar, R.: On the Use of Kappa Coefficients to Measure the Reliability of the Annotation of Non-acted Emotions. In: André, E., Dybkjær, L., Minker, W., Neumann, H., Pieraccini, R., Weber, M. (eds.) PIT 2008. LNCS (LNAI), vol. 5078, pp. 221–232. Springer, Heidelberg (2008)
Gajšek, R., Štruc, V., Mihelič, F., Podlesek, A., Komidar, L., Sočan, G., Bajec, B.: Multi-Modal Emotional Database: AvID. Informatica (Ljubljana) 33(1), 101–106 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gajšek, R., Štruc, V., Vesnicer, B., Podlesek, A., Komidar, L., Mihelič, F. (2009). Analysis and Assessment of AvID: Multi-Modal Emotional Database. In: Matoušek, V., Mautner, P. (eds) Text, Speech and Dialogue. TSD 2009. Lecture Notes in Computer Science(), vol 5729. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04208-9_38
Download citation
DOI: https://doi.org/10.1007/978-3-642-04208-9_38
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04207-2
Online ISBN: 978-3-642-04208-9
eBook Packages: Computer ScienceComputer Science (R0)