Please use this identifier to cite or link to this item: https://www.um.edu.mt/library/oar/handle/123456789/102389
Title: Supervised contrastive learning for affect modelling
Authors: Pinitas, Kosmas
Makantasis, Konstantinos
Liapis, Antonios
Yannakakis, Georgios N.
Keywords: Artificial intelligence
Human-computer interaction
Neural networks (Computer science)
Arousal (Physiology)
Issue Date: 2022
Publisher: Association for Computing Machinery
Citation: Pinitas, K., Makantasis, K., Liapis, A. & Yannakakis, G. N. (2022). Supervised contrastive learning for affect modelling. The 24th ACM International Conference on Multimodal Interaction, Bengaluru.
Abstract: Affect modeling is viewed, traditionally, as the process of mapping measurable affect manifestations from multiple modalities of user input to affect labels. That mapping is usually inferred through endto- end (manifestation-to-affect) machine learning processes. What if, instead, one trains general, subject-invariant representations that consider affect information and then uses such representations to model affect? In this paper we assume that affect labels form an integral part, and not just the training signal, of an affect representation and we explore how the recent paradigm of contrastive learning can be employed to discover general high-level affect-infused representations for the purpose of modeling affect.We introduce three different supervised contrastive learning approaches for training representations that consider affect information. In this initial study we test the proposed methods for arousal prediction in the RECOLA dataset based on user information from multiple modalities. Results demonstrate the representation capacity of contrastive learning and its efficiency in boosting the accuracy of affect models. Beyond their evidenced higher performance compared to end-to-end arousal classification, the resulting representations are general purpose and subject-agnostic, as training is guided though general affect information available in any multimodal corpus.
URI: https://www.um.edu.mt/library/oar/handle/123456789/102389
Appears in Collections:Scholarly Works - InsDG

Files in This Item:
File Description SizeFormat 
supervised_contrastive_learning_for_affect_modelling_2022.pdf1.35 MBAdobe PDFView/Open


Items in OAR@UM are protected by copyright, with all rights reserved, unless otherwise indicated.