Assessment of Social Anxiety Disorder (SAD) is limited by paradigms that often target performance-related fears and its reliance on static self-reports that fail to capture the dynamic, in-the-moment nature of anxiety. This study introduces a novel Virtual Reality (VR) framework to assess SAD that integrates continuous self-ratings of anxiety with objective physiological data. Sixty-three participants were immersed in an ecologically valid VR waiting room scenario. They continuously rated their anxiety levels in real-time using a joystick, providing a direct subjective-behavioral measure, while their electrocardiogram and electrodermal activity were simultaneously recorded. We systematically compared three machine learning pipelines - a static Support Vector Machine, an end-to-end Transformer Encoder, and a Transformer Encoder on engineered features - to classify participants into the high and the low anxiety groups previously determined through the Liebowitz Social Anxiety Scale (L-SAS). To effectively harness mental disorder and subjective data and overcome integration challenges, we developed a Late Fusion Transformer framework. This optimized architecture yielded the most robust and balanced classification (F1-score = 0.853 for the anxious class; 82.5% overall accuracy). The results show a synergistic benefit from combining modalities: while feature selection identified continuous self-rated anxiety as the most powerful individual predictor, the late integration of physiological markers provided a significant complementary contribution that enhanced the model's overall accuracy, particularly in identifying non-anxious controls. In conclusion, our framework provides a dynamic assessment of SAD that captures the moment-to-moment experience of anxiety. By integrating a continuous self-rating of social anxiety with objective physiological data, this approach can identify clinically relevant incongruencies between a person's retrospective self-perception and their in-the-moment reactivity, paving the way for a new era of objective, personalized, and temporally sensitive psychometrics.

Multimodal Classification of Social Anxiety Using Continuous Self-Rating and Physiological Data in Virtual Reality

Matteo Martini;Manuela Chessa;
2026-01-01

Abstract

Assessment of Social Anxiety Disorder (SAD) is limited by paradigms that often target performance-related fears and its reliance on static self-reports that fail to capture the dynamic, in-the-moment nature of anxiety. This study introduces a novel Virtual Reality (VR) framework to assess SAD that integrates continuous self-ratings of anxiety with objective physiological data. Sixty-three participants were immersed in an ecologically valid VR waiting room scenario. They continuously rated their anxiety levels in real-time using a joystick, providing a direct subjective-behavioral measure, while their electrocardiogram and electrodermal activity were simultaneously recorded. We systematically compared three machine learning pipelines - a static Support Vector Machine, an end-to-end Transformer Encoder, and a Transformer Encoder on engineered features - to classify participants into the high and the low anxiety groups previously determined through the Liebowitz Social Anxiety Scale (L-SAS). To effectively harness mental disorder and subjective data and overcome integration challenges, we developed a Late Fusion Transformer framework. This optimized architecture yielded the most robust and balanced classification (F1-score = 0.853 for the anxious class; 82.5% overall accuracy). The results show a synergistic benefit from combining modalities: while feature selection identified continuous self-rated anxiety as the most powerful individual predictor, the late integration of physiological markers provided a significant complementary contribution that enhanced the model's overall accuracy, particularly in identifying non-anxious controls. In conclusion, our framework provides a dynamic assessment of SAD that captures the moment-to-moment experience of anxiety. By integrating a continuous self-rating of social anxiety with objective physiological data, this approach can identify clinically relevant incongruencies between a person's retrospective self-perception and their in-the-moment reactivity, paving the way for a new era of objective, personalized, and temporally sensitive psychometrics.
File in questo prodotto:
File Dimensione Formato  
Multimodal_Classification_of_Social_Anxiety_Using_Continuous_Self-Rating_and_Physiological_Data_in_Virtual_Reality.pdf

accesso aperto

Tipologia: Documento in versione editoriale
Dimensione 2.09 MB
Formato Adobe PDF
2.09 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11567/1325278
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? 0
social impact