A Multimodal Framework for Depression Detection during Covid-19 via Harvesting Social Media: A Novel Dataset and Method

• URL: http://arxiv.org/abs/2511.00424v1
• Date: Sat, 01 Nov 2025 06:33:14 GMT
• Title: A Multimodal Framework for Depression Detection during Covid-19 via Harvesting Social Media: A Novel Dataset and Method
• Authors: Ashutosh Anshul, Gumpili Sai Pranav, Mohammad Zia Ur Rehman, Nagendra Kumar,
• Abstract summary: The recent coronavirus disease (Covid-19) has become a pandemic and has affected the entire globe.<n>Depression significantly influences most diseases worldwide, making it difficult to detect mental health conditions in people.<n>We propose a novel framework that combines textual, user-specific, and image analysis to detect depression among social media users.
• Score: 3.884231159866055
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The recent coronavirus disease (Covid-19) has become a pandemic and has affected the entire globe. During the pandemic, we have observed a spike in cases related to mental health, such as anxiety, stress, and depression. Depression significantly influences most diseases worldwide, making it difficult to detect mental health conditions in people due to unawareness and unwillingness to consult a doctor. However, nowadays, people extensively use online social media platforms to express their emotions and thoughts. Hence, social media platforms are now becoming a large data source that can be utilized for detecting depression and mental illness. However, existing approaches often overlook data sparsity in tweets and the multimodal aspects of social media. In this paper, we propose a novel multimodal framework that combines textual, user-specific, and image analysis to detect depression among social media users. To provide enough context about the user's emotional state, we propose (i) an extrinsic feature by harnessing the URLs present in tweets and (ii) extracting textual content present in images posted in tweets. We also extract five sets of features belonging to different modalities to describe a user. Additionally, we introduce a Deep Learning model, the Visual Neural Network (VNN), to generate embeddings of user-posted images, which are used to create the visual feature vector for prediction. We contribute a curated Covid-19 dataset of depressed and non-depressed users for research purposes and demonstrate the effectiveness of our model in detecting depression during the Covid-19 outbreak. Our model outperforms existing state-of-the-art methods over a benchmark dataset by 2%-8% and produces promising results on the Covid-19 dataset. Our analysis highlights the impact of each modality and provides valuable insights into users' mental and emotional states.

Related papers

• ReDepress: A Cognitive Framework for Detecting Depression Relapse from Social Media [48.56586765769052]
  We present ReDepress, the first clinically validated social media dataset focused on relapse.<n>Our framework draws on cognitive theories of depression, incorporating constructs such as attention bias, interpretation bias, memory bias and rumination.<n>Our findings validate psychological theories in real-world textual data and underscore the potential of cognitive-informed computational methods for early relapse detection.
  arXiv  Detail & Related papers  (2025-09-22T16:33:59Z)
• Reading Between the Frames: Multi-Modal Depression Detection in Videos from Non-Verbal Cues [11.942057763913208]
  Depression, a prominent contributor to global disability, affects a substantial portion of the population. Efforts to detect depression from social media texts have been prevalent, yet only a few works explored depression detection from user-generated video content. We propose a simple and flexible multi-modal temporal model capable of discerning non-verbal depression cues from diverse modalities in noisy, real-world videos.
  arXiv  Detail & Related papers  (2024-01-05T10:47:42Z)
• Depression detection in social media posts using affective and social norm features [84.12658971655253]
  We propose a deep architecture for depression detection from social media posts. We incorporate profanity and morality features of posts and words in our architecture using a late fusion scheme. The inclusion of the proposed features yields state-of-the-art results in both settings.
  arXiv  Detail & Related papers  (2023-03-24T21:26:27Z)
• It's Just a Matter of Time: Detecting Depression with Time-Enriched Multimodal Transformers [24.776445591293186]
  We propose a flexible time-enriched multimodal transformer architecture for detecting depression from social media posts. Our model operates directly at the user-level, and we enrich it with the relative time between posts by using time2vec positional embeddings. We show that our method, using EmoBERTa and CLIP embeddings, surpasses other methods on two multimodal datasets.
  arXiv  Detail & Related papers  (2023-01-13T09:40:19Z)
• Self-supervised Hypergraph Representation Learning for Sociological Analysis [52.514283292498405]
  We propose a fundamental methodology to support the further fusion of data mining techniques and sociological behavioral criteria. First, we propose an effective hypergraph awareness and a fast line graph construction framework. Second, we propose a novel hypergraph-based neural network to learn social influence flowing from users to users.
  arXiv  Detail & Related papers  (2022-12-22T01:20:29Z)
• Semantic Similarity Models for Depression Severity Estimation [53.72188878602294]
  This paper presents an efficient semantic pipeline to study depression severity in individuals based on their social media writings. We use test user sentences for producing semantic rankings over an index of representative training sentences corresponding to depressive symptoms and severity levels. We evaluate our methods on two Reddit-based benchmarks, achieving 30% improvement over state of the art in terms of measuring depression severity.
  arXiv  Detail & Related papers  (2022-11-14T18:47:26Z)
• Data set creation and empirical analysis for detecting signs of depression from social media postings [0.0]
  Depression is a common mental illness that has to be detected and treated at an early stage to avoid serious consequences. We developed a gold standard data set that detects the levels of depression as not depressed', moderately depressed' and severely depressed' from the social media postings.
  arXiv  Detail & Related papers  (2022-02-07T10:24:33Z)
• Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data [74.60507696087966]
  Mental health conditions remain underdiagnosed even in countries with common access to advanced medical care. One promising data source to help monitor human behavior is daily smartphone usage. We study behavioral markers of daily mood using a recent dataset of mobile behaviors from adolescent populations at high risk of suicidal behaviors.
  arXiv  Detail & Related papers  (2021-06-24T17:46:03Z)
• DepressionNet: A Novel Summarization Boosted Deep Framework for Depression Detection on Social Media [12.820775223409857]
  Twitter is a popular online social media platform which allows users to share their user-generated content. One of the applications is in automatically discovering mental health problems, e.g., depression. Previous studies to automatically detect a depressed user on online social media have largely relied upon the user behaviour and their linguistic patterns.
  arXiv  Detail & Related papers  (2021-05-23T08:05:53Z)
• Assessing the Severity of Health States based on Social Media Posts [62.52087340582502]
  We propose a multiview learning framework that models both the textual content as well as contextual-information to assess the severity of the user's health state. The diverse NLU views demonstrate its effectiveness on both the tasks and as well as on the individual disease to assess a user's health.
  arXiv  Detail & Related papers  (2020-09-21T03:45:14Z)
• Machine Learning-based Approach for Depression Detection in Twitter Using Content and Activity Features [0.0]
  Recent studies have indicated a correlation between high usage of social media sites and increased depression. The present study aims to exploit machine learning techniques for detecting a probable depressed Twitter user based on both, his/her network behavior and tweets.
  arXiv  Detail & Related papers  (2020-03-09T11:27:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.