Depression Detection from Text using Machine Learning

The growth of online communication creates an attractive opportunity for early and scalable detection of mental health risks, such as depression. However, the challenge of creating machine learning models that are accurate and generalizable across different text domains (e.g., clinical forum and social media), remains a challenge. This work demonstrates a thorough, reproducible framework for detection of risk of depression from online text. We used three different publicly available datasets: Suicide Watch, Mental Health Corpus, and Mental Health/Data, with a serious space used to preprocess the text, normalize the dataset by balancing the label, and rendering a poor quality dataset through stratified subsampling. Our approach included multi-dimensional feature engineering through the construction of a strong feature set, which included linguistic/readability measures (e.g., Flesch), conventional sentiment estimates (VADER, TextBlob), and advanced transformer-based sentiment (RoBERTa, DistilBERT). We conducted systematic training and evaluation of a number of supervised classifiers, including Logistic Regression, SVM, Random Forest, and Gradient Boosting. We used two-protocols for reliable model evaluation: within domain reliability through stratified 5-fold cross validation and cross-domain transfer testing for generalizability. Since our model evaluation included accuracy, precision, recall, F1-score, and ROC-AUC, we provide model interpretability through feature importance that reflects the most predictive signals.