Machine Learning May Improve Screening and Prediction of Suicidal Ideation

Machine learning models that evaluate individual depressive symptoms may predict suicidal ideation (SI) endorsement better than models relying on overall depression severity, while incorporating network-derived symptom features provides little additional predictive benefit. These study findings were published in Psychiatry Research.

Researchers conducted a machine learning analysis to determine whether individual depressive symptoms and patterns of symptom co-occurrence could predict SI endorsement.

Using 2 nationally representative South Korean datasets, the 2024 Korea Community Health Survey (KCHS) and the Korea National Health and Nutrition Examination Survey (KNHANES), researchers selected adults aged at least 19 years and older in metropolitan cities and provinces in South Korea. The KNHANES was used for the generalizability test. Researchers compared a conventional model based on the Patient Health Questionnaire (PHQ)-8 total score with models that considered the 8 depressive symptoms separately. They also evaluated Network-Augmented Machine Learning Utility (NAMU) models that combined the individual PHQ-8 items with 37 features representing person-specific symptom co-activation. The Patient Health Questionnaire-9 (PHQ-9) was also used for final analysis of depressive symptoms and SI.

Future prospective studies should validate [Network-Augmented Machine Learning Utility] against independent clinical and behavioral outcomes, including clinician-assessed suicidal ideation and suicidal behavior.

A total of 231,469 participants  (mean age, 56.08 years; 54.2% women) from the KCHS and 21,473 participants from the KNHANES were selected for final analysis. The participants from KCHS were separated into those with (n=218,996) vs without (n=12,473) SI.

The first 8 items of the PHQ-9, collectively referred to as the PHQ-8, served as predictors, while any endorsement of a score of at least 1 on PHQ-9 item 9 served as the outcome. 

Among the KCHS participants, 5.4% endorsed PHQ-9 item 9. Those with vs without item 9 endorsement were more frequently women (63.8% vs 53.6%, respectively) and were older (60.53 years vs 55.82 years, respectively). Depressive symptom burden also differed substantially between those with vs without item 9 endorsement groups, with mean PHQ-8 scores of 8.66 vs 2.01, respectively. Additionally, 15.0% vs 0.3%, respectively, had PHQ-8 scores of at least 15. 

The PHQ-8 total-score logistic regression model achieved a precision-recall area under the curve (PR AUC) of .4553. Modeling the individual symptoms improved discrimination, with symptom-only XGBoost achieving a PR AUC of .5570. 

Relative to the PHQ-8 total-score model, symptom-only XGBoost reduced the estimated number of screening alerts by approximately 39% and false-positive results by 47% per 1000 participants for participants in the KCHS. NAMU-full XGBoost reduced alerts by approximately 41% and false-positive results by 49%. These results suggest that the improvement over the total-score approach was driven primarily by evaluating depressive symptoms individually rather than by the addition of network-derived information.

Researchers also found that the 45-feature NAMU model could be reduced to 6 features without substantially sacrificing predictive performance. The retained features included depressed mood severity (PHQ-2), worthlessness/guilt severity (PHQ-6), sleep disturbance severity (PHQ-3), psychomotor change severity (PHQ-8), network density, and depressed-mood strength centrality. In the KCHS test sample, the reduced model achieved a PR AUC of .5596 and a receiver operating characteristic area under the curve (ROC AUC) of .9234. The 6 features accounted for 84.3% of the aggregate SHapley Addictive exPlanations (SHAP) importance in the full model.

Among KNHANES participants, 6.2% endorsed item 9. Participants with vs without endorsing item 9 were more frequently women (68.7% vs 56.4%, respectively), were older (56.68 years vs 51.16 years, respectively), and had higher mean PHQ-8 scores (8.87 vs 2.06, respectively).

When applied to KNHANES, the NAMU-full XGBoost achieved a PR AUC of .4931 and ROC AUC of .9010. Using the original KCHS-derived threshold (0.58), the model demonstrated a sensitivity of .7867 and specificity of .8629. 

Study limitations include the cross-sectional design, reliance on PHQ-9 item 9 rather than an independent clinical assessment of SI, and the lack of clinical or behavioral outcomes such as suicide attempts.

“Future prospective studies should validate [Network-Augmented Machine Learning Utility] against independent clinical and behavioral outcomes, including clinician-assessed suicidal ideation and suicidal behavior,” the study authors concluded.