Skip to main content
SpringerLink
Log in

Computer-Aided Detection of Depressive Severity Using Multimodal Behavioral Data

  • Chapter
  • First Online:
Handbook of Artificial Intelligence in Healthcare

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 211))

Abstract

This chapter presents depressive severity detection using deep learning methods for automatic audio, visual and audiovisual emotion sensing. The article starts from basic methods on experimental design and data acquisition systems for computer-aided depressive severity diagnosis. Next, typical baseline behavioral features such as facial expressions and speech prosody will be introduced. From the experimental results of the baseline systems introduced in this chapter, readers can not only compare between the performance of different baseline features but also have a general understanding of computer-aided depressive severity diagnosis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Chen, L., Wang, L., Qiu, X. H., Yang, X. X., Qiao, Z. X., Yang, Y. J., Liang, Y.: Depression among Chinese university students: prevalence and socio-demographic correlates. PloS One 8(3), e58379 (2013)

    Google Scholar 

  2. Lei, X. Y., Xiao, L. M., Liu, Y. N., Li, Y. M.:Prevalence of depression among Chinese University students: a meta-analysis. PLoS One 11(4), e0153454 (2016)

    Google Scholar 

  3. Setterfield, M., Walsh, M., Frey, A.L., McCabe, C.: Increased social anhedonia and reduced helping behaviour in young people with high depressive symptomatology. J. Affect. Disord. 205, 372–377 (2016)

    Article  Google Scholar 

  4. Brinkmann, K., Franzen, J.: Blunted cardiovascular reactivity during social reward anticipation in subclinical depression. Int. J. Psychophysiol. 119, 119–126 (2017)

    Article  Google Scholar 

  5. American Psychiatric Association.: Diagnostic and statistical manual of mental disorders (DSM-5®). American Psychiatric Pub (2013)

    Google Scholar 

  6. Hysenbegasi, A., Hass, S. L., Rowland, C. R.: The impact of depression on the academic productivity of university students. J. Ment. Health Policy Econ. 8(3), 145 (2005)

    Google Scholar 

  7. Hu, T.W.: The economic burden of depression and reimbursement policy in the Asia Pacific region. Australas. Psych. 12(sup1), s11–s15 (2004)

    Article  Google Scholar 

  8. Sobocki, P., Lekander, I., Borgström, F., Ström, O., Runeson, B.: The economic burden of depression in Sweden from 1997 to 2005. Eur. Psychiatry 22(3), 146–152 (2007)

    Article  Google Scholar 

  9. Aalto-Setälä, T., Marttunen, M., Tuulio-Henriksson, A., Poikolainen, K., Lö-nnqvist, J.: Depressive symptoms in adolescence as predictors of early adulthood depressive disorders and maladjustment. Am. J. Psychiatry 159(7), 1235–1237 (2002)

    Article  Google Scholar 

  10. Liu, X.C., Ma, D.D., Kurita, H., Tang, M.Q.: Self-reported depressive symptoms among Chinese adolescents. Soc. Psychiatry Psychiatr. Epidemiol. 34(1), 44–47 (1999)

    Article  Google Scholar 

  11. Jan, A., Meng, H., Gaus, Y. F.B.A., Zhang, F.: Artificial intelligent system for automatic depression level analysis through visual and vocal expressions. IEEE Trans. Cogn. & Dev. Syst. 99, 1–1 (2017)

    Google Scholar 

  12. Jan, A., Meng, H., Gaus, Y. F. A., Zhang, F., Turabzadeh, S.: Automatic depression scale prediction using facial expression dynamics and regression. In Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge (pp. 73–80). ACM (2014, November)

    Google Scholar 

  13. Yang, L., Jiang, D., Xia, X., Pei, E., Oveneke, M.C., Sahli, H.: Multimodal measurement of depression using deep learning models. In Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge (pp. 53–59). ACM(2017, October).

    Google Scholar 

  14. Dibeklioğlu, H., Hammal, Z., Cohn, J.F.: Dynamic multimodal measurement of depression severity using deep autoencoding. IEEE J. Biomed. Health Inform. 22(2), 525–536 (2018)

    Article  Google Scholar 

  15. Girard, J.M., Cohn, J.F., Mahoor, M.H., Mavadati, S., Rosenwald, D.P.: Social risk and depression: Evidence from manual and automatic facial expression analysis. In 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) (pp. 1–8). IEEE (2013, April).

    Google Scholar 

  16. Wang, Z., Yuan, C. M., Huang, J., Li, Z. Z., Chen, J., Zhang, H. Y., et al.: Reliability and validity of the Chinese version of Beck Depression Inventory-II among depression patients. Chinese Mental Health J. 25(6), 476–480 (2011)

    Google Scholar 

  17. Beck, A.T., Steer, R.A., Brown, G.K.: Beck depression inventory-II. San Antonio 78(2), 490–498 (1996)

    Google Scholar 

  18. Wang, X. D., Wang, X. L., Ma, H.: Manual of mental health assessment scales. Chinese Mental Health Journal(supplement) (1999)

    Google Scholar 

  19. Radloff, L.: The CES-D scale: a self-report depression scale for research in the general population. Applied Psychological Measurement, 1(3), 385–401(1977)

    Google Scholar 

  20. Hamilton, M.: Development of a rating scale for primary depressive illness. Br. J. Clin. Psychol. 6(4), 278–296 (1967)

    Article  Google Scholar 

  21. Liu, J.Q., Huang, Y., Huang, X.Y., Xia, X.T., Niu, X.X., Chen, Y.W.: Multimodal Behavioral Dataset of Depressive Symptoms in Chinese College Students–Preliminary Study. In: Chen YW. et. al. (eds) Innovation in Medicine and Healthcare Systems, and Multimedia. Smart Innovation, Systems and Technologies, vol 145, pp. 79–190. Springer, Singapore (2019)

    Google Scholar 

  22. McIntyre, G., Göcke, R., Hyett, M., et al.: An approach for automatically measuring facial activity in depressed subjects[C]//2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. IEEE, 1–8 (2009)

    Google Scholar 

  23. Pan, W., Wang, J., Liu, T., et al.: Depression recognition based on speech analysis[J]. Chin. Sci. Bull. 63(20), 2081–2092 (2018)

    Article  Google Scholar 

  24. Wang, J.Y.: An exploratory study on auxiliary diagnosis of depression based on speech. Doctoral Dissertation. Chinese Academy of Science, Beijing (2017)

    Google Scholar 

  25. Joshi, J., Goecke, R., Parker, G., Breakspear, M.: Can body expressions contribute to automatic depression analysis?. In 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) (pp. 1–7). IEEE (2013, April)

    Google Scholar 

  26. Ringeval F., Schuller B., Valstar M., Cummins N., Cowie R., Tavabi L., Schmitt M., Alisamir S., Amiriparian S., Messner E.-M., et al.: Avec 2019 workshop and challenge: state-of-mind, detecting depression with ai, and cross-cultural affect recognition. In Proceedings of the 9th International on Audio/Visual Emotion Challenge and Workshop, pp. 3–12, (2019)

    Google Scholar 

  27. Florian E., Felix W., Florian G., and Björn S.: Recent Developments in openSMILE, the Munich Open-Source Multimedia Feature Extractor. In Proc. 21st ACM International Conference on Multimedia (ACM MM). ACM, Barcelona, Spain, 835–838 (2013)

    Google Scholar 

  28. Florian, E., Klaus, R.S., Björn, S., Johan, S., Elisabeth, A., Carlos, B., Laurence, D., Julien, E., Petri, L., Shrikanth, S.N., Khiet, P.T.: The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing. IEEE Trans. Affect. Comput. 7(2), 190–202 (2016)

    Article  Google Scholar 

  29. Maximilian S., Björn S.: openXBOW—Introducing the Passau Open-Source Crossmodal Bag-of-Words Toolkit. Journal of Machine Learning Research (2017)

    Google Scholar 

  30. Karen S., Andrew Z.: Very deep convolutional networks for large-scale image recognition. https://arxiv.org/abs/1409.1556. 14 pp (2014)

  31. Gao H., Zhuang L., Laurens van der M., Kilian Q.W.: Densely Connected Convolutional Networks. In The IEEE Conference on Computter Vision and Pattern Recognition (CVPR). IEEE, Honolulu, HW, 4700–4708 (2017)

    Google Scholar 

  32. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: A Large-Scale Hierarchical Image Database, In The IEEE Conference on Computter Vision and Pattern Recognition (CVPR) (2009)

    Google Scholar 

  33. Tadas, B., Amir, Z., Yao, C.L., Louis, P.M.: OpenFace 2.0: Facial Behavior Analysis Toolkit. In Proc. 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). IEEE, Xi’an, P. R. China, 59–66 (2018)

    Google Scholar 

  34. Dimitrios K., Panagiotis T., Mihalis A. N., Athanasios P., Guoying Z., Björn S., Irene K., Stefanos Z.: Deep affect prediction in-the-wild: Aff-wild database and challenge, deep architectures, and beyond. Int. J. Comput. Vis. 127(6), 907–929 (2019)

    Google Scholar 

  35. Cho, K., van Merrienboer, B., Gulcehre, C., Bougares, F., Schwenk, H., and Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. CoRR, abs/1406.1078, (2014)

    Google Scholar 

  36. Chung J., Gülçehre C.,Cho K., Bengio Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR, abs/1412.3555, (2014)

    Google Scholar 

  37. Lawrence I-Kuei L.: A concordance correlation coef- ficient to evaluate reproducibility, Biometrics, pp. 255–268 (1989)

    Google Scholar 

Download references

Acknowledgements

This work is supported in part by Japan Society of Promotion of Science (20J13009).

Author information

Authors and Affiliations

  1. College of Information Science and Engineering, Ritsumeikan University, Shiga, Japan

    Jiaqing Liu, Shurong Chai & Yen-Wei Chen

  2. School of Education, Soochow University, Jiangsu, China

    Yue Huang & Xinyin Huang

  3. College of Computer Science and Technology, Zhejiang University, Hangzhou, China

    Hao Sun, Lanfen Lin & Yen-Wei Chen

  4. Zhejiang Lab, Research Center for Healthcare Data Science, Hangzhou, China

    Yen-Wei Chen

  5. Westlake University, Hangzhou, China

    Yue Huang

Authors
  1. Jiaqing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yue Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shurong Chai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hao Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xinyin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lanfen Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yen-Wei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xinyin Huang .

Editor information

Editors and Affiliations

  1. Institute for Intelligent Systems Research and Innovation, Deakin University, Waurn Ponds, VIC, Australia

    Chee-Peng Lim

  2. Royal Adelaide Hospital, Adelaide, SA, Australia

    Ashlesha Vaidya

  3. St. Joseph’s Hospital and Medical Center, Stockton, CA, USA

    Kiran Jain

  4. Avanti Institute of Cardiology, Nagpur, Maharashtra, India

    Virag U. Mahorkar

  5. KES International, Shoreham-by-Sea, UK

    Lakhmi C. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Liu, J. et al. (2022). Computer-Aided Detection of Depressive Severity Using Multimodal Behavioral Data. In: Lim, CP., Vaidya, A., Jain, K., Mahorkar, V.U., Jain, L.C. (eds) Handbook of Artificial Intelligence in Healthcare. Intelligent Systems Reference Library, vol 211. Springer, Cham. https://doi.org/10.1007/978-3-030-79161-2_14

Download citation

Publish with us

Policies and ethics

Search

Navigation