Abstract

E-health is opening new prospects in the management of mental disorders. Virtual reality (VR)-based interventions, which provide a safe nonjudging environment, may improve symptoms awareness in patients with depressive disorders. This study aimed to explore the feasibility of a novel VR software, the VRight, in depressive patients. Adult outpatients with depressive disorders presenting to our mental health clinic during December 2018 were invited to participate in the study. Participants completed a satisfaction survey at the end of the session, including questions about user-friendliness and about usefulness of the software. The Patient Health Questionnaire-9 (PHQ-9) was administered to participants before and after the VR-based intervention to assess depressive symptoms severity. We explored the correlation between the score difference and the variables: age, gender, and initial score. Twenty-eight patients (71.4 percent women, mean age: 51.18 ± 16.13 years) were recruited. Thirteen subjects had major depressive disorder (46.4 percent). Most of the patients (n = 26) were satisfied with the VR experience, which they found to be helpful. PHQ-9 score difference and the initial score correlated positively at a significant level. There was no significant correlation between score difference and age or gender. VRight was well accepted among depressive patients, showing high levels of satisfaction. The VRight could contribute to increase depressive symptoms awareness, which is of clinical relevance given the association of greater insight with positive outcomes in depression. Further studies are needed to confirm the effectiveness of VRight as a psychoeducational tool in clinical practice.

Introduction

Depression is a major cause of disability worldwide and a well-recognized risk factor for suicide.1,2 According to the World Health Organization, >300 million people are currently estimated to be suffering from depression.3 However, data from the World Mental Health surveys from 23 countries showed that only 16.5 percent of patients with major depression received adequate treatment.4
One of the main barriers to access to treatment among depressed patients is self-stigma—the internalization of social prejudices by the affected person. Self-stigma can make patients unaware of their own symptoms and unwilling to seek help.5 False beliefs, such as the notion that recovery from depression will occur with no treatment, can also result in poorer insight and lower rates of help-seeking.6,7
To address the gap in depression treatment, new approaches to mental disorders are emerging, including the application of new technologies in health care settings.8 Although this technology cannot replace human interaction, it may have some advantages compared with the traditional treatment. For instance, patients may feel more comfortable when discussing their symptoms through interactive software than when doing so on a face-to-face basis.9 This technology could be, therefore, used as a psychoeducational tool, which may improve patient's symptoms awareness and patient involvement in his/her own care. In keeping with this, virtual reality (VR) has already been demonstrated to be useful in some mental disorders.10,11
VR consists of a 3D virtual environment in which users navigate and interact with objects and characters.11 The first clinical trial exploring the use of VR in mental health focused on phobias in 1995.12 Since then, VR-based interventions have been proven effective as a coadjuvant treatment in some mental disorders, such as post-traumatic stress disorder (PTSD), phobias, and depression.12–14 The effectiveness of a VR-Based Stress Management Program in patients with mood disorders, which resulted in decreased anxiety and depression levels, was also reported.15 Furthermore, a 2018 meta-analysis of VR interventions for anxiety and depression found moderate to large effect sizes of VR on several outcome measures compared with controls.16 In addition, VR may improve symptoms awareness since it creates an immersive experience in a nonjudgmental environment.
In this study, we explore the feasibility of VRight, a VR psychoeducational tool designed to increase symptom awareness in depressive patients. We hypothesize that VRight will be easily applicable to participants, and will result in high levels of satisfaction.

Methods

This study was performed in agreement with the 1964 Declaration of Helsinki and further amendments.17 Ethical approval was obtained from the Local Research Ethics Committee of University Hospital Jimenez Diaz Foundation (Madrid, Spain). Participants gave written informed consent.

Sample

Adult patients (aged 18–65 years) receiving care at the outpatient Mental Health Clinic of University Hospital Jimenez Diaz Foundation (Madrid, Spain) were invited to participate in the study. Having a depressive spectrum disorder based on ICD-10 criteria18 was the only inclusion criterion. Patients were excluded if they suffered from cognitive impairment that precluded understanding and signing the informed consent. Diagnosis was established clinically and collected from the patient's clinical records.

Measures

Depressive symptoms were measured using the Patient Health Questionnaire-9 (PHQ-9) scale.19 The PHQ-9 is an assessment tool based on DSM-IV criteria,20 which has been validated in different populations, including high interrater reliability.21 This questionnaire has been used by previous studies exploring e-health interventions.9
Participants' satisfaction with the VRight was surveyed based on a previously used questionnaire.22,23 Elements explored in the survey included usefulness of the software, user-friendliness, and overall satisfaction with the VRight experience. The survey included four yes/no questions, four Likert-scale questions, and six open questions, which allowed participants to comment on relevant aspects of the software. The satisfaction survey is shown in Supplementary Table S1.
In addition, age and gender were considered for the analyses.

VR software: VRight

A novel VR software, which is known as VRight, was designed by a multidisciplinary team of computing engineers and senior consultant psychiatrists and psychologists. VRight runs on the head-mounted-device (HMD) Oculus' Rift® Reality Smartglasses, which is a portable and user-friendly VR device.24
The VRight simulates a peer-to-peer interaction, which consists of a conversation that the participant must hold with a digital character. The conversation, which takes place in a bar (Fig. 2), is about symptoms of depression. Participants find themselves seated in front of the character, which asks them yes/no questions. The VRight has a voice recognition system capable of interpreting the yes/no answers of the participants. A therapist supervised the intervention in case any computer error occurred. Characters were matched with gender and they were created with unspecific physical features to avoid cultural biases.25 Figure 1 shows snapshots of the VRight software. The conversation script is shown in Figure 2.
FIG. 1. Female and male VRight avatars.
FIG. 2. VRight's avatar screenplay.

Procedure

At the beginning of the interview the PHQ-9 was administered to all participants by trained research staff. Shortly after, the VRight intervention was delivered. As alluded to earlier, each participant interacted with a gender-matched avatar and all VR experiences were recorded. The VRight intervention takes ∼10 minutes, including about 5 minutes of preparation and another 5 minutes of conversation with the avatar. Five minutes after the VRight intervention the PHQ-9 was administered again by a different researcher. At the end of the intervention, participants were asked to complete the satisfaction survey.

Statistical analysis

A descriptive statistical analysis of the satisfaction questionnaire was performed to evaluate the acceptability of the developed VR software among the participants.
In addition, difference between the final and initial PHQ-9 scores was calculated and a Spearman`s rho correlation was performed to analyze the linear relationship between the initial PHQ score and the score difference. Finally, a Wilcoxon rank test was conducted to find out if the age or gender of the participants influences the change of the PHQ-9 scores.

Results

Twenty-eight individuals took part in this study, including n = 20 women (71.4 percent). Mean age of the sample was 51.2 ± 16.1 years. ICD-10 diagnoses of participants were as follows: major depressive disorder (n = 13, 46.4 percent), adjustment disorder (n = 8, 28.6 percent), and dysthymia (n = 4, 14.3 percent).
Most patients (n = 26) were overall satisfied with the VRight software, which they considered to be helpful. Fourteen patients considered that the session was too short (they wished it would have lasted for longer). In the open questions, the most common positive comments were on “enhancement of help-seeking” (n = 6), “identification with the character” (n = 6), and “self-awareness gain” (n = 16). The most common negative comments included “poor quality of graphs” (n = 3) and “excessive simplicity” (n = 2). See further details of the satisfaction survey in Tables 1 and 2.
Table 1. Quantitative Results of the Satisfaction Survey
QuestionMean ± SD95 percent CI
Was the software easy to use? (yes = 1; no = 0)0.96 ± 0.190.89–1.00
Was the software engaging/inviting/interesting? (yes = 1; no = 0)0.89 ± 0.310.77–1.00
Did the software do what you expected? (yes = 1; no = 0)0.84 ± 0.370.68–0.99
Has VRight changed the way you perceive your symptoms? (yes = 1; no = 0)0.78 ± 0.410.62–0.9
Did you find it entertaining? (scale 1–5)3.82 ± 1.163.37–4.27
Please rate your overall satisfaction with the session (scale 1–5)3.92 ± 1.023.53–4.32
Please rate the usefulness of the VRight software (scale 1–5)3.74 ± 1.243.27–4.23
How much would you recommend this software to other patients? (scale 1–5)4.18 ± 1.103.76–4.60
95 percent CI, 95 percent confidence interval; SD, standard deviation.
Table 2. A Sample of the Answers to the Open Questions of the Satisfaction Survey
Were there any features that you specially liked? Which ones?
She said a lot of things that felt familiar
I liked that it was a casual conversation
The level of detail, the walls, the blackboard, the furniture… It all looked the part
Closeness, although I prefer a real person
Novelty
The insulation of the environment and the intimacy with the character
Were there any features you did not like or found useless? Which ones?
It made me feel dizzy
It was too basic
It was too obvious, and very similar to the questionnaire
The main character didn´t feel real
I feel anguished after having identified myself with the character
I couldn't identify myself with the character
Have you learnt anything after using VRight? What have you learnt?
To better understand myself
It is easier to identify symptoms in other people rather than yourself
It is worthy asking for help instead of thinking about death all the time. Life is worth living
Hiding my problems won't help
That there are a lot of people in my situation
Sometimes you perceive things differently if you compare yourself to others
Were there any features more useful than others? Which ones?
To recognize your own symptoms and problems
The conversation
That there are other people getting through a hard time
The description of the symptoms of depression is very complete
Realizing that even though I have everything (a home, a family, a job) I do not feel happy
That describes well what depression is
Would you like to have more sessions with the VRight? Why?
It was too short
It's something original and there's an interchange of roles. The patient is now the one that has to listen and give counsel
I would have liked to dig deeper into some topics (life habits, apathy, wish to die…)
To help other people
It would be interesting to interact with different characters or in different settings
To identify my symptoms more clearly
Do you have any questions or would like to add anything?
I think is a good initiative to use new technologies to help patients
I think it would be helpful to be able to create your own avatar
I would have liked it to be more interactive
Sessions should be a little longer
I empathized a lot with the character. She was so young and I would like to help her
It could be an interesting approach
The initial score and the score difference correlated significantly (r = 0.39, p = 0.02), that is, those patients with higher baseline scores showed a greater difference in PHQ-9 scores before and after the VRight intervention. There were no significant associations between score difference and age and gender (Fig. 3).
FIG. 3. Scatterplot of initial and difference score variables. The color and size of the markers indicate the gender and age of the participants, respectively.

Discussion

The VRight intervention showed high levels of satisfaction among users. Patients considered it useful and user-friendly. The participants' feedback suggests that the VRight is a potentially useful tool to increase symptoms awareness, which may lead to greater help-seeking levels among patients with depressive disorders.
Only a few previous studies investigated the possibilities of VR-based interventions as psychoeducational tools. Falconer et al. designed a VR software called avatar-MBT to deliver conventional mentalization-based treatment (MBT) for patients with borderline personality disorder, which was found to enhanced treatment adherence.26 Previous studies also explored the use of VR as part of depression treatment.16,25,27 However, interventions focused on symptoms awareness have been neglected in research.
In our study, we found that those participants with lower baseline scores showed a decrease in PHQ-9 scores after the VRight intervention. In contrast, those with a higher baseline scores showed a greater increase in PHQ-9 scores after the VRight intervention. This difference in the PHQ-9 score may indicate an increase in symptoms awareness. More factors may be involved in this finding. For instance, we cannot rule out that the VRight environment works as negative induction scenario. However, the participants' answers to the satisfaction survey indicate the VRight intervention is valued as a positive experience.
By interacting with a digital avatar, patients may feel in a safe nonjudging environment, while feeling that they are empathizing with a peer. Several studies showed the benefits of peer-to-peer interactions in mental health care.28,29 Expressing negative emotions and worries to someone who has gone through the same experience may increase insight and reduce stigma.30
Improvement in symptoms awareness, which can be considered as an insight dimension, has important consequences for clinical outcomes as a poor insight is linked with decreased treatment adherence.31 Being aware of your own symptoms is an important step toward adequate treatment adherence and treatment response.31 Better insight also contributes to treatment-seeking and achieving recovery,32 which is especially relevant in mental health given the low adherence rates and frequent lack of patient cooperation.33,34 The effect of VRight on insight might be due to the simulation of the peer-to-peer experience. Some studies found peer-to-peer interventions to be more effective for gaining self-awareness than other therapies.35,36 However, longer-term studies using this technology need to clarify whether the observed symptoms awareness gain is sustained over time and whether this may increase the risk of hopelessness, suicidal ideation, and suicidal behavior.
VR-based interventions and other e-health interventions, such as MEmind or Eb2, therefore, offer novel alternatives for the management of mental disorders, which have also been supported from a cost-effectiveness perspective.23 However, several issues remain to be addressed before this technology can be routinely used in clinical settings. Engagement of patients with e-health interventions faces some challenges. For instance, a study exploring the use of m-health technologies in PTSD patients showed that, whereas around half of them were interested on m-health interventions, the number of subjects engaged in using mobile health applications for the management of their mental health conditions was around five times lower.37 This could be, in part, explained by the fact that e-health interventions are not usually covered by national health care systems, which may lead to concerns about their effectiveness, including those raised by health care professionals. In addition, stigma and related issues may play a part in this. For instance, veterans without PTSD were shown to be more interested in engaging in mental health care applications than those with PTSD according to one study, even though the latter group used mental health services more frequently.38 Difficulties in the use of the new technologies by both users and health care professionals are also important barriers that could be overcome by the implementation of specific training programs.
VR has been relatively neglected by ongoing research in the field of e-health. Although the number of available e-health interventions has significantly increased for the past few years, studies exploring the effectiveness of these interventions are limited.39 Most e-health interventions have not been sufficiently investigated in terms of tolerability and effectiveness. Not only further effectiveness studies are needed, but also new tools may be developed in the years ahead. In particular, the graphs and interface may be improved to make the software more attractive to patients as the poor quality of the graphs was reported by the participants in our study as one of the main negatives. Of concern, commercial interests may hinder research in this area, hence proper investment from public health systems is required. To progress in this field, health care professionals and computing engineers need to work collaboratively, which will more likely result in better clinical outcomes for patients.
To our knowledge, this is the first study exploring the feasibility of a VR-based intervention to improve symptoms awareness in depressive patients.
Our findings should be considered in light of some limitations. First, the small sample size may have limited the study power. Second, although the PHQ-9 scale was shown to have very good inter- and intrarater reliability,21,22 the assessor was not blind to the pre-VRight scores and the patient's exposure to the intervention, which may have affected the post-VRight ratings. Finally, post-VRight PHQ-9 scores could be the result of a negative induction scenario.

Supplementary Material

File (suppl_tables1.pdf)

References

1. Richards D. Prevalence and clinical course of depression: a review. Clinical Psychology Review 2011; 31:1117–1125.
2. Hawton K, Casañas I, Comabella C, et al. Risk factors for suicide in individuals with depression: a systematic review. Journal of Affective Disorders 2013; 147:17–28.
3. WHO. Depression. www.who.int/news-room/fact-sheets/detail/depression. Published 2018 (accessed Jan. 30, 2019).
4. Thornicroft G, Chatterji S, Evans-Lacko S, et al. Undertreatment of people with major depressive disorder in 21 countries. British Journal of Psychiatry 2017; 210:119–124.
5. Oexle N, Müller M, Kawohl W, et al. Self-stigma as a barrier to recovery: a longitudinal study. European Archives Of Psychiatry And Clinical Neuroscience 2017; 268:209–212.
6. Boerema AM, Ten Have M, Kleiboer A, et al. Demographic and need factors of early, delayed and no mental health care use in major depression: a prospective study. BMC Psychiatry 2017; 17:367.
7. Tomczyk S, Muehlan H, Freitag S, et al. Is knowledge “half the battle”? The role of depression literacy in help-seeking among a non-clinical sample of adults with currently untreated mental health problems. Journal of Affective Disorders 2018; 238:289–296.
8. O'Dea B, Calear A, Perry Y. Is e-health the answer to gaps in adolescent mental health service provision? Current Opinion in Psychiatry 2015; 28:336–342.
9. Torous J, Staples P, Shanahan M, et al. Utilizing a personal smartphone custom app to assess the Patient Health Questionnaire-9 (PHQ-9) depressive symptoms in patients with major depressive disorder. JMIR Mental Health 2015; 2:e8.
10. Freeman D, Reeve S, Robinson A, et al. Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychological Medicine 2017; 47:2393–2400.
11. Mishkind M, Norr A, Katz A, et al. Review of virtual reality treatment in psychiatry: evidence versus current diffusion and use. Current Psychiatry Reports 2017; 19:80.
12. Šendula-Jengi V, Šendula-Pavelić M, Hodak J. Mind in the gap between neural and social networks—cyberspace and virtual reality in psychiatry and healthcare. Psychiatria Danubina 2016; 28:100–103.
13. Li J, Theng Y-L, Foo S. Game-based digital interventions for depression therapy: a systematic review and meta-analysis. Cyberpsychology, Behavior and Social Networking 2014; 17:519–527.
14. Cardoş RAI, David OA, David DO. Virtual reality exposure therapy in flight anxiety: a quantitative meta-analysis. Computers in Human Behavior 2017; 72:371–380.
15. Shah LBI, Torres S, Kannusamy P, et al. Efficacy of the Virtual Reality-Based Stress Management Program on Stress-Related Variables in People With Mood Disorders: the Feasibility Study. Archives of Psychiatric Nursing 2015; 29:6–13.
16. Fodor LA, Coteț CD, Cuijpers P, et al. The effectiveness of virtual reality based interventions for symptoms of anxiety and depression: a meta-analysis. Scientific Reports 2018; 8:10323.
17. Shephard DA. Declaration of Helsinki and consent. Canadian Medical Association Journal 1976; 115:1191–1192.
18. World Health Organization (WHO). (1992) Tenth Revision of the International Classification of Diseases, Chapter V (F): Mental and Behavioural Disorders. Clinical descriptions and diagnostic guidelines. Geneva: WHO.
19. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. Journal of General Internal Medicine 2001; 16:606–613.
20. American Psychiatric Association (APA). (2000) Diagnostic and Statistical Manual of Mental Disorders: DSM-IV. Washington: American Psychiatric Association.
21. Kroenke K, Spitzer RL, Williams JBW, et al. The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: a systematic review. General Hospital Psychiatry 2010; 32:345–359.
22. Kocalevent R, Hinz A, Brähler E. Standardization of the depression screener Patient Health Questionnaire (PHQ-9) in the general population. General Hospital Psychiatry 2013; 35:551–555.
23. Mosadeghi S, Reid MW, Martinez B, et al. Feasibility of an immersive virtual reality intervention for hospitalized patients: an observational cohort study. JMIR Mental Health 2016; 3:e28.
24. Jerdan SW, Grindle M, van Woerden HC, et al. Head-mounted virtual reality and mental health: critical review of current research. JMIR Serious Games 2018; 6:e14.
25. Sanchez K, Eghaneyan B, Killian M, et al. Measurement, Education and Tracking in Integrated Care (METRIC): use of a culturally adapted education tool versus standard education to increase engagement in depression treatment among Hispanic patients: study protocol for a randomized control trial. Trials 2017; 18:363.
26. Falconer CJ, Cutting P, Bethan Davies E, et al. Adjunctive avatar therapy for mentalization-based treatment of borderline personality disorder: a mixed-methods feasibility study. Evidence Based Mental Health 2017; 20:123–127.
27. Decker V, Valenti M, Montoya V, et al. Maximizing new technologies to treat depression. Issues Mental Health Nursing 2019;40:200–207.
28. Bouchard L, Montreuil M, Gros C. Peer support among inpatients in an adult mental health setting. Issues Mental Health Nursing 2010; 31:589–598.
29. Joo JH, Hwang S, Abu H, et al. An innovative model of depression care delivery: peer mentors in collaboration with a mental health professional to relieve depression in older adults. The American Journal of Geriatric Psychiatry: Official Journal of the American Association for Geriatric Psychiatry 2016; 24:407–416.
30. Hamblen J, Grubaugh A, Davidson T, et al. An online peer educational campaign to reduce stigma and improve help seeking in veterans with posttraumatic stress disorder. Telemedicine and e-Health 2019; 25:41–47.
31. Tessier A, Boyer L, Husky M, et al. Medication adherence in schizophrenia: the role of insight, therapeutic alliance and perceived trauma associated with psychiatric care. Psychiatry Research 2017; 257:315–321.
32. Magaard J, Seeralan T, Schulz H, et al. Factors associated with help-seeking behaviour among individuals with major depression: a systematic review. PLoS One 2017; 12:e0176730.
33. Lacro J, Dunn L, Dolder C, et al. Prevalence of and risk factors for medication nonadherence in patients with schizophrenia. The Journal of Clinical Psychiatry 2002; 63:892–909.
34. Bulloch A, Patten S. Non-adherence with psychotropic medications in the general population. Social Psychiatry and Psychiatric Epidemiology 2009; 45:47–56.
35. Naslund JA, Aschbrenner KA, Marsch LA, et al. The future of mental health care: peer-to-peer support and social media. Epidemiology and Psychiatric Sciences 2016; 25:113–122.
36. Debyser B, Berben K, Beeckman D, et al. The transition from patient to mental health peer worker: a grounded theory approach. International Journal of Mental Health Nursing 2019; 28:560–571.
37. Erbes CR, Stinson R, Kuhn E, et al. Access, utilization, and interest in mHealth applications among veterans receiving outpatient care for PTSD. Military Medicine 2014; 179:1218–1222.
38. Miller CJ, McInnes DK, Stolzmann K, et al. Interest in use of technology for healthcare among veterans receiving treatment for mental health. Telemedicine Journal and e-health: The Official Journal of the American Telemedicine Association 2016; 22:847–854.
39. Martínez-Pérez B, de la Torre-Díez I, López-Coronado M. Mobile health applications for the most prevalent conditions by the World Health Organization: review and analysis. Journal of Medical Internet Research 2013; 15:e120.

Information & Authors

Information

Published In

cover image Cyberpsychology, Behavior, and Social Networking
Cyberpsychology, Behavior, and Social Networking
Volume 23Issue Number 4April 2020
Pages: 246 - 252
PubMed: 32207997

History

Published online: 7 April 2020
Published in print: April 2020
Published ahead of print: 24 March 2020

Permissions

Request permissions for this article.

Topics

Authors

Affiliations

Marta Migoya-Borja
Department of Psychiatry, University Hospital Jimenez Díaz Foundation, Madrid, Spain.
David Delgado-Gómez
Department of Statistics, Universidad Carlos III, Getafe, Madrid, Spain.
Rodrigo Carmona-Camacho
Department of Psychiatry, University Hospital Jimenez Díaz Foundation, Madrid, Spain.
Alejandro Porras-Segovia
Department of Psychiatry, University Hospital Jimenez Díaz Foundation, Madrid, Spain.
Javier-David López-Moriñigo
Department of Psychiatry, Madrid Autonomous University, Madrid, Spain.
Montserrat Sánchez-Alonso
Department of Psychiatry, University Hospital Jimenez Díaz Foundation, Madrid, Spain.
Lucía Albarracín García
Department of Psychiatry, University Hospital Jimenez Díaz Foundation, Madrid, Spain.
Nuria Guerra
Department of Psychiatry, University Hospital Jimenez Díaz Foundation, Madrid, Spain.
María Luisa Barrigón
Department of Psychiatry, University Hospital Jimenez Díaz Foundation, Madrid, Spain.
Department of Psychiatry, Madrid Autonomous University, Madrid, Spain.
Margarita Alegría
Massachusetts General Hospital, Disparities Research Unit and Harvard Medical School, Boston, Massachusetts.
Enrique Baca-García [email protected]
Department of Psychiatry, University Hospital Jimenez Díaz Foundation, Madrid, Spain.
Department of Psychiatry, Madrid Autonomous University, Madrid, Spain.
Department of Psychiatry, University Hospital Rey Juan Carlos, Mostoles, Spain.
Department of Psychiatry, General Hospital of Villalba, Madrid, Spain.
Department of Psychiatry, University Hospital Infanta Elena, Valdemoro, Spain.
Department of Psychiatry, Universidad Catolica del Maule, Talca, Chile.
CIBERSAM (Centro de Investigacion en Salud Mental), Carlos III Institute of Health, Madrid, Spain.
Department of Psychiatry, Centre Hospitalier Universitaire de Nîmes, Nîmes, France.

Notes

Address correspondence to: Dr. Enrique Baca-García, Department of Psychiatry, University Hospital Jimenez Diaz Foundation, Av. de los Reyes Católicos, 2 Madrid 28040, Spain [email protected]

Author Disclosure Statement

No competing financial interests exist.

Funding Information

This study was partially supported by a grant from Harvard University, Instituto de Investigación Carlos III (ISCIII PI16/01852 & CM19/00026), Delegacion del Gobierno para el Plan Nacional de Drogas (20151073), American Foundation for Suicide Prevention (LSRG-1-005-16), the Madrid Regional Government (B2017/BMD-3740 AGESCM 2CM; Y2018/TCS-4705 PRACTICO-CM), Structural Funds of the European Union. MINECO/FEDER (“ADVENTURE”, ID. TEC2015- 69868-C2-1-R) and MCIU Explora Grant “aMBITION” (ID. TEC2017-92552- EXP), Universidad Autónoma de Madrid (Spain) and European Union-European Commission through the Intertalentum Project-Marie Sklodovska Curie Actions research grant (GA 713366).

Metrics & Citations

Metrics

Citations

Export citation

Select the format you want to export the citations of this publication.

View Options

View options

PDF/EPUB

View PDF/ePub

Get Access

Access content

To read the fulltext, please use one of the options below to sign in or purchase access.

Society Access

If you are a member of a society that has access to this content please log in via your society website and then return to this publication.

Restore your content access

Enter your email address to restore your content access:

Note: This functionality works only for purchases done as a guest. If you already have an account, log in to access the content to which you are entitled.

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share on social media

Back to Top