Outcomes and Alliance Results of Teletherapy versus In-Person Therapy

Ashley E. Simon, Jeb S. Brown, and Justin K. Turner

Center for Clinical Informatics

Abstract

During the last year and a half, the COVID-19 pandemic has precipitated a massive increase in the use of teletherapy. The purpose of the article is to utilize data collected by the ACORN collaboration to explore the differences between teletherapy services, in-person psychotherapy services and a mix of both modes of treatment. This study aims to provide further evidence that teletherapy is as effective as in person therapy, and to expand the literature by presenting findings on the efficacy of mixing these administration methods using large data sets (N=12,192 for adults and N=5,979 for youth). For adults, the comparison in Severity Adjusted Effect Size between all teletherapy (M=.73 SD=1.08) and all in-person conditions (M=.76 SD=1.12) was non-significant. However, the mean SAES for the mixed condition (M=.85 SD=1.16) was significantly larger than either all teletherapy or all in-person (p<.001). An analysis of variance indicated that while much of the difference was due to the greater number of sessions, the difference in SAES for the mixed condition remained significant after controlling for session count (p<.001). For youth, unlike with the adults, the comparison in SAES between all teletherapy (M=.87 SD=1.00) and all in-person conditions (M=.73 SD=1.05) was significant at (p<.001). The mixed condition (M=87 SD=1.10) had the same SAES as the all-teletherapy group. This result indicates that teletherapy and mixed delivery had better outcomes for youth when compared to all in-person therapy.

Purpose

The COVID-19 pandemic and lockdown measures associated with it brought about a rapid and large-scale increase in the use of teletherapy (Burgoyne et al., 2020). Teletherapy is defined here as psychotherapy conducted over internet enabled devices rather than in person. The purpose of the article is to utilize data collected by the ACORN collaboration to explore the differences between teletherapy services and in-person psychotherapy services. Beyond those two groups, we also examine mental health outcomes and therapeutic alliance of a third group: combination of teletherapy and in-person psychotherapy. This study aims to provide further evidence that teletherapy is as effective as in person therapy, and to expand the literature by presenting findings on the efficacy of mixing these administration methods. Prior published research has indicated little difference in satisfaction and feasibility between in-person or teletherapy services (Backhaus et al., 2012) and efficacy (Fernandez et al., 2021). This paper will expand this question using large data sets (N=12,192 for adults and N=5,979 for youth), as well as comparing client-reported alliance scores across groups.

Method

Measures

The ACORN database consists of real-world mental health outcomes being collected across a variety of clinic types and payers. The populations served reflect a wide spectrum of diagnostic groups across regions within the United States, Australia and Canada. Treatment types included are outpatient, inpatient, and intensive outpatient settings. Data was used for clients entering treatment between 4/01/2020 and 04/01/2021.

The magnitude of improvement for all questionnaires was calculated using what is referred as a severity adjusted effect size (SAES). This statistic controls for severity of reported problems as well as diagnosis at intake, using a general linear model. Specifically, the SAES score comes from the mean of all non-missing items on the ACORN questionnaire that correlate with global distress (a single factor mental health metric used to measure distress) (Brown et al., 2015).

When evaluating the possible impact of teletherapy versus in-person therapy, the samples were divided into three groups: 100% teletherapy, mixed teletherapy and in-person, and 100% in-person. While this represents the reality of teletherapy use in a real-world setting, it also allowed a more nuanced look at the impact of mode of therapy rather than simply dividing the sample into just teletherapy versus an in-person comparison.

Previous research has found therapeutic alliance to be comparable between in-person and teletherapy (Simpson et al., 2001; Germain et al., 2010), and we examined therapeutic alliance in this analysis with the hypothesis that it would be similar across the three conditions. At least one program report has also found an increase in the number of sessions utilized for teletherapy (Burgoyne & Cohn, 2020).

Results

Adult Session Count

For adults, the average session count was significantly higher (p<.001) for the mixed condition (M=10.56 SD=8.79) compared to the all teletherapy group (M=6.26 SD=7.03) and all in-person group (M=7.56 SD=7.44) (see table 1).

Adult Severity Adjusted Effect Size

The comparison in SAES between all teletherapy (M=.73 SD=1.08) and all in-person conditions (M=.76 SD=1.12) were non-significant, consistent with finding in a large meta-analysis comparing the efficacy of one condition to another (Fernandez et al., 2021). However, the mean SAES for the mixed condition (M=.85 SD=1.16) was significantly larger than either all teletherapy or all in-person (p<.001). An analysis of variance indicated that while much of the difference was due to the greater number of sessions, the difference in SAES for the mixed condition remained significant after controlling for session count (p<.001). See discussion and summary for possible implications.

Adult Alliance

A note: lower alliance scores indicate more satisfaction with the therapeutic alliance.

For adults, last alliance scores were lower for the mixed condition (M=.14 SD=.42) compared to the all teletherapy group (M=.35 SD=.71) and the all in-person group (M=.33 SD=.66). This indicates that the mixed condition had better alliance scores at the last session when compared to the all teletherapy or all in person group (p<.001).

Youth Session Count

 One of the first apparent differences was that the youth were more likely to use teletherapy exclusively (M=7.02 SD=5.71) than adults (M=6.26 SD=7.03). As with the adult sample, the mixed condition had the highest average number of sessions (M=8.86 SD=6.92).

Youth Severity Adjusted Effect Size

Unlike with the adults, the comparison in SAES between all teletherapy (M=.87 SD=1.00) and all in-person conditions (M=.73 SD=1.05) was significant at (p<.001). The mixed condition (M=87 SD=1.10) had the same SAES as the all-teletherapy group (M=.87 SD=1.00). This result indicates that teletherapy and mixed delivery had better outcomes for youth when compared to all in-person therapy.

Youth Alliance

For youth last alliance averages, the mixed condition last alliance average (M=.40 SD=.67) was similar to the all-teletherapy group (M=.40 SD=.68). There was a significant difference (p<.001) between the all in-person group (M=.53 SD=.88).and the other two groups: all teletherapy (M=.40 SD=.68) and the mixed condition (M=.40 SD=.67).

Discussion and Summary

Overall, the differences in outcomes for in-person therapy compared to teletherapy were negligible for adults while youth reported significantly greater improvement in the teletherapy condition. However, for the clients receiving a mixture of in-person and teletherapy (average mix of services between 49% and 50%), both the number of sessions provided, and the mean SAES were significantly higher, with a SAES of .87 for youth and .85 for adults. Alliance between the three conditions was comparable for both Adults and Youth. The implications for therapists are to consider continuing to routinely offer the option of teletherapy when possible after the COVID pandemic has receded. Results suggest that many clients may find a mix of modalities helpful. One program report found that one downside of teletherapy in their clinic is a decrease in the quantity and quality of feedback (Burgoyne & Cohn, 2020). It is therefore recommended that a process of feedback informed treatment is implemented to monitor client feedback, especially in teletherapy settings.

 References

Backhaus, A., Agha, Z., Maglione, M. L., Repp, A., Ross, B., Zuest, D., & Thorp, S. R. (2012). Videoconferencing psychotherapy: A systematic review. Psychological Services, 9, 111-131. doi: 10.1037/10027924

Brown, G. S. (J.), Simon, A., Cameron, J., & Minami, T. (2015). A collaborative outcome resource network (ACORN): Tools for increasing the value of psychotherapy. Psychotherapy, 52(4), 412–421. https://doi.org/10.1037/pst0000033

Burgoyne, N., & Cohn, A.S. (2020). Lessons from the transition to relational teletherapy during COVID-19. Family Process, 59, 974-988. https://doi.org/10.1111/famp.12589

Fernandez, E., Woldgabreal, Y., Day, A., Pham, T., Gleich B., Aboujaoude, E. (2021). Live psychotherapy by video versus in-person: A meta-analysis of efficacy and its relationship to types and targets of treatment. Clinical Psychology & Psychotherapy. doi: 10.1002/cpp.2594

Germain, V., Marchand, A., Bouchard, S. Guay, S., & Drouin, M. (2010). Assessment of the therapeutic alliance in face-to-face or videoconference treatment for posttraumatic stress disorder. Cyberpsychology, Behavior, and Social Networking, 13, 29-25. doi:10.1089=cyber.2009.0139

Simpson, S. (2001) The provision of a telepsychology service to Shetland: Client and therapist satisfaction and the ability to develop a therapeutic alliance. Journal of Telemedicine and Telecare, 7, 34-36. doi:10.1258/1357633011936633