The Relationship Between Session-to-Session Change on a Therapeutic Alliance Measure and Outcome of Treatment for Short Term Psychotherapy

Daryl Mahon, D.SoC, MA, Department of Applied Social Studies, National University of Ireland, Maynooth

George (Jeb) S. Brown, PhD, Center for Clinical Informatics

Ashley Simon, MA, Center for Clinical Informatics

Published on the Society for the Advancement of Psychotherapy

The therapeutic alliance (Bordin, 1979) remains one of the most studied constructs in psychotherapy outcome research (Norcross & Lambert, 2019), and is a robust predictor of client outcome (Flückiger et al., 2018; 2020; Wampold & Imel, 2015). It has been established that therapists vary in their ability to build and maintain an effective alliance, which has implications for practice as it has been identified that the clinicians contribution to the alliance is more important than that of the patients (Baldwin et al., 2007; Del Re et al., 2012). Furthermore,  early alliance predicts approximately 5% of the variance in treatment outcomes (Flückiger et al., 2018). In their meta-analysis with a dataset of 5,350, with a mean of 5.07 sessions per client, Flückiger et al. (2020) found improvements in alliance scores in one session are associated with reductions in symptoms in subsequent sessions. Specifically, every 1-point increase in alliance was followed by a 1-point decrease in symptoms.

This paper is a first in a series of two papers that further investigate the relationship between alliance scores, session by session change, and final outcome, using a larger sample (n=23,050) than reported in previous studies. The data comes from the ACORN collaboration database (Brown et al, 2015a; Brown et al, 2015b). These papers explore how the relationship between alliance scores and outcome may change depending on the length of treatment: up to five sessions, and six sessions up to 20 sessions. This first paper is focused on clients completing treatment within five sessions (n=14,134, 56% of total sample). This paper also explores the psychometric properties of an alliance measure and proposes a simple method to interpret changes in alliance scores.

Methods

QUESTIONNAIRES

This study utilizes a suite of questionnaires developed within the ACORN collaboration (Brown et al, 2015a).  See https://acorncollaboration.org/adult-question-bank for a list of items used on ACORN adult questionnaires. The questionnaires typically are 16 to 20 items in length, with subscales for global distress, substance abuse, and alliance. The global distress score includes items inquiring about symptoms of depression, anxiety, daily functioning, and social isolation/conflict. All the clusters of items load on a common factor, referred to as the Global Distress Scale (GDS). Specific versions of the questionnaires contain minor variations in item content to depending on the needs and preferences of the various clinics involved in the ACORN collaboration. Choice of item content is based also on the psychometric properties of each item, assuring that the alternative versions of the questionnaires will have very similar psychometric properties with regards to distribution of scores and pattern of change over time (Brown et al, 2015a). All items use a 5-point Likert scale. The questionnaire subscales are scored as the mean of the non-missing items resulting in a range of 0-4. Higher scores indicate greater disturbance in the alliance.

The alliance items are included in the questionnaire administered at the start of each session. Clinicians administer the alliance measure by asking the patient to score it based on their experience of the previous session. This method of collecting alliance scores in ACORN is used because clinicians are more likely to collect data this way, rather than administering another measure at the end of the session. The practical advantages of this approach (and the demonstrable effects on outcomes) arguably exceed any hypothetical advantages of getting feedback at the end of a session instead.

SAMPLE

A total of 41,171 (75%) adult clients with intake scores in a clinical range entered treatment between 1/1/2017 and 12/31/2019. The average number of sessions was 6.7. Of these, 25,719 (70%) completed at least two assessments, with an average number of sessions of 9.4. Of this subset with multiple assessments, 14,134 (56%) completed treatment within five sessions. A total of 19,637 (76%) completed treatment within 10 sessions, and 23,050 (90%) completed treatment within 20 sessions.

RESULTS

The following analyses are based on the subsample completing treatment within five sessions. Results for longer lengths of treatment will be addressed in subsequent articles.

ALLIANCE DESCRIPTIVE STATISTICS

The alliance scores are notable in that they do not begin to approach a normal distribution, with scores heavily skewed towards a positive rating of alliance.  The following table displays the mean score and percentage of scores that reflect the highest rating possible (perfect scores).

Table 1

Alliance Change and Global Distress Scale Change

Because of the non-normal distribution of alliance scores, the Spearman’s rank order correlation (Spearman’s Rho, Rs) was employed to explore the relationship between alliance and change on global distress. For those clients completing five sessions (n=1,875), the correlation between the average alliance score and change on global distress between the first and fifth session was quite small (Rs=.05, p<.05). The correlation between change on alliance from session two to session five and change on the global distress scale from session one to session five was slightly stronger, but hardly robust (Rs=.13, p<.01).

Given the small degree of variation in alliance scores, the variation in change in alliance scores is equally quite small. However, if change in alliance score is coded as a function of direction of change (regardless of magnitude), then alliance change can be coded into three categories: better, same, or worse.  Using this method, alliance change scores are divided into three groups at each session. Table 2 presents the frequency of the three change categories for session three through five.

Table 2

Table 3 presents results for change in alliance from the start of therapy and outcome (reported as effect size) for those clients completing five sessions.

Table 4 presents the session-to-session GDS change score (reported as effect size) as well as the change score from the first score to the score at each session, broken out by the Alliance Change Category.

An analysis of variance (ANOVA) for session-to-session change scores at each session as a function of Alliance change category revealed the differences were significant for all sessions (p<001).

A multivariate analysis of variance (General Linear Model) from session three to five revealed that the Alliance Category of Change was the best predictor of GDS change from one session to the next.

Alliance Presence or Absence and Global Distress Scale Change

Another important indicator is simply whether the client completed the alliance items at each session. Table 5 demonstrates a clear association between completion of the alliance scale and the magnitude of improvement (effect size) on the outcome measure.

Table 5

The differences in GDS change scores for session two and three as a function of Alliance present or absent (t-test) were significant (p<.001). At sessions four and five, the change in GDS scores failed to reach significance, probably due to shrinking sample sizes. However, the pattern of greater change when the Alliance score is present remains consistent.

Summary

The results support the conclusion that a measure of alliance can be used to improve treatment outcomes as measured by change on an outcome questionnaire. While the lack of a control group limits the extent that inferences of causality can be drawn, the results demonstrate the strength of the alliance-outcome correlation. Failure to complete the alliance measure at any session is associated with less change on that session.  Alliance scores are heavily skewed in a positive direction, and the magnitude of change from session to session is quite small in most cases. Spearman rank order correlations between alliance and change of the outcome measure are weak, but statistically significant. Simply looking at the direction of change in alliance (better, same or, worse) provides the clinician with a simple way of evaluating the meaning of alliance scores. Improvement in alliance scores is associated with greater change on the outcome measure. However, for alliance scores to show improvement, there must also be some variability in scores. Perfect alliance scores at every session are not associated with the best outcomes.  This suggest that clinicians need to be skillful at eliciting honest feedback on alliance, especially at the start of treatment, and to not necessarily assume that an alliance rating of perfect means all is well.

Clinician Recommendations

Therefore, clinicians should establish a culture of feedback from the very first session, orientating patients towards these measures as a core component of treatment, and building expectancy with regards to their impact on treatment outcome.  Clinicians who demonstrate professional humility may be in the best position to elicit such feedback from patients, thus, some level of prior training may need to be provided in how to communicate humility. Likewise, clinicians should be aware of those instances where the alliance items are left blank as this is associated with less change. For clinicians, it is important that they have a way to capture when alliance items are not completed so they can attend to possible ruptures nearest the point that they occur. Thus, using alliance measures as a core clinical tool in sessions is integral and clinicians should be aware that even the smallest change of direction in scores on the alliance have implications for changes on global distress measures.