Does clinical supervision lead to better patient outcomes in mental health nursing?

26 April, 2010 Clinical supervision is claimed to be beneficial, but does the evidence support this assertion? This trial examined the impact on supervisors, supervisees and patients

Authors
Edward White, PhD, MSc (SocPol), MSc (SocRes), PGCEA, RMN, DipCPN, RNT, FRCNA, FACMHN, FCN, is conjoint professor, School of Psychiatry, University of New South Wales; Julie Winstanley, PhD, MSc, BSc, CStat, CSc, is associate professor in biostatistics, Melanoma Institute Australia, University of Sydney; both are also directors, Osman Consulting Pty Ltd, Sydney, Australia.

Abstract
White E, Winstanley J (2010) Does clinical supervision lead to better patient outcomes in mental health nursing? Nursing Times; 106: 16, early online publication. Background It is claimed that clinical supervision contributes to clinical governance of healthcare services, but many claims for its positive effects are unsubstantiated. Aim To test relationships between supervision, quality of nursing care and patient outcomes. Method A randomised controlled trial was conducted across Queensland, Australia, supplemented by qualitative data collection. Results Supervision had sustainable beneficial effects for supervisors and supervisees. The individual performance of clinical supervisors was affected by the culture of the organisation. A positive relationship between supervision, quality of care and patient outcomes could not be established statistically, except in one location. Discussion and conclusion Proposals are made for future supervision implementation and further research is required. Keywords Clinical supervision, Mental health, Patient outcomes

This article has been double blind peer reviewed

Research and development of clinical supervision: main points

Select a single, discrete clinical location. Agree an explicit, unified, positive position on clinical supervision owned by all levels of management. Carefully identify and prepare supervisors.

Recruit all staff to participate in supervision, according to standard protocols (size, frequency, length, ground rules and so on). Allocate no more than nine supervisees to one supervisor. Retain almost all (>90%) of supervisees over the period of data collection (>1 year). Deliver sustained, effective supervision (indicated by a median supervisee total Manchester Clinical Supervision Scale score of >136). Support supervisors through regular supervision sessions. Measure, theorise, modify and retest.

Background
While interest increases in establishing a causal relationship between nursing clinical supervision and better patient outcomes, claims about the benefits remain unproven. Supervision provides time out and a chance to engage in guided reflection on practice. This is designed to develop and enhance future practice, in the context of an ongoing professional relationship between a supervisee and an experienced practitioner. International interest in nursing supervision has grown over the last 20 years; it has underpinned researchers interest in establishing the evidence base for causal relationships between supervision, quality of care and patient outcomes (White et al, 1998). Mental health nursing workforces around the world, especially in Scandinavia and the UK, were early adopters of clinical supervision but, with limited exceptions (Bambling et al, 2006), its efficacy has rarely been examined. Although unsubstatiated in the literature, benefits have been attributed to:

Nurses (increased levels of self esteem, morale, job satisfaction, enhanced skills and knowledge, greater personal and professional development); Patients (improved standards of care, more effective interdisciplinary communication); The organisation (reduced absenteeism and staff turnover rates, kudos from exciting and innovative practices).

Aim
The research study summarised here (White and Winstanley, 2010) aimed to inform the evidence base for such claims by testing relationships between clinical supervision, quality of nursing care and patient outcomes.

Literature review
Although several models of delivering supervision have been reported (Winstanley and White, 2003), practically this usually means small groups of some six people attending a prearranged monthly meeting with a trained supervisor for 45-60 minutes.

Interest in supervision increased after the Clothier Report (Department of Health, 1994), which raised concerns about standards of supervision and training for nurses. Supervision was quickly incorporated into the clinical governance of healthcare services in the UK and is increasingly found internationally (Butterworth et al, 2008). A review of the literature on reasons for leaving mental health nursing (Edwards and Burnard, 2003) reported:

Low overall job satisfaction; Dissatisfaction with perceived quality of decision making by managers; Dissatisfaction with lack of in-service training; Dissatisfaction with physical working conditions and burnout; Lack of staff support; Lack of involvement with the organisation; Poor job satisfaction.

Moreover, high levels of emotional exhaustion were associated with direct patient care, the work environment and lack of support. A fit can be found between the problems that mental health nurses report and the claimed benefits of supervision. Before 2000, few large scale evaluations of supervision had been undertaken. Probably the best known is the Clinical Supervision Evaluation Project (CSEP) (Butterworth et al, 1997) which informed evaluations of assessment tools used to measure the impact of supervision. Although the study found that supervisees welcomed the opportunity to talk to a trusted colleague (White et al, 1998), it was not set up to examine the effect of supervision on the quality of service provision and patient outcomes. CSEP provided the test bed for the development of the first internationally validated, clinical supervision specific research instrument. The Manchester Clinical Supervision Scale (Winstanley, 2000), available at www.osmanconsulting.com, has been used as an outcome measure of the efficacy of supervision in more than 80 evaluation studies in 12 countries and is available in five languages. Its supervision scores range from 36 (worst) to 180 (best). The median total score to emerge from these international evaluations is 136 and is the threshold above which supervision is thought to become effective.

Method
Sample
This trial took place in 17 adult mental health facilities, in nine locations, in public and private inpatient and community settings across Queensland, Australia. Units were randomised to the intervention or control arm of the study. Those in the intervention arm implemented clinical supervision while controls did not. Ethical approval was obtained. Data was collected from July 2007 until January 2009.

Supervision training course

Across the clinical areas in the intervention group, 24 nurses were trained as supervisors. Seventeen were female; median age was 46 and they had been qualified for an average of 21

years. They attended an intensive four day course, led by the research team in July 2007 (White and Winstanley, 2009). The course comprised practical sessions with direct feedback, each of which followed a programme of theory based seminars. It was well reviewed by trainee supervisors. A year long intervention phase followed the end of the course, during which trainee supervisors set up and delivered group supervision sessions in their respective mental health service locations. Each supervisor received their own supervision from one of three area coordinators, who visited participating facilities at least once a month for 12 months. At baseline, participants in the intervention arm (n = 9 sites) comprised trainee clinical supervisors (n = 24); supervisees (n = 115); patients (n = 82) and non-nursing unit clinical staff (n = 43). In the control arm (n = 6 sites), they comprised mental health nurses (n = 71); patients (n = 88); and non-nursing unit clinical staff (n = 11).

Data collection and analysis

Qualitative data collection methods were also used to illuminate the quantitative data. Semi structured interviews were conducted with a purposive sample of senior nursing managers and other clinical staff (n = 17) in each participating location, and diary accounts provided on a monthly basis by each trainee clinical supervisor, for 12 months after completing their course. Quantitative data was collected at three levels (nurse, patient and unit) and used a range of outcome measures. All had well established psychometric properties. These included: the General Health Questionnaire (Goldberg and Williams, 1988); the Manchester Clinical Supervision Scale (Winstanley, 2000); Maslach Burnout Inventory (Maslach and Jackson, 1986); SF-8 Health Survey (Ware et al, 2001); Service Attachment Questionnaire (Goodwin et al, 2003); Psychiatric Care Satisfaction Questionnaire (Barker and Orrell, 1999) and the Perception of Unit Quality (Cronenwett, 1997). Data were analysed using SPSS version 16.

Selected results
No statistically significant differences were found in the demographic data (age, sex, grades) between mental health nurses in the intervention and control arms (n = 186). As anticipated, no statistically significant differences were found in the control arm by any of the research instruments over time during the 12 months of data collection for nurses. Findings from the qualitative data at the studys outset found trainee supervisors were most daunted by the anticipated lack of support from their immediate managers and peers in their clinical areas. In the event, this belief was often confirmed by their experience. Senior managers reported they were optimistic and enthusiastic about supervision, but were also disappointed and embarrassed when junior managerial colleagues and other clinical nursing staff did not have a similar conviction when attempts were made to implement it. The personal disposition of individual middle managers emerged as the main factor which substantially influenced - if not determined - the outcome of supervision. This ranged from

enthusiastic to unsupportive to hostile and resistant. Control and management of the staffing roster was found to be the mechanism by which supervision was both facilitated and stymied. It also conveyed how it was conceptualised at local level, as either integral to local nursing practice arrangements, or as extra-curricular.

Supervisors
In the intervention arm, quantitative findings revealed that trainee supervisor total MCSS scores at the end of the course were significantly higher compared with their perception of supervision at baseline. This difference was maintained after 12 months experience (Fig 1). Two subscales revealed particularly significant differences over time with regard to trust and rapport and importance/value. In general terms, trainee supervisors scores revealed an association between high MCSS scores and low MBI burnout; that is, the better the supervision, the less burnt out they felt. They also revealed an overall reduction in the level of GHQ psychiatric caseness, a quantitative assessment of the likelihood that an individual would be identified as a psychiatric case by a psychiatrist, over time.

Supervisees
At baseline, supervisees in the intervention arm revealed normal levels of physical and mental health, as measured by the SF-8. About a quarter recorded high emotional exhaustion levels on the MBI; nearly 40% recorded caseness on the GHQ. In six of nine intervention locations, supervisees reported their supervision experience as having met or exceeded their expectations, between baseline and 12 months. Overall, their total MCSS scores did not change significantly, in a systematic fashion, over the 12 months of receiving supervision. However, two MCSS subscales associated with Proctors (1986) normative and restorative domains did increase significantly over time. This finding suggested the following proposition: Significant changes in the formative domain (concerned with developing skills and knowledge and, therefore, most relevant to impact on patient outcomes) may only become demonstrable after significant changes to the restorative (concerned with personal wellbeing) and normative (concerned with promoting standards and clinical audit issues) domains have become established, caused by sustained and efficacious supervision. Significantly more supervisees who scored less than the MCSS median value (136) were found to have moved into GHQ caseness over time. There was no significant change in the level of caseness for those who scored more than 136.

Patients
Statistically significant differences in care quality or patient satisfaction could not be shown overall during this trial. However, in one location (a private sector mental health service), all outcome measures moved significantly in a positive direction and in an intuitively convincing manner. Both the Psychiatric Care Satisfaction Questionnaire and the Perception of Unit Quality instruments revealed statistically significant improvements over 12 months. Many factors were found to be present in this setting, to which the best advice for clinical practice development is reported in this article.

Discussion
Many senior officers who managed mental health nursing services involved in this trial expressed a preference for their clinical settings to be allocated to the intervention rather than the control arm. Indeed, such was their optimism about supervision that some were not willing to participate unless this was so. In addition to these preferences, the attrition of respondents subsequently varied by location and, in some settings, was significant. These features challenged the trials methodological requirements, not only in terms of ensuring a balance between intervention and control arm numbers, but also in limiting the scope of appropriate analyses that could be carried out on the data. The absence of evidence, however, is not evidence of absence. Randomised controlled trials that do not show a significant difference are often called negative. This term wrongly implies that a study has shown there is no difference, whereas usually all that is shown is an absence of evidence of a difference (Altman and Bland, 1995). Moreover, a literature review on clinical supervision (Butterworth et al, 2008) lamented the tired discussions in the literature that offered no new insights, but was encouraged that new ideas related to patient outcome and professional development are emerging and cited this trial as an example. Findings reported here support the premise that the training and subsequent participation had beneficial effects for supervisors. The individual performance of supervisors, however, as judged by the reported experience of supervisees (and vice versa), may have been mediated by the organisational culture in which they were expected to be at the vanguard of a nursing practice innovation. At baseline, patients (n=159) median score on the Psychiatric Care Satisfaction Questionnaire suggested they were reasonably satisfied with the level of care and service provided. Across intervention and control arm settings and over time, from a descriptive point of view, only small changes were seen in either direction. Similarly, no systematic differences could be found in the Service Attachment Questionnaire scores for both groups across three time points. Therefore, a positive relationship between supervision, care quality and patient outcomes could not be established, except in one location.

Conclusion
The insights reported here may help in conceptualising and setting up clinical supervision research, education, practice, management and policy decision making in the future.

This is an edited version of the research published in the Journal of Research in Nursing (White and Winstanley, 2010)

Acknowledgement
The authors are grateful to the Queensland Treasury/Golden Casket Foundation, Brisbane, Australia, for the substantial grant which afforded this study. Special thanks are due to Natalie Stuart, project research officer.

The authors also thank Sue OSullivan (CS educator), and Kerrie Counihan, Jo Munday and Christine Palmer (area coordinators), each of whom were experienced clinical supervisors and course facilitators. The authors confirm the centrality of the 24 trainee clinical supervisors, whose identities have been kept intentionally confidential, together with every data-providing physician, psychiatrist, psychologist, social worker, clinical mental health nurse and senior nursing manager. Finally, the overarching motivation to conduct this study was to make an additional contribution to clinical nursing practice development and to better understanding ways in which consumers of mental health services could be helped to recover. To each and every one of them, and their carers, the authors acknowledge with gratitude their willingness to share the same ambition and to have actively participated in this novel attempt to do so. As hugely important and necessary as these respective contributions were to this study, however, the responsibility for any errors and omissions and all matters of interpretation in this article rest solely with the authors.

References:
Altman D, Bland M (1995) Absence of evidence is not evidence of absence. British Medical Journal; 311: 485. Bambling M et al (2006) Clinical Supervision: Its influence on client-rated working alliance and client symptom reduction in the brief treatment of major depression. Psychotherapy Research; 16: 3, 317-331. Barker D, Orrell M (1999) The Psychiatric Care Satisfaction Questionnaire: a reliability and validity study. Social Psychiatry and Psychiatric Epidemiology; 34: 111-116. Butterworth T et al (2008) Wicked spell or magic bullet? A review of the clinical supervision literature 2001-2007. Nurse Education Today; 28: 3, 264-272. Butterworth T et al (1997) Clinical Supervision and Mentorship. It is Good to Talk: An Evaluation study in England and Scotland. Manchester: University of Manchester. Cronenwett L (1997) A multidisciplinary measure of Perceptions of Unit Quality. Unpublished PhD thesis. University of North Carolina, USA. Department of Health (1994) Independent Inquiry Relating to Deaths and Injuries on the Childrens Ward at Grantham and Kesteven General Hospital. Clothier Report. London: HMSO. Edwards D, Burnard P (2003) A systematic review of stress and stress management interventions for mental health nurses. Journal of Advanced Nursing; 42: 169-200. Goldberg D, Williams P (1988) A Users Guide to the General Health Questionnaire. Windsor: NFER Nelson. Goodwin I et al (2003) The ability of adult mental health services to meet clients attachment needs: The development and implementation of the Service Attachment Questionnaire. Psychology and Psychotherapy: Theory Research and Practice; 76: 145-161. Maslach C, Jackson S (1986) The Maslach Burnout Inventory. Palo Alto, CA: Consulting Psychologists Press. Proctor B (1986) Supervision: a cooperative exercise in accountability. In: Marken M, Payne M (eds) Enabling and Ensuring: Supervision in Practice. Leicester: National Youth Bureau

and Council for Education and Training in Youth and Community Work. Ware J et al (2001) A Manual for Users of the SF-8 Health Survey. Lincoln, RI/Boston, MA: Quality Metric Inc/Health Assessment Laboratory. White E et al (1998) Clinical supervision: insider reports of a private world. Journal of Advanced Nursing; 28: 1, 185-192. White E, Winstanley J (2010) A randomised controlled trial of clinical supervision: selected findings from a novel Australian attempt to establish the evidence base for causal relationships with quality of care and patient outcomes, as an informed contribution to mental health nursing practice development. Journal of Research in Nursing; 15: 2, 151-166. White E, Winstanley J (2009) Implementation of clinical supervision: educational preparation and subsequent diary accounts of the practicalities involved, from an Australian mental heath nursing innovation. Journal of Psychiatric and Mental Health Nursing; 14: 3, 263-276. Winstanley J (2000) Manchester Clinical Supervision Scale. Nursing Standard; 14: 19, 3132. Winstanley J, White E (2003) Clinical supervision: Models, measures and best practice. Nurse Researcher; 10: 4, 7-38.