The development, reliability and validity testing of an observation tool designed to measure recovery orientation in inpatient recovery and rehabilitation services

AIM: Recovery orientation is important to inpatient mental health services. However, it has proved difficult to measure and hence to implement. Using an observational method, this project aimed to design a tool for assessing the recovery orientation of a service, and to test its reliability and vali...

Full description

Bibliographic Details
Main Author: Williams, Christian
Published: Cardiff University 2016
Subjects:
Online Access:http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.709569
Description
Summary:AIM: Recovery orientation is important to inpatient mental health services. However, it has proved difficult to measure and hence to implement. Using an observational method, this project aimed to design a tool for assessing the recovery orientation of a service, and to test its reliability and validity. METHOD: From the recovery literature over-arching themes in the recovery process were identified. From these, descriptions of types of interactions between patients and staff members, termed ‘codes’, were developed. Using a Delphi method, experts working in inpatient Rehabilitation services agreed which codes should be included in the tool. Staff teams in seven wards completed a standardised measure of Recovery (Recovery-Self Assessment [RSA]). The wards with the highest and lowest mean scores were used to pilot the new tool. Time sampling was used to capture interactions between staff and patients on both wards. These were coded using the new tool. Interactions coded as supporting recovery were rated positive; those coded as hindering recovery were rated negative. It was hypothesised that more positively coded interactions would be observed on the ward with the highest RSA score. Reliability was assessed by comparing percentage agreement between two researchers independently coding 20% of the observed interactions. Validity was assessed by comparing the number of positive and negative interactions on the two wards. FINDINGS: 23 of the 30 codes developed were included in the final tool. The reliability of the tool was good with 93.8% agreement on positive/negative ratings (Kappa 0.827). However, the validity was poor, with Chi-square showing no significant difference between wards in the number of positive and negative interactions. CONCLUSION: The evidence base for the tool is sound. Its reliability is encouraging. The poor validity may be because the two test wards did not differ greatly on the RSA. Further development of both the codes used to construct the tool and the method in which it is used could provide a clinically useful tool for Services.