Development of a computerized multistage role-play testing of employment social skills in patients with schizophrenia

• Main Authors: Yi-Jing Huang, 黃怡靜
• Other Authors: Ching-Lin Hsieh
• Format: Others
• Language: zh-TW
• Published: 2018
• Online Access: http://ndltd.ncl.edu.tw/handle/t7wh3g

博士 === 國立臺灣大學 === 職能治療研究所 === 106 === Background and purposes: Employment social skills can be defined as the capability to interact appropriately with others or deal with social situations that are required at work. For patients with schizophrenia, a lack of employment social skills has been recognized as one of the major barriers in seeking employment, adapting to a new job, and learning job-specific skills. Thus, accurate, precise, and efficient assessments of employment social skills is the basis for successful vocational rehabilitation. However, there are no content-comprehensive, low-burden, and psychometrically sound measures of employment social skills. Such a limitation has restricted clinicians in efficiently capturing the problems of patients, thereby compromising the treatment effects of social skills training and vocational rehabilitation. To ameliorate the drawbacks of the existing measures, this dissertation had 2 purposes: (1) to develop a content-comprehensive, efficient, and precise measure for assessing employment social skills in patients with schizophrenia; i.e., the COmputerized Multistage role-play Testing of Employment Social Skills (COMTESS); (2) to preliminarily examine the reliability, convergent validity, known-group validity, and efficiency of the COMTESS. Methods: This dissertation contained 2 stages. Stage 1: the development of the COMTESS was divided into 6 steps: (1) literature review; (2) development of the draft item bank; (3) expert review of the draft item bank; (4) construction of the test interface and platform; (5) Rasch analysis of the item bank; (6) selection of routing items and hard/easy modules to establish the COMTESS. Stage 2: we used the standard errors for patients’ ability estimates to calculate the individual reliabilities and average reliability of the COMTESS. We investigated the correlations between the scores of the COMTESS and the Personal and Social Performance Scale (PSP) to examine the convergent validity of the COMTESS. To examine the known-group validity, we divided the patients into groups with and without significant cognitive impairments by the Montreal Cognitive Assessment. We further used independent t test and Cohen’s d to test the significance and amount of differences in the COMTESS scores between the two groups. The efficiency of the COMTESS was estimated by the average administration time per item multiplied by the number of items. Results: In Stage 1, we developed a total of 94 items equally divided into the domains of “personal social skills” and “task-related social skills”. Each item was rated based on patients’ both non-verbal and verbal expressions by a 5-point scale (0~4 points). The draft item bank was reviewed by 28 experts. Fourteen items that were less important or inappropriate were deleted, and 80 items remained in the item bank. Thereafter, the item bank were examined in a total of 144 patients with schizophrenia using Rasch analysis. The items fitted the model well. However, the rating scale was modified as a 3-ponit scale (0~2 points) due to the reversed step difficulties. Finally, we selected 1 routing item and 4 items in each of the hard and easy modules, from each domain of item bank, to establish the COMTESS (a total of 18 items). In Stage 2, the average reliabilities of the two domains of the COMTESS both were 0.80, and 97.8% patients had reliabilities of ≥ 0.70 in both domains. These results indicate that the COMTESS had sufficient reliability. The correlation between the scores of the COMTESS and the PSP was 0.41, which indicates good convergent validity. Significant (p < 0.001) and large (d = 0.91~0.92) differences in the COMTESS scores were found between the groups with and without significant cognitive impairments, which indicate good known-group validity. The administration time of the COMTESS was about 9.9 minutes, which indicate that the COMTESS is an efficient test. Conclusion: The COMTESS may be the first computerized multistage role-play testing of employment social skills for patients with schizophrenia. The COMTESS comprehensively contains two domains of “personal social skills” and “task-related social skills”. Our results show that the COMTESS has sufficient reliability, convergent validity, and known-group validity. Moreover, the administration time of the COMTESS is only around 10 minutes. These findings imply that the COMTESS is able to comprehensively assess employment social skills in a timely and precise fashion and to discriminate the levels of employment social skills among patients. Besides, computerized testing can largely reduce the training time and burden of the administrators and increase the feasibility.