Development of a computerized multistage role-play testing of employment social skills in patients with schizophrenia

博士 === 國立臺灣大學 === 職能治療研究所 === 106 === Background and purposes: Employment social skills can be defined as the capability to interact appropriately with others or deal with social situations that are required at work. For patients with schizophrenia, a lack of employment social skills has been recogn...

Full description

Bibliographic Details
Main Authors: Yi-Jing Huang, 黃怡靜
Other Authors: Ching-Lin Hsieh
Format: Others
Language:zh-TW
Published: 2018
Online Access:http://ndltd.ncl.edu.tw/handle/t7wh3g
id ndltd-TW-106NTU05738010
record_format oai_dc
collection NDLTD
language zh-TW
format Others
sources NDLTD
description 博士 === 國立臺灣大學 === 職能治療研究所 === 106 === Background and purposes: Employment social skills can be defined as the capability to interact appropriately with others or deal with social situations that are required at work. For patients with schizophrenia, a lack of employment social skills has been recognized as one of the major barriers in seeking employment, adapting to a new job, and learning job-specific skills. Thus, accurate, precise, and efficient assessments of employment social skills is the basis for successful vocational rehabilitation. However, there are no content-comprehensive, low-burden, and psychometrically sound measures of employment social skills. Such a limitation has restricted clinicians in efficiently capturing the problems of patients, thereby compromising the treatment effects of social skills training and vocational rehabilitation. To ameliorate the drawbacks of the existing measures, this dissertation had 2 purposes: (1) to develop a content-comprehensive, efficient, and precise measure for assessing employment social skills in patients with schizophrenia; i.e., the COmputerized Multistage role-play Testing of Employment Social Skills (COMTESS); (2) to preliminarily examine the reliability, convergent validity, known-group validity, and efficiency of the COMTESS. Methods: This dissertation contained 2 stages. Stage 1: the development of the COMTESS was divided into 6 steps: (1) literature review; (2) development of the draft item bank; (3) expert review of the draft item bank; (4) construction of the test interface and platform; (5) Rasch analysis of the item bank; (6) selection of routing items and hard/easy modules to establish the COMTESS. Stage 2: we used the standard errors for patients’ ability estimates to calculate the individual reliabilities and average reliability of the COMTESS. We investigated the correlations between the scores of the COMTESS and the Personal and Social Performance Scale (PSP) to examine the convergent validity of the COMTESS. To examine the known-group validity, we divided the patients into groups with and without significant cognitive impairments by the Montreal Cognitive Assessment. We further used independent t test and Cohen’s d to test the significance and amount of differences in the COMTESS scores between the two groups. The efficiency of the COMTESS was estimated by the average administration time per item multiplied by the number of items. Results: In Stage 1, we developed a total of 94 items equally divided into the domains of “personal social skills” and “task-related social skills”. Each item was rated based on patients’ both non-verbal and verbal expressions by a 5-point scale (0~4 points). The draft item bank was reviewed by 28 experts. Fourteen items that were less important or inappropriate were deleted, and 80 items remained in the item bank. Thereafter, the item bank were examined in a total of 144 patients with schizophrenia using Rasch analysis. The items fitted the model well. However, the rating scale was modified as a 3-ponit scale (0~2 points) due to the reversed step difficulties. Finally, we selected 1 routing item and 4 items in each of the hard and easy modules, from each domain of item bank, to establish the COMTESS (a total of 18 items). In Stage 2, the average reliabilities of the two domains of the COMTESS both were 0.80, and 97.8% patients had reliabilities of ≥ 0.70 in both domains. These results indicate that the COMTESS had sufficient reliability. The correlation between the scores of the COMTESS and the PSP was 0.41, which indicates good convergent validity. Significant (p < 0.001) and large (d = 0.91~0.92) differences in the COMTESS scores were found between the groups with and without significant cognitive impairments, which indicate good known-group validity. The administration time of the COMTESS was about 9.9 minutes, which indicate that the COMTESS is an efficient test. Conclusion: The COMTESS may be the first computerized multistage role-play testing of employment social skills for patients with schizophrenia. The COMTESS comprehensively contains two domains of “personal social skills” and “task-related social skills”. Our results show that the COMTESS has sufficient reliability, convergent validity, and known-group validity. Moreover, the administration time of the COMTESS is only around 10 minutes. These findings imply that the COMTESS is able to comprehensively assess employment social skills in a timely and precise fashion and to discriminate the levels of employment social skills among patients. Besides, computerized testing can largely reduce the training time and burden of the administrators and increase the feasibility.
author2 Ching-Lin Hsieh
author_facet Ching-Lin Hsieh
Yi-Jing Huang
黃怡靜
author Yi-Jing Huang
黃怡靜
spellingShingle Yi-Jing Huang
黃怡靜
Development of a computerized multistage role-play testing of employment social skills in patients with schizophrenia
author_sort Yi-Jing Huang
title Development of a computerized multistage role-play testing of employment social skills in patients with schizophrenia
title_short Development of a computerized multistage role-play testing of employment social skills in patients with schizophrenia
title_full Development of a computerized multistage role-play testing of employment social skills in patients with schizophrenia
title_fullStr Development of a computerized multistage role-play testing of employment social skills in patients with schizophrenia
title_full_unstemmed Development of a computerized multistage role-play testing of employment social skills in patients with schizophrenia
title_sort development of a computerized multistage role-play testing of employment social skills in patients with schizophrenia
publishDate 2018
url http://ndltd.ncl.edu.tw/handle/t7wh3g
work_keys_str_mv AT yijinghuang developmentofacomputerizedmultistageroleplaytestingofemploymentsocialskillsinpatientswithschizophrenia
AT huángyíjìng developmentofacomputerizedmultistageroleplaytestingofemploymentsocialskillsinpatientswithschizophrenia
AT yijinghuang fāzhǎnsījuéshīdiàozhènghuànzhězhīgōngzuòshèjiāojìqiǎodiànnǎohuàduōjiēduànjiǎosèbànyǎncèyàn
AT huángyíjìng fāzhǎnsījuéshīdiàozhènghuànzhězhīgōngzuòshèjiāojìqiǎodiànnǎohuàduōjiēduànjiǎosèbànyǎncèyàn
_version_ 1719174065427578880
spelling ndltd-TW-106NTU057380102019-05-16T01:07:39Z http://ndltd.ncl.edu.tw/handle/t7wh3g Development of a computerized multistage role-play testing of employment social skills in patients with schizophrenia 發展思覺失調症患者之工作社交技巧電腦化多階段角色扮演測驗 Yi-Jing Huang 黃怡靜 博士 國立臺灣大學 職能治療研究所 106 Background and purposes: Employment social skills can be defined as the capability to interact appropriately with others or deal with social situations that are required at work. For patients with schizophrenia, a lack of employment social skills has been recognized as one of the major barriers in seeking employment, adapting to a new job, and learning job-specific skills. Thus, accurate, precise, and efficient assessments of employment social skills is the basis for successful vocational rehabilitation. However, there are no content-comprehensive, low-burden, and psychometrically sound measures of employment social skills. Such a limitation has restricted clinicians in efficiently capturing the problems of patients, thereby compromising the treatment effects of social skills training and vocational rehabilitation. To ameliorate the drawbacks of the existing measures, this dissertation had 2 purposes: (1) to develop a content-comprehensive, efficient, and precise measure for assessing employment social skills in patients with schizophrenia; i.e., the COmputerized Multistage role-play Testing of Employment Social Skills (COMTESS); (2) to preliminarily examine the reliability, convergent validity, known-group validity, and efficiency of the COMTESS. Methods: This dissertation contained 2 stages. Stage 1: the development of the COMTESS was divided into 6 steps: (1) literature review; (2) development of the draft item bank; (3) expert review of the draft item bank; (4) construction of the test interface and platform; (5) Rasch analysis of the item bank; (6) selection of routing items and hard/easy modules to establish the COMTESS. Stage 2: we used the standard errors for patients’ ability estimates to calculate the individual reliabilities and average reliability of the COMTESS. We investigated the correlations between the scores of the COMTESS and the Personal and Social Performance Scale (PSP) to examine the convergent validity of the COMTESS. To examine the known-group validity, we divided the patients into groups with and without significant cognitive impairments by the Montreal Cognitive Assessment. We further used independent t test and Cohen’s d to test the significance and amount of differences in the COMTESS scores between the two groups. The efficiency of the COMTESS was estimated by the average administration time per item multiplied by the number of items. Results: In Stage 1, we developed a total of 94 items equally divided into the domains of “personal social skills” and “task-related social skills”. Each item was rated based on patients’ both non-verbal and verbal expressions by a 5-point scale (0~4 points). The draft item bank was reviewed by 28 experts. Fourteen items that were less important or inappropriate were deleted, and 80 items remained in the item bank. Thereafter, the item bank were examined in a total of 144 patients with schizophrenia using Rasch analysis. The items fitted the model well. However, the rating scale was modified as a 3-ponit scale (0~2 points) due to the reversed step difficulties. Finally, we selected 1 routing item and 4 items in each of the hard and easy modules, from each domain of item bank, to establish the COMTESS (a total of 18 items). In Stage 2, the average reliabilities of the two domains of the COMTESS both were 0.80, and 97.8% patients had reliabilities of ≥ 0.70 in both domains. These results indicate that the COMTESS had sufficient reliability. The correlation between the scores of the COMTESS and the PSP was 0.41, which indicates good convergent validity. Significant (p < 0.001) and large (d = 0.91~0.92) differences in the COMTESS scores were found between the groups with and without significant cognitive impairments, which indicate good known-group validity. The administration time of the COMTESS was about 9.9 minutes, which indicate that the COMTESS is an efficient test. Conclusion: The COMTESS may be the first computerized multistage role-play testing of employment social skills for patients with schizophrenia. The COMTESS comprehensively contains two domains of “personal social skills” and “task-related social skills”. Our results show that the COMTESS has sufficient reliability, convergent validity, and known-group validity. Moreover, the administration time of the COMTESS is only around 10 minutes. These findings imply that the COMTESS is able to comprehensively assess employment social skills in a timely and precise fashion and to discriminate the levels of employment social skills among patients. Besides, computerized testing can largely reduce the training time and burden of the administrators and increase the feasibility. Ching-Lin Hsieh 謝清麟 2018 學位論文 ; thesis 123 zh-TW