Applying Q-matrix theory to determine the optimized threshold of the ordering coefficient for item ordering theory-Using fraction multiplication as an example

• Main Authors: Wu, Shih-Neng, 吳世能
• Other Authors: Liu, Hsiang-Chuan
• Format: Others
• Language: zh-TW
• Published: 2013
• Online Access: http://ndltd.ncl.edu.tw/handle/5qz8tw

博士 === 亞洲大學 === 資訊工程學系碩士班 === 101 === The item ordering structure-based CAT (computer adaptive test) has been widely used on educational fields. Speaking of the item ordering structure theory, Airasian and Bart (1973) first proposed the ordering theory (OT) based on the testing results; Takeya (1991) further considered the item relationships and proposed item relational structure theory (IRS). Hsiang-Chuan Liu (2004) proposed the item ordering structure theory (IOS) that combines the essence of the above two theories. The mentioned OT does not limit to the assumption that items are mutually independent, and it does not exclude the possibility of item-guessing or ignoring. Therefore, it is suitable for establishing student item structure as well as conduct cognitive categorization and diagnosis. However, for existing tests that current item ordering theories refer to, the items are not always effective items that exactly correlate with cognitive attributes; these ordering theories also lack the true values or validity criterion and thus cannot distinguish results by quantitative approach. Tatsuoka (1983) proposed rule space model (RSM), in which the Q-matrix theory is a relational theory between cognitive attributes and items. Leighton, Gier, and Hunka (2004) proposed the attribute hierarchy model (AHM). The AHM emphasizes that cognitive attributes are pre-determined before the test, instead of being concluded after the test; the AHM thus avoids the RSM’s drawbacks that different items will conclude different cognitive attributes and prevents the phenomenon that number of cognitive attributes is higher than the number of items. Hsiang-Chuan Liu (2012) applied the Q-matrix theory of the AHM and induced all effective items that exactly correlated with cognitive attributes. Hsiang-Chuan Liu, Hsien-Chang Tsai, and Shih-Neng Wu (2013) proposed an approach to apply Q-matrix theory to identify the optimal threshold values of item ordering theories. This approach is an effective tool to solve the problem that item ordering theories lacked validity criteria and it can employ the optimal threshold values to improve the effectiveness of these item ordering theories. This study first applied the Q-matrix theory to establish effective items that exactly correlated with cognitive attributes. This study then designed an elementary school test paper, “Multiplying Fractions”, and conducted the test. The validity criteria were effective items’ IRS that was used to evaluate the effectiveness of each kind of item ordering theories. The results are as follows: 1. The Q-matrix theory can be used to produce the correlation matrix of cognitive attributes and item categories, and the process of simplifying Q-matrix can induce effective items, whose rational structure can be used as the validity criteria for evaluating each item ordering theory. 2. Since these effective items contain all reachability matrix categories of cognitive attributes, the induced rational response types do not overlap each other. In other words, the rational response types correspond to the cognitive attributes in a one-to-one relationship, which is helpful for identifying the attributes of the participants after the test. 3. The optimal threshold values of the item ordering theories concluded by the Q-matrix theory cam significantly improve the effectiveness of the OT, IRS, and IOS. 4. For comparing the performances among the three item structure theories, if all of the thresholds of them are fixed as before, then OT outperforms the others, if all of the thresholds of them are optimized respectively, then all of them are not significantly different, therfore, for estimating the item stucture, we sugest that in unsupervised case with fixed threshold, OT is the better, in supervised case with optimized threshold, all of them are almost the same, we may select any one of them.