The State of our Toolbox: A Meta-analysis of Reliability Measurement Precision

• Main Author: Duniewicz, Krzysztof
• Format: Others
• Published: FIU Digital Commons 2012
• Subjects: reliability; inter-item correlations; meta-analysis
• Online Access: http://digitalcommons.fiu.edu/etd/818; http://digitalcommons.fiu.edu/cgi/viewcontent.cgi?article=1929&context=etd

My study investigated internal consistency estimates of psychometric surveys as an operationalization of the state of measurement precision of constructs in industrial and organizational (I/O) psychology. Analyses were conducted of samples used in research articles published in the Journal of Applied Psychology between 1975 and 2010 in five year intervals (K = 934) from 480 articles yielding 1427 coefficients. Articles and their respective samples were coded for test-taker characteristics (e.g., age, gender, and ethnicity), research settings (e.g., lab and field studies), and actual tests (e.g., number of items and scale anchor points). A reliability and inter-item correlations depository was developed for I/O variables and construct groups. Personality measures had significantly lower inter-item correlations than other construct groups. Also, internal consistency estimates and reporting practices were evaluated over time, demonstrating an improvement in measurement precision and missing data.