Statistical conclusion validity: Some common threats and simple remedies

The ultimate goal of research is to produce dependable knowledge or to provide the evidence that may guide practical decisions. Statistical conclusion validity (SCV) holds when the conclusions of a research study are founded on an adequate analysis of the data, generally meaning that adequate statis...

Full description

Bibliographic Details
Main Author: Miguel A García-Pérez
Format: Article
Language:English
Published: Frontiers Media S.A. 2012-08-01
Series:Frontiers in Psychology
Subjects:
Online Access:http://journal.frontiersin.org/Journal/10.3389/fpsyg.2012.00325/full
id doaj-b5cb61ab398e4cb190e58e2190059509
record_format Article
spelling doaj-b5cb61ab398e4cb190e58e21900595092020-11-24T22:54:17ZengFrontiers Media S.A.Frontiers in Psychology1664-10782012-08-01310.3389/fpsyg.2012.0032529584Statistical conclusion validity: Some common threats and simple remediesMiguel A García-Pérez0Universidad ComplutenseThe ultimate goal of research is to produce dependable knowledge or to provide the evidence that may guide practical decisions. Statistical conclusion validity (SCV) holds when the conclusions of a research study are founded on an adequate analysis of the data, generally meaning that adequate statistical methods are used whose small-sample behavior is accurate, besides being logically capable of providing an answer to the research question. Compared to the three other traditional aspects of research validity (external validity, internal validity, and construct validity), interest in SCV has recently grown on evidence that inadequate data analyses are sometimes carried out which yield conclusions that a proper analysis of the data would not have supported. This paper discusses evidence of three common threats to SCV that arise from widespread recommendations or practices in data analysis, namely, the use of repeated testing and optional stopping without control of Type-I error rates, the recommendation to check the assumptions of statistical tests, and the use of regression whenever a bivariate relation or the equivalence between two variables is studied. For each of these threats, examples are presented and alternative practices that safeguard SCV are discussed. Educational and editorial changes that may improve the SCV of published research are also discussed.http://journal.frontiersin.org/Journal/10.3389/fpsyg.2012.00325/fullregressiondata analysisvalidity of researchstopping rulespreliminary tests
collection DOAJ
language English
format Article
sources DOAJ
author Miguel A García-Pérez
spellingShingle Miguel A García-Pérez
Statistical conclusion validity: Some common threats and simple remedies
Frontiers in Psychology
regression
data analysis
validity of research
stopping rules
preliminary tests
author_facet Miguel A García-Pérez
author_sort Miguel A García-Pérez
title Statistical conclusion validity: Some common threats and simple remedies
title_short Statistical conclusion validity: Some common threats and simple remedies
title_full Statistical conclusion validity: Some common threats and simple remedies
title_fullStr Statistical conclusion validity: Some common threats and simple remedies
title_full_unstemmed Statistical conclusion validity: Some common threats and simple remedies
title_sort statistical conclusion validity: some common threats and simple remedies
publisher Frontiers Media S.A.
series Frontiers in Psychology
issn 1664-1078
publishDate 2012-08-01
description The ultimate goal of research is to produce dependable knowledge or to provide the evidence that may guide practical decisions. Statistical conclusion validity (SCV) holds when the conclusions of a research study are founded on an adequate analysis of the data, generally meaning that adequate statistical methods are used whose small-sample behavior is accurate, besides being logically capable of providing an answer to the research question. Compared to the three other traditional aspects of research validity (external validity, internal validity, and construct validity), interest in SCV has recently grown on evidence that inadequate data analyses are sometimes carried out which yield conclusions that a proper analysis of the data would not have supported. This paper discusses evidence of three common threats to SCV that arise from widespread recommendations or practices in data analysis, namely, the use of repeated testing and optional stopping without control of Type-I error rates, the recommendation to check the assumptions of statistical tests, and the use of regression whenever a bivariate relation or the equivalence between two variables is studied. For each of these threats, examples are presented and alternative practices that safeguard SCV are discussed. Educational and editorial changes that may improve the SCV of published research are also discussed.
topic regression
data analysis
validity of research
stopping rules
preliminary tests
url http://journal.frontiersin.org/Journal/10.3389/fpsyg.2012.00325/full
work_keys_str_mv AT miguelagarciaperez statisticalconclusionvaliditysomecommonthreatsandsimpleremedies
_version_ 1725660955083276288