| Summary: | When conducting scientific research, obtaining high-quality data is important. When collecting data from a college student participant pool, however, factors such as the time of the semester in which data are collected could cause validity issues, especially if the survey is completed in an online, non-laboratory setting. Near the end of the semester, students may experience more time pressures and constraints than at other times in the semester. These additional pressures may encourage participants to multi-task while completing the study, or to rush through the survey in order to receive credits as quickly as possible. The hypothesis of this study was that responses collected at the end of the semester would exhibit lower data quality than responses collected at the beginning of the semester. Data were collected online during the last two weeks of the fall 2018 semester (n = 312) and the first two weeks of the spring 2019 semester (n = 55). Participants were asked to write about an embarrassing situation and then completed a number of questionnaires assessing their thoughts and feelings about the event, personality traits, and participant engagement. Data quality was assessed using several different previously validated methods, including time spent on survey; the number of missed items; the number of incorrect embedded attention-check items (out of 12); the length of responses on two open-ended questions; self-reported diligence, interest, effort, attention, and whether their data should be used; and Cronbach’s alphas on the scales. Results showed that between the two groups, there were significant differences on length of open-ended responses, self-reported diligence, self-reported interest, effort, attention, neuroticism, and conscientiousness. Participants completing the study in the first two weeks of the spring 2019 semester had significantly longer open-ended responses and significantly higher levels of self-reported diligence, self-reported interest, effort, attention, neuroticism, and conscientiousness. Although there was not a significant difference in number of incorrect attention-check items between the two groups, it should be noted that only 46% of the total participants did not miss any check items. These results lend support to the hypothesis that data collected at the end of the semester may be of lower quality than data collected at the beginning of the semester. However, because the groups significantly differed on neuroticism and conscientiousness, we cannot determine whether the time of semester effect is a product of internal participant characteristics or external pressures. Nevertheless, researchers should take into account this end-of-semester data quality difference when deciding the time-frame of their data collection.
|
|---|
