Mental models : have users' mental models of search engines improved in the last ten years?

• Main Author: Mlilo, Sifiso
• Format: Others
• Language: en
• Published: 2011
• Online Access: http://hdl.handle.net/10539/9948

Mental models can be defined as mental representations or cognitive mappings of external systems that allow for effective descriptions, explanations and predictions of those systems (Staggers & Norcio, 1993). The quality of users’ mental models of search engines has been investigated in a number of studies across a range of disciplines. The results have been uniform in concluding that users’ mental models of search engines are inaccurate and incomplete (Jansen & Spink, 2006; Muramatsu & Pratt, 2001; Thatcher & Greyling, 2003). This conclusion has corroborated studies proposing that users show poor search performance (Muramatsu & Pratt, 2001). This study investigated the accuracy and completeness of mental models users have of search engines in the context of a comparison of matched data obtained from samples from 2000 and 2010. The same methodology employed in the study conducted on the 2000 sample was used. Eighty volunteer participants were asked to complete a maximum of two computer-based tasks and onscreen capture software was used to record their search actions (time, steps, etc.). The recordings were then played back to the participants and a retrospective protocol interview was conducted to determine the reasoning behind their actions. They were also asked to provide an illustration of how they thought a search engine worked. A short satisfaction and confidence questionnaire was also administered. Statistical significance was found for the performance measures of time and steps. The differences were quite large with the 2010 sample using less than half the time used by the 2000 sample and almost half the number of steps. The performance was still significant after controlling for experience. The chi-squared test conducted for the performance measure accuracy also revealed statistical significance with the 2010 sample showing significantly greater accuracy. The chi-squared comparison of the salient features indicated that seven of the seventeen features were more frequent at Time 2. Seven of the salient features were more frequent at Time 1. Where the cluster analysis of the 2000 sample revealed four distinct clusters, three clusters emerged from the 2010 sample. It was clear from the analyses conducted that, even if not to a large degree, the accuracy and completeness of users’ mental models of search engines had demonstrably improved in the last decade. So while users’ mental models of search engines remain inaccurate and incomplete, their alignment with designer’s conceptualisations has improved.