A Multi-Criteria Decision Analysis Framework for Selecting Suitable Learning Management Systems

• Main Author: Hayes, Rosemary Q.
• Published: NSUWorks 2004
• Subjects: Computer Sciences
• Online Access: http://nsuworks.nova.edu/gscis_etd/576

Choosing suitable enterprise level systems is an important and complicated decision in any domain. The difficulty in selecting suitable Learning Management Systems (LMSs) is compounded by the sheer number of available products from which to choose. The decision process is further complicated by the large number of features and functions contained in these products and the heterogeneous groups of intended audiences (faculty, staff, and students) that will be using these features. The problem addressed by this researcher is the lack of a multi-criteria decision analysis framework for selecting Learning Management Systems (LMSs). Multi-criteria decision analysis (MCDA) provides decision makers with a set of theories, methodologies and techniques that can provide structure and manageability to complex decisions. In developing and testing the MCDA framework for selecting suitable LMSs, a number of steps were taken. First, a set of criteria that could be used in the evaluation of these systems was identified. A panel of experts reviewed the criteria set to ensure that it was essentially complete, and yet small enough to be manageable by an evaluation committee. In addition, the panel reviewed the criteria for scoring independence, operability, and non-redundancy. Then, the criteria set was incorporated into a step-by step model of a multi-criteria decision analysis framework for selecting suitable LMSs. In addition to the master set of criteria, the framework provided techniques for weighting criteria and aggregating scores, as well as an Excel spreadsheet tool for managing this information. A university LMS selection committee then successfully implemented this MCDA framework to evaluate three systems for possible use. As a result, the committee found that the system that was selected through this process satisfactorily represented their preferences with regards to selecting a suitable LMS for their unique environment.