Peer Assessment and Self-Assessment in Social Learning Environments Through a New Crowd-Sourced Mechanism

• Main Authors: Fatemeh Orooji, Fattaneh Taghiyareh
• Format: Article
• Language: English
• Published: IEEE 2018-01-01
• Series: IEEE Access
• Subjects: Online peer-assessment; crowd-sourced assessment; voting; social learning; online self-assessment
• Online Access: https://ieeexplore.ieee.org/document/8274966/

Social learning environments generally provide learners with the grounds to collaboratively create and share different learning contents. The variety and considerably large amount of created contents makes them infeasible for students to read through and often results in a continuous reduction in students' contribution. Therefore, social learning environments should be equipped with effective mechanisms to evaluate and accredit learner-created content relying on students' participation. In order to suggest a voluntary mechanism for peer assessment with the least overhead, the current study proposed a new crowd sourced approach. The approach called content-dependent multi-label voting (COMVO) offers various assessment options for each type of learning content consisting of resource, assignment, forum, discussion, reply, and comment. COMVO was implemented in a social learning environment and was utilized by students and experts during educational activities in a university course. Peer voting, self-voting, voting to experts, and expert voting were qualitatively analyzed. The results indicated that in contrast to peer voting, which mostly consists of positively describing labels, self-voting labels match those given by experts. Analysis implied that peer voting is reliable and expert-independent. This paper also provided insights about student behaviors and reciprocal effects in identified voting, investigating the role of students' extrinsic and intrinsic motivational orientation in their voting behavior. Results of a subjective evaluation indicated that the majority of respondents found COMVO an enthusiastic and efficient tool with the potential to complete other similar crowd sourced peer assessment mechanisms.