Using Student Annotated Hashtags and Emojis to Collect Nuanced Affective States

• Main Authors: Zhang, Amy Xian (Author), Igo, Michele (Author), Facciotti, Marc (Author), Karger, David R (Author)
• Other Authors: Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory (Contributor)
• Format: Article
• Language: English
• Published: Association for Computing Machinery (ACM), 2021-01-25T18:05:41Z.
• Subjects: Article
• Online Access: Get fulltext

 Determining affective states such as confusion from students' participation in online discussion forums can be useful for instructors of a large classroom. However, manual annotation of forum posts by instructors or paid crowd workers is both time-consuming and expensive. In this work, we harness affordances prevalent in social media to allow students to self-Annotate their discussion posts with a set of hashtags and emojis, a process that is fast and cheap. For students, selfannotation with hashtags and emojis provides another channel for self-expression, as well as a way to signal to instructors and other students on the lookout for certain types of messages. This method also provides an easy way to acquire a labeled dataset of affective states, allowing us distinguish between more nuanced emotions such as confusion and curiosity. From a dataset of over 25,000 discussion posts from two courses containing self-Annotated posts by students, we demonstrate how we can identify linguistic differences between posts expressing confusion versus curiosity, achieving 83% accuracy at distinguishing between the two affective states.