Accuracy of Automated Developmental Sentence Scoring Software

• Main Author: Judson, Carrie Ann
• Format: Others
• Published: BYU ScholarsArchive 2006
• Subjects: Developmental Sentence Scoring; DSS; language sample; automated; software; Communication Sciences and Disorders
• Online Access: https://scholarsarchive.byu.edu/etd/788; https://scholarsarchive.byu.edu/cgi/viewcontent.cgi?article=1787&context=etd

Developmental Sentence Scoring (DSS; Lee 1974) is a well established, structured method for analyzing a child's expressive syntax within the context of a conversational speech sample. Automated DSS programs may increase efficiency of DSS analysis; however the program must be accurate in order to yield valid and reliable results. A recent study by Sagae, Lavie, and MacWhinney (2005) proposed a new method for analyzing the accuracy of automated language analysis programs. This method was used in addition to previously established methods to analyze the accuracy of a new automated DSS program, entitled DSSA (Channell, 2006). Previously collected language samples from 118 children between the ages of 3 and 11 years in age were manually and automatedly coded for DSS. The overall accuracy of DSSA was about 86%, while the mean point difference was approximately .7. DSSA generally scored language samples of children achieving lower manual DSS scores or children with language impairment with less accuracy than those of other children. While some precautions may need to be taken, accuracy levels are sufficiently high to allow the fully automated use of DSSA as an alternative to manual DSS scoring.