Weighting of items in a tutorial performance evaluation instrument: Statistical analysis and results

Weighting of items in an evaluation instrument contributes to more meaningful and valid interpretations of student performance in respect of each learning outcome or item being assessed. It follows that the validity of instruments is important for meaningful inferences about students’ learning perfo...

Full description

Bibliographic Details
Main Authors: Melanie L. Lack, Judith C. Bruce, Piet J. Becker
Format: Article
Language:Afrikaans
Published: AOSIS 2009-05-01
Series:Health SA Gesondheid: Journal of Interdisciplinary Health Sciences
Subjects:
Online Access:https://hsag.co.za/index.php/hsag/article/view/408
id doaj-96fc01a1066343feb231b04c27a2cfcf
record_format Article
spelling doaj-96fc01a1066343feb231b04c27a2cfcf2020-11-25T00:26:52ZafrAOSISHealth SA Gesondheid: Journal of Interdisciplinary Health Sciences1025-98482071-97362009-05-01141e1e510.4102/hsag.v14i1.408404Weighting of items in a tutorial performance evaluation instrument: Statistical analysis and resultsMelanie L. Lack0Judith C. Bruce1Piet J. Becker2University of the WitwatersrandUniversity of the WitwatersrandUniversity of the WitwatersrandWeighting of items in an evaluation instrument contributes to more meaningful and valid interpretations of student performance in respect of each learning outcome or item being assessed. It follows that the validity of instruments is important for meaningful inferences about students’ learning performance, including their performance in tutorial groups. The Delphi technique was used to elicit experts’ subjective judgement of the content validity of items in the tutorial performance evaluation instrument in rounds one and two. A sample of eight experts (n = 8) was selected by purposive, maximum variation sampling. In round three Delphi a weighted score was determined for each of the instrument items, subitems and Likert scale points through pairwise comparison by the experts. Mathematical modelling of experts’ weighting comparisons, recorded on visual analogue scales, resulted in proportional weights for each item; these weights are expressed as a percentage. The final instrument comprised weighted items measured on a rating scale with points that are not equidistant. A computerised tutorial performance evaluator (TPE) was developed for accurate, economical and efficient calculation of student scores. The purpose of this article is to report on the statistical analysis and results of the weighting of items in an instrument to assess and evaluate baccalaureate nursing students’ performance in problem-based learning tutorials. Opsomming Die waardebepaling van items in ‘n evalueringsinstrument dra by tot ‘n meer betekenisvolle en geldige interpretasie van studente se vermoë ten opsigte van elke leeruitkoms of item wat geassesseer word. Hieruit volg dit dat die geldigheid van instrumente belangrik is vir betekenisvolleafleidings betreffende studente se leervermoë, insluitend hulle prestasie in leergroepe. Die Delphi-tegniek is gebruik in rondtes een en twee om kundiges se subjektiewe oordeel oor die inhoudsgeldigheid van items in die leergroepgedragsevalueringsinstrument te bekom. ‘n Steekproef van agt kundiges (n = 8) is deur doelgerigte, maksimale-variasiesteekproeftrekking gekies. Die kundiges het in rondte drie van die Delphi-tegniektoepassing deur middel van gepaarde vergelyking ‘n waarde bepaal vir elk van die instrumentitems, sub-items en Likertskaalpunte. Die wiskundige modellering van kundiges se waardebepalingsvergelykings, aangeteken op visuele analogiese skale, het proporsionele gewigte vir elke item tot gevolg gehad. Hierdie gewigte word persentasiegewys voorgestel. Die finale instrument het bestaan uit items wat gemeet word teen ‘n graderingskaal met punte wat nie op ‘n gelyke afstand van mekaar is nie. ‘n Rekenaargebaseerde leergroepgedragsevalueringsinstrument is ontwikkel vir die akkurate, ekonomiese en doeltreffende berekening van studente se punte. Die doel van die artikel is om verslag te lewer oor die statistiese analise en resultate van die waardebepaling van items in ‘n instrument om baccalaureate-verpleegkundestudente se prestasie in probleemgebaseerde leergroepe te meet en te evalueer.https://hsag.co.za/index.php/hsag/article/view/408weightingtutorial performanceproblem- based learningevaluation instrumentstatistical analysisDelphi technique
collection DOAJ
language Afrikaans
format Article
sources DOAJ
author Melanie L. Lack
Judith C. Bruce
Piet J. Becker
spellingShingle Melanie L. Lack
Judith C. Bruce
Piet J. Becker
Weighting of items in a tutorial performance evaluation instrument: Statistical analysis and results
Health SA Gesondheid: Journal of Interdisciplinary Health Sciences
weighting
tutorial performance
problem- based learning
evaluation instrument
statistical analysis
Delphi technique
author_facet Melanie L. Lack
Judith C. Bruce
Piet J. Becker
author_sort Melanie L. Lack
title Weighting of items in a tutorial performance evaluation instrument: Statistical analysis and results
title_short Weighting of items in a tutorial performance evaluation instrument: Statistical analysis and results
title_full Weighting of items in a tutorial performance evaluation instrument: Statistical analysis and results
title_fullStr Weighting of items in a tutorial performance evaluation instrument: Statistical analysis and results
title_full_unstemmed Weighting of items in a tutorial performance evaluation instrument: Statistical analysis and results
title_sort weighting of items in a tutorial performance evaluation instrument: statistical analysis and results
publisher AOSIS
series Health SA Gesondheid: Journal of Interdisciplinary Health Sciences
issn 1025-9848
2071-9736
publishDate 2009-05-01
description Weighting of items in an evaluation instrument contributes to more meaningful and valid interpretations of student performance in respect of each learning outcome or item being assessed. It follows that the validity of instruments is important for meaningful inferences about students’ learning performance, including their performance in tutorial groups. The Delphi technique was used to elicit experts’ subjective judgement of the content validity of items in the tutorial performance evaluation instrument in rounds one and two. A sample of eight experts (n = 8) was selected by purposive, maximum variation sampling. In round three Delphi a weighted score was determined for each of the instrument items, subitems and Likert scale points through pairwise comparison by the experts. Mathematical modelling of experts’ weighting comparisons, recorded on visual analogue scales, resulted in proportional weights for each item; these weights are expressed as a percentage. The final instrument comprised weighted items measured on a rating scale with points that are not equidistant. A computerised tutorial performance evaluator (TPE) was developed for accurate, economical and efficient calculation of student scores. The purpose of this article is to report on the statistical analysis and results of the weighting of items in an instrument to assess and evaluate baccalaureate nursing students’ performance in problem-based learning tutorials. Opsomming Die waardebepaling van items in ‘n evalueringsinstrument dra by tot ‘n meer betekenisvolle en geldige interpretasie van studente se vermoë ten opsigte van elke leeruitkoms of item wat geassesseer word. Hieruit volg dit dat die geldigheid van instrumente belangrik is vir betekenisvolleafleidings betreffende studente se leervermoë, insluitend hulle prestasie in leergroepe. Die Delphi-tegniek is gebruik in rondtes een en twee om kundiges se subjektiewe oordeel oor die inhoudsgeldigheid van items in die leergroepgedragsevalueringsinstrument te bekom. ‘n Steekproef van agt kundiges (n = 8) is deur doelgerigte, maksimale-variasiesteekproeftrekking gekies. Die kundiges het in rondte drie van die Delphi-tegniektoepassing deur middel van gepaarde vergelyking ‘n waarde bepaal vir elk van die instrumentitems, sub-items en Likertskaalpunte. Die wiskundige modellering van kundiges se waardebepalingsvergelykings, aangeteken op visuele analogiese skale, het proporsionele gewigte vir elke item tot gevolg gehad. Hierdie gewigte word persentasiegewys voorgestel. Die finale instrument het bestaan uit items wat gemeet word teen ‘n graderingskaal met punte wat nie op ‘n gelyke afstand van mekaar is nie. ‘n Rekenaargebaseerde leergroepgedragsevalueringsinstrument is ontwikkel vir die akkurate, ekonomiese en doeltreffende berekening van studente se punte. Die doel van die artikel is om verslag te lewer oor die statistiese analise en resultate van die waardebepaling van items in ‘n instrument om baccalaureate-verpleegkundestudente se prestasie in probleemgebaseerde leergroepe te meet en te evalueer.
topic weighting
tutorial performance
problem- based learning
evaluation instrument
statistical analysis
Delphi technique
url https://hsag.co.za/index.php/hsag/article/view/408
work_keys_str_mv AT melaniellack weightingofitemsinatutorialperformanceevaluationinstrumentstatisticalanalysisandresults
AT judithcbruce weightingofitemsinatutorialperformanceevaluationinstrumentstatisticalanalysisandresults
AT pietjbecker weightingofitemsinatutorialperformanceevaluationinstrumentstatisticalanalysisandresults
_version_ 1725342128649797632