Files in this item
Halo effects in rating data : assessing speech fluency
Item metadata
dc.contributor.author | O’Grady, Stefan | |
dc.date.accessioned | 2023-03-31T11:30:04Z | |
dc.date.available | 2023-03-31T11:30:04Z | |
dc.date.issued | 2023-08-01 | |
dc.identifier | 283683614 | |
dc.identifier | 0a4f7b6d-52e8-4aaf-9cd6-2fd17c004a01 | |
dc.identifier | 85173848668 | |
dc.identifier.citation | O’Grady , S 2023 , ' Halo effects in rating data : assessing speech fluency ' , Research Methods in Applied Linguistics , vol. 2 , no. 2 , 100048 . https://doi.org/10.1016/j.rmal.2023.100048 | en |
dc.identifier.issn | 2772-7661 | |
dc.identifier.other | ORCID: /0000-0003-3810-713X/work/132214270 | |
dc.identifier.uri | https://hdl.handle.net/10023/27311 | |
dc.description.abstract | Fluency is a common objective in English language learning and teaching. However, researchers have commented on the absence of a widely accepted definition of the construct and this sense of uncertainty may hinder efforts to measure fluency for purposes of research or assessment. To date, the extent to which rating instruments measure fluency independently from other areas of speech production such as complexity and accuracy has been under-explored. This is a significant gap because the literature broadly suggests that rater scores are susceptible to halo effects that have a distorting influence on the measurement of speaking skills and blur boundaries between assessment criteria. To investigate this issue, the current study examines a data set of scores assigned to 77 English language learners on two speaking tasks using an analytic rating scale featuring criteria for spech complexity, accuracy and fluency (CAF). The tasks were transcribed and analysed using measures of CAF. Rater scores were analysed using many-facet Rasch measurement and multiple regression. Results revealed that rated fluency was influenced by lexical complexity, indicating that fluency scores represented more than the fluency construct outlined in the analytic scale. Measures of speech speed, phonation time ration, length of utterance, lexical complexity, total speaking time and repair fluency explained the largest amount of variance in the fluency scores. Implications for research, language teaching and assessment are discussed. | |
dc.format.extent | 12 | |
dc.format.extent | 686320 | |
dc.language.iso | eng | |
dc.relation.ispartof | Research Methods in Applied Linguistics | en |
dc.subject | PE English | en |
dc.subject | MCC | en |
dc.subject.lcc | PE | en |
dc.title | Halo effects in rating data : assessing speech fluency | en |
dc.type | Journal article | en |
dc.contributor.institution | University of St Andrews. International Education Institute | en |
dc.identifier.doi | https://doi.org/10.1016/j.rmal.2023.100048 | |
dc.description.status | Peer reviewed | en |
This item appears in the following Collection(s)
Items in the St Andrews Research Repository are protected by copyright, with all rights reserved, unless otherwise indicated.