Application of Generalizability Theory in Measurement Error in 2019 WAEC Mathematics Objective Examination in Benin Metropolis

Kennedy Imasuen; Praise Kehinde Adeosun

doi:doi:10.11648/j.ijpbs.20230802.11

| Peer-Reviewed

Application of Generalizability Theory in Measurement Error in 2019 WAEC Mathematics Objective Examination in Benin Metropolis

Kennedy Imasuen, Praise Kehinde Adeosun

Published in International Journal of Psychological and Brain Sciences (Volume 8, Issue 2)

Received: 2 April 2022 Accepted: 19 April 2022 Published: 29 May 2023

Views: Downloads:

Download PDF

Share This Article

Twitter
Linked In
Facebook

Abstract

This study investigated measurement error in 2019 WAEC senior secondary school examination using generalizability theory. The study was specifically concerned with identifying and analyzing measurement error in the senior secondary school 2019 WAEC mathematics objective examination using generalizability theory, and also to determine the highest contribution of facets: students, items and teachers to measurement error. Four research questions were raised to guide the study. The study was survey which adopted a random effect two-facet fully crossed s×t×i design for a generalizability (G) and decision (D) studies. The population consisted of fifty-six thousand, seven hundred and ninety-seven (5697) senior secondary three (SS3) students in the seventy-five (75) public secondary schools in Benin metropolis for the 2019/2020 academic session. The instrument for data collection was a fifty (50) multiple choice WAEC, mathematics 2019 examination. The instrument has been validated by the West African Examination Council (WAEC). The reliability of the items was ascertained using the Kuder – Richardson 20 (KR 20) to obtain internal consistency. It gave a value of 0.92. Data collected were analyzed using a software EduG version 6.0-e based on analysis of variance (ANOVA) and generalizability. The findings which emerged from the study were: the highest contribution to measurement error in examination scores was the students - teacher interaction which accounted for 68.9%, this was followed by the student factor (27.5%) and the residual, that is, interaction of student, teachers and items (3.6%). A generalizability coefficient of 0.97 high enough to rank order students according to their relative abilities in examinations was obtained when the number of teachers was increased to 78. Based on the findings, it was therefore recommended that generalizability analysis should be carried out by researchers, test developers and examination bodies so as to reduce or eliminate measurement error and hence maximize reliability.

Published in	International Journal of Psychological and Brain Sciences (Volume 8, Issue 2)
DOI	10.11648/j.ijpbs.20230802.11
Page(s)	13-18
Creative Commons	This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.
Copyright	Copyright © The Author(s), 2024. Published by Science Publishing Group

Keywords

Error, Measurement Error, Generalizability, Variance Component

References

[1]	Ameh, V. G. (2014). WAEC releases Nov/Dec 2014 results, says 70% of students failed. Daily Post Nigeria. Retrieved from http://www.google.com/amp/s/dailypost.ng/2014/12/19/waecreleases-nov/dec-2014-results-says-70-students-failed/%3famp.
[2]	Bamidele, S. T., Gana, A. Y., Kehinde, A., & Adekunle, A. R. (2021). Estimating generalizability and dependability indices of students’ scores in teaching practice assessment in a Nigerian College of Education. Sapientia Foundation Journal of Education, Sciences and Gender Studies (SFJESGS), 3 (2); 7 – 15 ISSN: 2734-2522 (Print); ISSN: 2734-2514 (Online).
[3]	Bichi, A. A, Suleiman, A. H & Ali, H. (2019) Students’ achievement in Mathematics: Analyzing the influence of Gender and School Nature. Contemporary Educational Researches journal. 9 (3) 50-56.
[4]	Brennan, R. L (2001) Generalizability Theory: Statistics for Social Science and Public Policy. Springer-Verlag Berlin Heidlberg. New York.
[5]	Cetin, B., Guler, N., & Sarica, R. (2016). Using generalizability theory to examine different concept map scoring methods. Eurasian Journal of Educational Research, 66, 212-228 http://dx.doi.org/10.14689/ejer.2016.66.12
[6]	Eckes, T. (2017). Guest editorial rater effects: Advances in item response modeling of human ratings–Part I. Psychological Test and Assessment Modeling, 59 (4), 443–452.
[7]	Edu G. English program, IRDP. Institut de recherche et de documentation pedagogique. Accessed from https://www.irdp.ch/institut/english-program-1968.html on 05.08.2021.
[8]	Egbulefu, C. A. (2013). Estimating measurement error and score dependability in examinations using generalizability theory. (Unpublished doctoral dissertation) University of Nigeria, Nsukka.
[9]	Esomonu, N. P., & Okeaba, J. U. (2021) Estimating Measurement Error and Score Dependability of the Inventory for Students’ Integration into the University Academic Culture (ISIUAC) Using Generalizability Theory. Rivers State University Journal of Education (RSUJOE), 24 (1): 35-46.
[10]	Güler, N. (2009). Generalizability Theory and Comparison of the Results of G and D Studies Computed by SPSS and Genova Packet Programs. Education and Science, 34, 154.
[11]	Hintze, J. M. & Pettite, H. A. (2001). The Generalizability of CEM Oral Reading Fluency Measures Across General and Special Education. Journal of Psycho Educational Assessment, 19 (1), 52-68.
[12]	Iheanyichukwu, O. R. & Orluwene, G. (2020). Application of Generalizability Theory in Estimating Variance Components in National Examinations Council Problem Solving Questions in Mathematics. European International Journal of Science and Technology, 9 (4), 61-69.
[13]	Johnson, S. Dulanay, C. & Bank, K. (2000). Measurement error. Retrieved from http:/www.wcpss.net/evaluationresearch/reports/2000/mment_error.pdf
[14]	Kaya Uyanik, G., & Guler, N. (2016). Investigation of concept map scores’ reliability: Example of crossed mixed design in generalizability theory. Hacettepe University Journal of Education, 31 (1), 97-111.
[15]	Lee, Y. W. (2005). Dependability of scores for a new ESL speaking assessment consisting of integrated and independent tasks. Language Testing, 23 (2), 131-166.
[16]	Lombardi, Seburn, Conley and Snow (2010) A Generalizability Investigation of Cognitive Demand and Rigor Ratings of Items and Standards. Educational Policy Improvement Center Eugene, Presented at the annual conference of the American Educational Research Association Denver, National Bureau of Statistics (2019).
[17]	National Bureau of Statistics, 2019.
[18]	Ogidi, R. C (2021) Application of generalizability theory in the estimation of variance components in national examination council essay questions in Christian religious studies in Ogba/Egbema/Ndoni local government area of Rivers State, Nigeria. European Journal of Research and Reflection in Educational Sciences, 9 (2): 1-8.
[19]	Omorogiuwa, O. K. (2019). An Introduction to Educational Measurement and Evaluation. Benin City: Mase-Perfect.
[20]	Shavelson, R. J., & Webb, N. M. (1991). Generalizability Theory: A Primer. USA: SAGE Publications.
[21]	Strube, M. J. (2002). Reliability and generalizability theory. In L. G. grimm & P. R. Yarnold (Eds.), Reading and understanding more multivariate statistics (pp. 23-66). Washington, DC: American Psychological Association.
[22]	Webb, N. M., Shavelson, R. J., & Haertel, E. H. (2007). Reliability coefficients and generalizability theory. Handbook of statistics 26: Psychometrics (C. Rao and Sinharay (Eds.) 81, 1-124, the Netherlands: Elsevier.

Cite This Article

Plain Text BibTeX RIS

APA Style

Kennedy Imasuen, Praise Kehinde Adeosun. (2023). Application of Generalizability Theory in Measurement Error in 2019 WAEC Mathematics Objective Examination in Benin Metropolis. International Journal of Psychological and Brain Sciences, 8(2), 13-18. https://doi.org/10.11648/j.ijpbs.20230802.11

Copy | Download

ACS Style

Kennedy Imasuen; Praise Kehinde Adeosun. Application of Generalizability Theory in Measurement Error in 2019 WAEC Mathematics Objective Examination in Benin Metropolis. Int. J. Psychol. Brain Sci. 2023, 8(2), 13-18. doi: 10.11648/j.ijpbs.20230802.11

Copy | Download

AMA Style

Kennedy Imasuen, Praise Kehinde Adeosun. Application of Generalizability Theory in Measurement Error in 2019 WAEC Mathematics Objective Examination in Benin Metropolis. Int J Psychol Brain Sci. 2023;8(2):13-18. doi: 10.11648/j.ijpbs.20230802.11

Copy | Download

@article{10.11648/j.ijpbs.20230802.11,
  author = {Kennedy Imasuen and Praise Kehinde Adeosun},
  title = {Application of Generalizability Theory in Measurement Error in 2019 WAEC Mathematics Objective Examination in Benin Metropolis},
  journal = {International Journal of Psychological and Brain Sciences},
  volume = {8},
  number = {2},
  pages = {13-18},
  doi = {10.11648/j.ijpbs.20230802.11},
  url = {https://doi.org/10.11648/j.ijpbs.20230802.11},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ijpbs.20230802.11},
  abstract = {This study investigated measurement error in 2019 WAEC senior secondary school examination using generalizability theory. The study was specifically concerned with identifying and analyzing measurement error in the senior secondary school 2019 WAEC mathematics objective examination using generalizability theory, and also to determine the highest contribution of facets: students, items and teachers to measurement error. Four research questions were raised to guide the study. The study was survey which adopted a random effect two-facet fully crossed s×t×i design for a generalizability (G) and decision (D) studies. The population consisted of fifty-six thousand, seven hundred and ninety-seven (5697) senior secondary three (SS3) students in the seventy-five (75) public secondary schools in Benin metropolis for the 2019/2020 academic session. The instrument for data collection was a fifty (50) multiple choice WAEC, mathematics 2019 examination. The instrument has been validated by the West African Examination Council (WAEC). The reliability of the items was ascertained using the Kuder – Richardson 20 (KR 20) to obtain internal consistency. It gave a value of 0.92. Data collected were analyzed using a software EduG version 6.0-e based on analysis of variance (ANOVA) and generalizability. The findings which emerged from the study were: the highest contribution to measurement error in examination scores was the students - teacher interaction which accounted for 68.9%, this was followed by the student factor (27.5%) and the residual, that is, interaction of student, teachers and items (3.6%). A generalizability coefficient of 0.97 high enough to rank order students according to their relative abilities in examinations was obtained when the number of teachers was increased to 78. Based on the findings, it was therefore recommended that generalizability analysis should be carried out by researchers, test developers and examination bodies so as to reduce or eliminate measurement error and hence maximize reliability.},
 year = {2023}
}

Copy | Download

TY  - JOUR
T1  - Application of Generalizability Theory in Measurement Error in 2019 WAEC Mathematics Objective Examination in Benin Metropolis
AU  - Kennedy Imasuen
AU  - Praise Kehinde Adeosun
Y1  - 2023/05/29
PY  - 2023
N1  - https://doi.org/10.11648/j.ijpbs.20230802.11
DO  - 10.11648/j.ijpbs.20230802.11
T2  - International Journal of Psychological and Brain Sciences
JF  - International Journal of Psychological and Brain Sciences
JO  - International Journal of Psychological and Brain Sciences
SP  - 13
EP  - 18
PB  - Science Publishing Group
SN  - 2575-1573
UR  - https://doi.org/10.11648/j.ijpbs.20230802.11
AB  - This study investigated measurement error in 2019 WAEC senior secondary school examination using generalizability theory. The study was specifically concerned with identifying and analyzing measurement error in the senior secondary school 2019 WAEC mathematics objective examination using generalizability theory, and also to determine the highest contribution of facets: students, items and teachers to measurement error. Four research questions were raised to guide the study. The study was survey which adopted a random effect two-facet fully crossed s×t×i design for a generalizability (G) and decision (D) studies. The population consisted of fifty-six thousand, seven hundred and ninety-seven (5697) senior secondary three (SS3) students in the seventy-five (75) public secondary schools in Benin metropolis for the 2019/2020 academic session. The instrument for data collection was a fifty (50) multiple choice WAEC, mathematics 2019 examination. The instrument has been validated by the West African Examination Council (WAEC). The reliability of the items was ascertained using the Kuder – Richardson 20 (KR 20) to obtain internal consistency. It gave a value of 0.92. Data collected were analyzed using a software EduG version 6.0-e based on analysis of variance (ANOVA) and generalizability. The findings which emerged from the study were: the highest contribution to measurement error in examination scores was the students - teacher interaction which accounted for 68.9%, this was followed by the student factor (27.5%) and the residual, that is, interaction of student, teachers and items (3.6%). A generalizability coefficient of 0.97 high enough to rank order students according to their relative abilities in examinations was obtained when the number of teachers was increased to 78. Based on the findings, it was therefore recommended that generalizability analysis should be carried out by researchers, test developers and examination bodies so as to reduce or eliminate measurement error and hence maximize reliability.
VL  - 8
IS  - 2
ER  -

Copy | Download

Author Information

Kennedy Imasuen

Institute of Education, University of Benin, Benin City, Nigeria
Praise Kehinde Adeosun

Department of Educational Evaluation and Counseling Psychology, University of Benin, Benin City, Nigeria

Download PDF

Sections

Plain Text BibTeX RIS

APA Style

Kennedy Imasuen, Praise Kehinde Adeosun. (2023). Application of Generalizability Theory in Measurement Error in 2019 WAEC Mathematics Objective Examination in Benin Metropolis. International Journal of Psychological and Brain Sciences, 8(2), 13-18. https://doi.org/10.11648/j.ijpbs.20230802.11

Copy | Download

ACS Style

Kennedy Imasuen; Praise Kehinde Adeosun. Application of Generalizability Theory in Measurement Error in 2019 WAEC Mathematics Objective Examination in Benin Metropolis. Int. J. Psychol. Brain Sci. 2023, 8(2), 13-18. doi: 10.11648/j.ijpbs.20230802.11

Copy | Download

AMA Style

Kennedy Imasuen, Praise Kehinde Adeosun. Application of Generalizability Theory in Measurement Error in 2019 WAEC Mathematics Objective Examination in Benin Metropolis. Int J Psychol Brain Sci. 2023;8(2):13-18. doi: 10.11648/j.ijpbs.20230802.11

Copy | Download

@article{10.11648/j.ijpbs.20230802.11,
  author = {Kennedy Imasuen and Praise Kehinde Adeosun},
  title = {Application of Generalizability Theory in Measurement Error in 2019 WAEC Mathematics Objective Examination in Benin Metropolis},
  journal = {International Journal of Psychological and Brain Sciences},
  volume = {8},
  number = {2},
  pages = {13-18},
  doi = {10.11648/j.ijpbs.20230802.11},
  url = {https://doi.org/10.11648/j.ijpbs.20230802.11},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ijpbs.20230802.11},
  abstract = {This study investigated measurement error in 2019 WAEC senior secondary school examination using generalizability theory. The study was specifically concerned with identifying and analyzing measurement error in the senior secondary school 2019 WAEC mathematics objective examination using generalizability theory, and also to determine the highest contribution of facets: students, items and teachers to measurement error. Four research questions were raised to guide the study. The study was survey which adopted a random effect two-facet fully crossed s×t×i design for a generalizability (G) and decision (D) studies. The population consisted of fifty-six thousand, seven hundred and ninety-seven (5697) senior secondary three (SS3) students in the seventy-five (75) public secondary schools in Benin metropolis for the 2019/2020 academic session. The instrument for data collection was a fifty (50) multiple choice WAEC, mathematics 2019 examination. The instrument has been validated by the West African Examination Council (WAEC). The reliability of the items was ascertained using the Kuder – Richardson 20 (KR 20) to obtain internal consistency. It gave a value of 0.92. Data collected were analyzed using a software EduG version 6.0-e based on analysis of variance (ANOVA) and generalizability. The findings which emerged from the study were: the highest contribution to measurement error in examination scores was the students - teacher interaction which accounted for 68.9%, this was followed by the student factor (27.5%) and the residual, that is, interaction of student, teachers and items (3.6%). A generalizability coefficient of 0.97 high enough to rank order students according to their relative abilities in examinations was obtained when the number of teachers was increased to 78. Based on the findings, it was therefore recommended that generalizability analysis should be carried out by researchers, test developers and examination bodies so as to reduce or eliminate measurement error and hence maximize reliability.},
 year = {2023}
}

Copy | Download

TY  - JOUR
T1  - Application of Generalizability Theory in Measurement Error in 2019 WAEC Mathematics Objective Examination in Benin Metropolis
AU  - Kennedy Imasuen
AU  - Praise Kehinde Adeosun
Y1  - 2023/05/29
PY  - 2023
N1  - https://doi.org/10.11648/j.ijpbs.20230802.11
DO  - 10.11648/j.ijpbs.20230802.11
T2  - International Journal of Psychological and Brain Sciences
JF  - International Journal of Psychological and Brain Sciences
JO  - International Journal of Psychological and Brain Sciences
SP  - 13
EP  - 18
PB  - Science Publishing Group
SN  - 2575-1573
UR  - https://doi.org/10.11648/j.ijpbs.20230802.11
AB  - This study investigated measurement error in 2019 WAEC senior secondary school examination using generalizability theory. The study was specifically concerned with identifying and analyzing measurement error in the senior secondary school 2019 WAEC mathematics objective examination using generalizability theory, and also to determine the highest contribution of facets: students, items and teachers to measurement error. Four research questions were raised to guide the study. The study was survey which adopted a random effect two-facet fully crossed s×t×i design for a generalizability (G) and decision (D) studies. The population consisted of fifty-six thousand, seven hundred and ninety-seven (5697) senior secondary three (SS3) students in the seventy-five (75) public secondary schools in Benin metropolis for the 2019/2020 academic session. The instrument for data collection was a fifty (50) multiple choice WAEC, mathematics 2019 examination. The instrument has been validated by the West African Examination Council (WAEC). The reliability of the items was ascertained using the Kuder – Richardson 20 (KR 20) to obtain internal consistency. It gave a value of 0.92. Data collected were analyzed using a software EduG version 6.0-e based on analysis of variance (ANOVA) and generalizability. The findings which emerged from the study were: the highest contribution to measurement error in examination scores was the students - teacher interaction which accounted for 68.9%, this was followed by the student factor (27.5%) and the residual, that is, interaction of student, teachers and items (3.6%). A generalizability coefficient of 0.97 high enough to rank order students according to their relative abilities in examinations was obtained when the number of teachers was increased to 78. Based on the findings, it was therefore recommended that generalizability analysis should be carried out by researchers, test developers and examination bodies so as to reduce or eliminate measurement error and hence maximize reliability.
VL  - 8
IS  - 2
ER  -

Copy | Download