Assessing the Effect Size and Homogeneity of Selected Standardized Tests in Education
DOI:
https://doi.org/10.37812/josse.v3i2.1829Keywords:
Effect size, Homogeneity, Test, Standardised, EducationAbstract
The study was conducted to analyse the empirical differences in the studies on procedures for validating standardised tests in Education. It examined the difference in the findings of the selected studies in terms of the sample size and also determined the homogeneity of the effect size of the selected studies. The study adopted an ex-post factor research design. The study population comprised empirical studies on validating standardised tests in Education (1988 to 2017). The sample size consisted of 130 empirical studies on the validation of standardised tests in education (1988-2017), which were selected using a purpose sampling technique. The research instrument comprised both published and unpublished articles, PhD and Masters theses. Data were analysed using counts and percentages, study variance, and statistical transformations to convert t-test and f-test to corresponding ‘r’ statistics. Also, Q statistics, which is the sum of weighted squares based on Chi-square, I2, Tau2 and Fishers Z, were used for the analysis. Results showed an inverse relationship between sample and effect sizes in the selected studies. Also, there is no significant difference in the effect sizes of studies (Zr=1.14403). Furthermore, the test for homogeneity showed that the studies significantly differ in their effect sizes; Q=24940.437, I2 = 99.48%, and Tau2 = 0.209>0 Tau=0.457 do not belong to the same population.
References
Abu Bakar, M., Umroh, K. A., & Hameed, F. (2023). Improving Quality Islamic Education for Today’s Generation. At-Tadzkir: Islamic Education Journal, 2(2), 118–128. https://doi.org/10.59373/attadzkir.v2i2.42
Arif, M., & Aziz, M. K. N. A. (2023). Islamic Religious Education Learning Model in the 21st Century: Systematic Literature Review. Indonesian Journal of Islamic Education Studies (IJIES), 6(2), 237-262. https://doi.org/10.33367/ijies.v6i2.4417
Arif, M., Chapakiya, S., & Dewi, A. Y. (2024). Character Education in Indonesia Islamic Elementary Schools: A Systematic Literature Review (2014-2024). J-PAI: Jurnal Pendidikan Agama Islam, 11(1).
Arif, M., Dorloh, S., & Abdullah, S. (2024). A Systematic Literature Review of Islamic Boarding School (Pesantren) Education in Indonesia (2014-2024). Tribakti: Jurnal Pemikiran Keislaman, 35(2), 161-180. https://doi.org/10.33367/tribakti.v35i2.5330
Barker, T.H., Habibi, N., Aromataris, E., Stone, J.C., Leonardi-Bee, J., Sears, K., Hasanoff, S., Klugar, M., Tufanaru, C., Moola, S. & Munn, Z. (2024). The revised JBI critical appraisal tool assesses the risk of bias for quasi-experimental studies. JBI Evidence Synthesis, 22(3), 378–388.
Carroll, L. M., Morris, M. E., O’Connor, W. T., & Clifford, A. M. (2020). Is aquatic therapy optimally prescribed for Parkinson’s disease? A systematic review and meta-analysis. Journal of Parkinson’s disease, 10(1), 59–76.
Çoğaltay, N., & Karadağ, E. (2015). Introduction to meta-analysis. Leadership and organisational outcomes: Meta-analysis of empirical studies, 19-28.
Cook, B. G., Cook, L., & Therrien, W. J. (2018). Group–difference effect sizes: Gauging the practical importance of findings from group–experimental research. Learning Disabilities Research & Practice, 33(2), 56-63.
Dattalo, P. (2013). Analysis of multiple dependent variables. Oxford University Press.
Erisen, Y., & Gunay, R. (2015). A meta-analysis into the effectiveness of doctoral dissertations on constructivist learning. The Anthropologist, 21(1-2), 202-212.
Gignac, G. E., & Szodorai, E. T. (2016). Effect size guidelines for individual differences researchers. Personality and individual differences, 102, 74-78.
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European journal of epidemiology, 31(4), 337–350.
Gurung, R. A., Chick, N. L., & Haynie, A. (Eds.). (2023). Exploring signature pedagogies: Approaches to teaching disciplinary habits of mind. Taylor & Francis.
Huda, M., Arif, M., Rahim, M. M. A., & Anshari, M. (2024). Islamic Religious Education Learning Media in the Technology Era: A Systematic Literature Review. At-Tadzkir: Islamic Education Journal, 3(2), 83–103. https://doi.org/10.59373/attadzkir.v3i2.62
Kadirhanoğulları, M. K. (2023). The Effect of Teaching with Concept Maps on Academic Success in Biology Teaching: A Meta-Analysis Study. Dokuz Eylül Üniversitesi Buca Eğitim Fakültesi Dergisi, (58), 2781-2796.
Komalasari, M., & Yakubu, A. B. (2023). Implementation of Student Character Formation Through Islamic Religious Education. At-Tadzkir: Islamic Education Journal, 2(1), 52–64. https://doi.org/10.59373/attadzkir.v2i1.16
Kulinskaya, E., Hoaglin, D. C., Bakbergenuly, I., & Newman, J. (2021). AQ statistic with constant weights for assessing heterogeneity in meta‐analysis. Research Synthesis Methods, 12(6), 711–730.
Liu, H., Li, T. W., Liang, L., & Hou, W. K. (2021). Trauma exposure and mental health of prisoners and ex-prisoners: A systematic review and meta-analysis. Clinical Psychology Review, 89, 102069.
Mellinger, C., & Hanson, T. (2016). Quantitative research methods in translation and interpreting studies. Routledge.
Mengist, W., Soromessa, T., & Legese, G. (2020). Method for conducting systematic literature review and meta-analysis for environmental science research. MethodsX, 7, 100777.
Mikolajewicz, N., & Komarova, S. V. (2019). Meta-analytic methodology for basic research: a practical guide. Frontiers in Physiology, 10, 203.
Nahar, L. (2023). The effects of standardised tests on incorporating 21st-century skills in science classrooms. Integrated Science Education Journal, 4(2), 36–42.
Nuijten, M. B., Van Assen, M. A., Augusteijn, H. E., Crompvoets, E. A., & Wicherts, J. M. (2020). Effect sizes, power, and biases in intelligence research: A meta-meta-analysis. Journal of Intelligence, 8(4), 36.
Ovute, A. O. (2015). Meta-Analysis of Research Findings on Influence of School Location on Students’ Achievement in Mathematics. American-Eurasian Journal of Scientific Research, 10(1), 18-21.
Paramole, O. C. (2021). A Meta-Analysis of Procedures for the Validation of Standardised Tests in Education (1988-2017) (Doctoral dissertation, Obafemi Awolowo University).
Paramole, O. C., & Afolabi, E. R. I. (2024). Meta-Analysis of Item Generation Procedures Used in Selected Standardised Tests in Education. Jurnal Pendidikan: Riset dan Konseptual, 8(4), 679-689.
Perdices, M. (2018). Null hypothesis significance testing, p-values, effects sizes and confidence intervals. Brain Impairment, 19(1), 70–80.
Pigott, T. D., & Polanin, J. R. (2020). Methodological guidance paper: High-quality meta-analysis in a systematic review. Review of Educational Research, 90(1), 24-46.
Pomroy, P. (2020). An investigation into the challenges faced by international school teachers when interpreting the results from standardised tests (Doctoral dissertation, Durham University).
Ratnah, Ali Shah, S. A., & Alam, M. (2024). Integrating Religious Moderation into Islamic Religious Education: Strategies and Impacts. At-Tadzkir: Islamic Education Journal, 3(2), 120–133. https://doi.org/10.59373/attadzkir.v3i2.67
Rohmadiyah, B., Zamroni, M. A., & Ismawati. (2024). Principal Strategies in School Management at the State Vocational High School. Kharisma: Jurnal Administrasi Dan Manajemen Pendidikan, 3(1), 1–15. https://doi.org/10.59373/kharisma.v3i1.43
Schäfer, T., & Schwarz, M. A. (2019). The meaningfulness of effect sizes in psychological research: Differences between sub-disciplines and the impact of potential biases. Frontiers in Psychology, 10, 813.
Simpson, A. (2017). The misdirection of public policy: Comparing and combining standardised effect sizes. Journal of Education Policy, 32(4), 450–466.
Soni, K. D. (2023). When do we need to do a meta-analysis? Indian Journal of Anaesthesia, 67(8), 673–674.
Tanha, K., Mohammadi, N., & Janani, L. (2017). P-value: What is and what is not. Medical journal of the Islamic Republic of Iran, 31, 65.
Tellinghuisen, J. (2022). Goodness-of-Fit tests in calibration: are they any good for selecting least-squares weighting formulas? Analytical Chemistry, 94(46), 15997-16005.
Veroniki, A.A., Jackson, D., Bender, R., Kuss, O., Langan, D., Higgins, J.P., Knapp, G. & Salanti, G. (2019). Methods to calculate uncertainty in the estimated overall effect size from a random‐effects meta‐analysis. Research synthesis methods, 10(1), 23-43.
Voyer, D., & Voyer, S. D. (2014). Gender differences in scholastic achievement: a meta-analysis. Psychological bulletin, 140(4), 1174.
Zaini, M., Barnoto, B., & Ashari, A. (2024). Improving Teacher Performance and Education Quality through Madrasah Principal Leadership. Kharisma: Jurnal Administrasi Dan Manajemen Pendidikan, 2(2), 79–90. https://doi.org/10.59373/kharisma.v2i2.23 (Original work published June 12, 2023)
Zhao, Y., Li, T., & Liu, W. (2023). The Benefits and Drawbacks of Standardized Curriculum in Education. Research and Advances in Education, 2(10), 41–47.
Zubar, F. A. (2014). A Meta-Analytic Assessment of Empirical Differences on Standard setting Procedures in Public Examinations. Unpublished PhD Thesis, Obafemi Awolowo University, Ile-Ife.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Temitope Babatimehin, Oyeronke Christiana Paramole, Olufunke Favour Akindahunsi, Temitope Sarah Ogungbaigbe
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
