Students’ perceptions of Computerised Adaptive Testing in higher education
DOI:
https://doi.org/10.17159/ijtl.v18i2.17314Keywords:
adaptive testing, assessment methodology, computerised, student perceptions, post- COVID- 19 pandemicAbstract
The COVID-19 pandemic has forced higher education institutions (HEIs) to re-look at their assessment strategy as learning, development, and engagement move more fluidly into the online arena. The purpose of this research was to investigate students' academic and personal perceptions of computer adaptive testing (CAT) in higher education to understand students’ confidence in adopting CAT. Using a quantitative descriptive research design, an online questionnaire was administered to students at private and public HEIs in South Africa, with 600 respondents. The study found that the students were comfortable engaging in online learning and expressed positive perceptions of adopting CAT, with most respondents recommending it for implementation. Students believe that CAT allowed for more productive interaction with material which meets their needs and learning preferences without feeling overburdened. The findings of the study provide HEIs with valuable information on key managerial implications to ensure the successful adoption and implementation of CAT.
References
Alderson, J.C. (2000) Technology in testing: The present and the future. System 28(4) pp.593-603. Alkamel, M.A.A., Chouthaiwale, S.S., Yassin, A.A.A., AlAjmi, Q. & Albaadany, H.Y. (2021) Online testing in higher education institutions during the outbreak of COVID-19: Challenges and opportunities. Studies in Systems, Decision and Control 348 pp.349-363.
Almahasees, Z., Mohsen, K. & Amin, M.O. (2021) Faculty's and students' perceptions of online learning during COVID-19. Frontiers in Education 6:638470, doi:10.3389/feduc.2021.638470 DOI: https://doi.org/10.3389/feduc.2021.638470
Apostolou, B., Blue, M.A. & Daigle, R.J. (2009) Student perceptions about computerised testing in introductory managerial accounting. Journal of Accounting Education 27(2) pp.59-70. DOI: https://doi.org/10.1016/j.jaccedu.2010.02.003
Baik, S. (2022) Computer adaptive testing – Pros & cons, https://codesignal.com/blog/techrecruiting/pros-cons-computer-adaptive-testing/ (Accessed 26 September 2022).
Barros, J.P. (2019) Students' perceptions of paper-based vs. computer-based testing in an introductory programming course. Proceedings of the 10th International Conference on Computer Supported Education, (CSEDU 2018) pp.303-308. DOI: https://doi.org/10.5220/0006794203030308
Charumilind, S., Craven, M., Lamb, J., Sabow, A., Singhal S. & Wilson, M. (2022) When will the COVID-19 pandemic end? https://www.mckinsey.com/industries/healthcare/our-insights/when-will-the-covid-19-pandemic-end (Accessed 13 March 2023).
Choi, I., Kim, K.S. & Boo, J. (2003) Comparability of a paper-based language test and a computerbased language test. Language Testing 20 pp.295-320. DOI: https://doi.org/10.1191/0265532203lt258oa
Chrysafiadi, K., Troussas. & Virvou, M. (2018) A framework for creating automated online adaptive tests using multiple-criteria decision analysis. 2018 IEEE International Conference on Systems, Man and Cybernetics (SMC) pp.226-231. DOI: https://doi.org/10.1109/SMC.2018.00049
Creswell, J.W. & Creswell, J.D. (2018) Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). USA: Sage Publications.
Denney, V., Dixon, Z., Gupta, A. & Hulphers, E. (2021) Exploring the perceived spectrum of plagiarism: A case study of online learning. Journal of Academic Ethics 19(2) pp.187-210. DOI: https://doi.org/10.1007/s10805-020-09364-3
Diahyleva, O.S., Gritsuk, I.V., Kononova, O.Y. &Yurzhenko, A.Y. (2020). Computerised adaptive testing in education electronic environment of maritime higher education institutions. CTE Workshop Proceedings 8 pp.411-422. DOI: https://doi.org/10.55056/cte.297
Dillon, A. (1992). Reading from paper versus screens: A critical review of the empirical literature. Ergonomics 35 pp.1297-1326. DOI: https://doi.org/10.1080/00140139208967394
Eggan, T.J.H.M. (2018). Multi-segment Computerized Adaptive Testing for educational testing purposes. Fronters in Education 3 p.111. DOI: https://doi.org/10.3389/feduc.2018.00111
Elmore, R.F. (2019) The Future of Learning and the Future of Assessment. ECNU Review of Education 2(3) pp.328-341. DOI: https://doi.org/10.1177/2096531119878962
Fynn, A. & Mashile, E.O. (2022) Continuous online assessment at a South African open distance and elearning institution, Frontiers in Education 7:791271, doi:10.3389/feduc.2022.791271 DOI: https://doi.org/10.3389/feduc.2022.791271
Gibbons C, Bower P, Lovell K, Valderas J. & Skevington S. (2016) Electronic quality of life assessment using computer-adaptive testing. Journal of Medical Internet Research 18(9) p.240. DOI: https://doi.org/10.2196/jmir.6053
Han, K.C.T. (2018) Conducting simulation studies for computerised adaptive testing using SimulCAT: An instructional piece. Journal of Educational Evaluation for Health Professions 15(20) p.20. DOI: https://doi.org/10.3352/jeehp.2018.15.20
Harrison, C.J., Geerards, D., Ottenhof, M.J., Klassen, A.F., Riff, K.W.Y. Q., Swan, M. C., Pusic, A. L. & Sidey-Gibbons, C.J. (2019) Computerised adaptive testing accurately predicts CLEFT-Q scores by selecting fewer, more patient-focused questions. Journal of Plastic, Reconstructive and Aesthetic Surgery 72 pp.1819-1824. DOI: https://doi.org/10.1016/j.bjps.2019.05.039
Hogan, T. (2014) Using a computer-adaptive test simulation to investigate test coordinators' perceptions of a high-stakes computer-based testing program. PhD Thesis, Georgia State University, US.
Ita, M.E., Kecskemety, K.M., Ashley, K.E. & Morin, B. (2014) Comparing student performance on computer-based vs. paper-based tests in a first-year engineering course. Proceedings of the 121st ASEE Annual Conference & Exposition Indianapolis, Indiana, US 15-18 June, pp.1-14.
Kaminskiene, L. & DeUrraza, M.J. (2020) The flexibility of curriculum for personalised learning. Proceedings of the 14th International Scientific Conference on Society. Integration, Education. Rezekne, Latvia, 22-23 May, 3 pp.266-273. DOI: https://doi.org/10.17770/sie2020vol3.5009
Khoshsima, H., Toroujeni, S.M.H.T., Thompson. N. & Ebramhimi, M.R. (2019). Computer-based (CBT) vs. paper-based (PBT) testing: Mode effect, relationship between computer familiarity, attitudes, aversion and mode preference with CBT test scores in an Asian private EFL context. Teaching English with Technology 19(1) pp.86-101.
Lee, K.H. (2021) The educational 'metaverse' is coming. The Campus Learn, Share, Connect. https://www.timeshighereducation.com/campus/educational-metaverse-coming (Accessed 15 August 2022).
Li, C. & Lalani, F. (2020) The COVID-19 Pandemic has changed education forever. This is how. World Economic Forum. https://www.weforum.org/agenda/2020/04/coronavirus-education-global-covid19-online-digital-learning (Accessed 20 September 2022).
Lilley, M. & Barker, T. (2007) Students' perceived usefulness of formative feedback for a computeradaptive test. Electronic Journal of e-Learning 5(1) pp.31-38.
Lilley, M., Barker, T. & Britton, C. (2005) Learners' perceived level of difficulty of a computer-adaptive test: A case study. In: Costabile, Human-Computer Interaction 3585 pp.1026-1029. DOI: https://doi.org/10.1007/11555261_96
Meuschke, N., Gondek, C., Seebacher, D., Breitinger, C., Keim, D. & Gipp. B. (2018) An adaptive imagebased plagiarism detection approach. Proceedings of the 18th ACM/IEEE on Joint Conference on Digital Libraries (JCDL '18). Association for Computing Machinery, New York, NY, US, pp.131-140. DOI: https://doi.org/10.1145/3197026.3197042
Minty, R. (2018) To use electronic assessment or paper-based assessment? That is the question (apologies to Shakespeare), The Independent Journal of Teaching and Learning 13(1) pp.17-27.
Moosbrugger, H. & Kelava, A. (2012) Testtheorie und Fragebogenkonstruktion. In Statistik für Humanund Sozialwissenschaftler, pp.7-26. DOI: https://doi.org/10.1007/978-3-642-20072-4_2
Mpungose, C.B. (2020) Emergent transition from face-to-face to online learning in a South African University in the context of the Coronavirus pandemic. Humanities and Social Sciences Communication 7(13) p.113. DOI: https://doi.org/10.1057/s41599-020-00603-x
Nergiz, C. & Seniz, OY. (2013) How Can We Get Benefits of Computer-Based Testing in Engineering Education? Computer Applications in Engineering Education 21(2) pp.287-293. DOI: https://doi.org/10.1002/cae.20470
New Meridian. (2022) Flexible Assessment Systems Enable Educators to Respond to Students' Needs and Expand Educational Equity. https://newmeridiancorp.org/ (Accessed 03 March 2022).
Noyes, J.M. & Garland, K.J. (2008) Computer- vs. paper-based tasks: Are they equivalent? Ergonomics 51(9) pp.1352-1375. DOI: https://doi.org/10.1080/00140130802170387
Oladela, J. & Ndlovu, M.C. (2021) A review of standardised assessment development procedure and algorithms for computer adaptive testing: applications and relevance for Fourth Industrial Revolution. International Journal of Learning, Teaching and Educational Research 20(5) pp.1-17. DOI: https://doi.org/10.26803/ijlter.20.5.1
Oladela, J. (2021) Computer-adaptive-testing performance for postgraduate certification in education as innovative Assessment. International Journal of Innovation, Creativity and Change 15(9) pp.87-101.
Oladele, J.I., Ayanwale, M.A. & Owolabi, H.O. (2020) Paradigm shifts in computer adaptive testing in Nigeria in terms of simulated evidence. Journal of Social Sciences 63(1-3) pp.9-20. DOI: https://doi.org/10.31901/24566756.2020/63.1-3.2264
Oppl, S., Reisinger, F., Eckaier, A. & Helm, C. (2017) A flexible online platform for computerised adaptive testing. International Journal of Educational Technology in Higher Education 14(2). DOI: https://doi.org/10.1186/s41239-017-0039-0
Peng, H., Ma, S. & Spector, J.M. (2019) Personalised adaptive learning: An emerging pedagogical approach enabled by a smart learning environment. Smart Learning Environments 6(1) pp.1-14. DOI: https://doi.org/10.1186/s40561-019-0089-y
Queiros, D.R. & de Villiers, M.R. (2016) Online learning in a South African higher education institution: determining the right connections for the student, International Review of Research in Open and Distributed Learning 17(5) pp.165-185. DOI: https://doi.org/10.19173/irrodl.v17i5.2552
Rahi, S. (2017) Research design and methods: A systematic review of research paradigms, sampling issues and instruments development. International Journal of Economics and Management Sciences 6(2) pp.1-5. DOI: https://doi.org/10.4172/2162-6359.1000403
Ramadan, A.M. &, A.E. (2018) Computerised Adaptive Testing. https://www.researchgate.net/publication/329935670_Computerized_Adaptive_Testing (Accessed 03 March 2022).
Rezaie, M. & Golshan, M. (2015) Computer Adaptive Test (CAT): advantages and limitations. International Journal of Educational Investigations 2(5) pp.128-137.
Saunders, M., Lewis, P. & Thornhill, A. (2019) Research methods for business students (8th ed.). Harlow: Pearson Education Limited.
Sekaran, U. & Bougie, R. (2016) Research methods for business: A skill building approach (7th ed.). Singapore: John Wiley and Sons.
Song, L., Singleton, E.S., Hill, J.R. & Koh, M.H. (2004) Improving online learning: Student perceptions of useful and challenging characteristics. The Internet and Higher Education 7(1) pp.59-70. DOI: https://doi.org/10.1016/j.iheduc.2003.11.003
Sorea D., Roșculeț G. & Bolborici A.M. (2021) Readymade solutions and students' appetite for plagiarism as challenges for online learning. Sustainability 13(7) p.3861. DOI: https://doi.org/10.3390/su13073861
Stochl, J., Ford, T., Perez, J. & Jones, P.B. (2021). Modernising measurement in psychiatry: item banks and computerised adaptive testing. Lancet Psychiatry 8(5) pp.354-356. DOI: https://doi.org/10.1016/S2215-0366(21)00041-9
Stone, E. & Davey, T. (2011) Computer-Adaptive Testing for students with disabilities: A review of the literature. ETS, Princeton, New Jersey. DOI: https://doi.org/10.1002/j.2333-8504.2011.tb02268.x
Surpass. (2019) Computer Adaptive Testing: Background, benefits and case study of a large-scale national testing programme. https://surpass.com/news/2019/computer-adaptive-testing-backgroundbenefits-and-case-study-of-a-large-scale-national-testing-programme/ (Accessed 27 September 2022).
Troussas, C., Krouska, A. & Virvou, M. (2017) Reinforcement theory combined with a badge system to foster student's performance in e-learning environments. 8th IEEE International Conference on Information, Intelligence, Systems & Applications (IISA), Larnaca, Cyprus, 27-30 August, pp.1-6. DOI: https://doi.org/10.1109/IISA.2017.8316421
Thompson, N.A. (2011) Advantages of Computerized Adaptive Testing (CAT). https://assess.com/docs/Advantages-of-CAT-Testing.pdf (Accessed 27 September 2022).
Thompson, N. (2019) Computerised Adaptive Testing: Definition, algorithm, benefits. https://assess.com/what-is-computerized-adaptive-testing/ (Accessed 25 September 2022).
Van Wyk, B., Mooney, G., Duma, M. & Faloye, S. (2020) Emergency remote learning in the times of COVID: A higher education innovation strategy. Proceedings of the European Conference on e-Learning, Berlin, Germany pp.28-30.
Vander Ark, T. (2014) The future of Assessment. Getting Smart. https://www.gettingsmart.com/2014/12/17/future-assessment/ (Accessed 12 May 2022).
Vie, JJ, Popineau, F., Tort, F., Marteau, B. & Denos, N. (2017) A heuristic method for large-scale cognitive-diagnostic computerised adaptive testing. Proceedings of the 4th ACM Conference on Learning at Scale, Cambridge Massachusetts, USA. pp.323-326. DOI: https://doi.org/10.1145/3051457.3054015
Xu, L., Jin, R., Huang, F., Zhou, Y., Li, Z. & Zhang, M. (2020) Development of computerised adaptive testing for emotion regulation. Frontiers in Psychology 11 pp.1-11. DOI: https://doi.org/10.3389/fpsyg.2020.561358
Yıldırım, O. G., Erdoğan, T. & ve Çiğdem, H. (2017) The investigation of the usability of web-based assignment system. Eğitimde Kuram ve Uygulama, 1 pp.1-9.
Zalat, M.H., Hamed, M.S. & Bolbol, S.A. (2021) The experiences, challenges, and acceptance of elearning as a tool for teaching during the COVID-19 pandemic among university medical staff. https://doi.org/10.1371/journal.pone.0248758 (Accessed 13 June 2022). DOI: https://doi.org/10.1371/journal.pone.0248758
Zandvliet, D. & Farragher, P. (1997) A comparison of computer-administered and written tests. Journal of Research on Computing in Education 29 pp.423-438. DOI: https://doi.org/10.1080/08886504.1997.10782209
.png){kind=spacer}