Artificial intelligence literacy development in preservice teachers through metacognitive reflection a mixed methods study

Journal title EXCELLENCE AND INNOVATION IN LEARNING AND TEACHING
Author/s Gabriele Biagini, Maria Ranieri
Publishing Year 2026 Issue 2026/1
Language English Pages 20 P. 5-24 File size 0 KB
DOI 10.3280/exioa1-2026oa23214
DOI is like a bar code for intellectual property: to have more infomation click here

View article Download Open Access article

FrancoAngeli is member of Publishers International Linking Association, Inc (PILA), a not-for-profit association which run the CrossRef service enabling links to and from online scholarly content.

<p>This study examines how future teachers’ AI literacy develops throughout a four-module professional development program and how metacognitive reflection supports transformative learning. Sixty-three Italian teacher-learners produced short reflective diaries after each module. Using a convergent mixed-methods design, we integrated computational text analyses with qualitative coding to characterize change across cognitive, operational, critical, and ethical dimensions of AI literacy. Texts were pre-processed in Italian with spaCy (lemmatization and extended stopword filtering) and represented with TF-IDF to compute inter-section cosine similarity between phases and within-participant coherence; lexical diversity (type-token ratio), unsupervised topic modeling (LDA, k = 5), and sentiment scores complemented the analysis. Qualitative thematic analysis contextualized patterns with coder agreement and representative excerpts. Results indicate declining inter-diary similarity across modules alongside stable or rising within-participant coherence, suggesting cohort-level conceptual reorganization while individuals consolidate their understanding. Lexical diversity decreases modestly as vocabulary specializes; topic distributions shift from generic to practice-oriented themes; and sentiment evolves from uncertainty to more balanced, professionally grounded appraisals. Taken together, the findings portray a progression from initial ambiguity toward increasingly articulated AI literacy, linking computational signals with reflective evidence of transformation. We discuss implications for teacher-education curricula that aim to build sustainable, ethically aware AI capacities without presupposing technical backgrounds.</p>

Keywords: artificial intelligence;teacher education;transformative learning;metacognition;AI literacy;mixed methods;natural language processing

  1. Russell S., & Norvig P. (2020). Artificial intelligence: A modern approach (4th ed.). Pearson.
  2. Salton G., & Buckley C. (1988). Term-weighting approaches in automatic text retrieval. Information Processing & Management, 24(5): 513-523.
  3. Schön D. A. (1983). The reflective practitioner: How professionals think in action. Basic Books.
  4. Akgun S., & Greenhow C. (2022). Artificial intelligence in education: Addressing ethical challenges in K-12 settings. AI and Ethics, 2(3): 431-440.
  5. Ayanwale M. A., Adelana O. P., Molefi R. R., Adeeko O., & Ishola A. M. (2024). Examining artificial intelligence literacy among pre-service teachers for future classrooms. Computers and Education Open, 6, 100179.
  6. Blei D. M., Ng A. Y., & Jordan M. I. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3: 993-1022.
  7. Braun V., & Clarke V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2): 77-101.
  8. Brooks R. E., & Brooks A. K. (2024). An emerging transformative learning journey to foster sustainability leadership in professional development programs. Journal of Transformative Education. Doi: 10.1177/15413446241255909.
  9. Casal-Otero L., Catala A., Fernández-Morante C., Taboada M., Cebreiro B., & Barro S. (2023). AI literacy in K-12: A systematic literature review. International Journal of STEM Education, 10(1), 29.
  10. Creswell J. W., & Plano Clark V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Sage Publications.
  11. Devlin J., Chang M. W., Lee K., & Toutanova K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT (pp. 4171-4186).
  12. Dilek M., Baran E., & Aleman E. (2025). AI literacy in teacher education: Empowering educators through critical co-discovery. Journal of Teacher Education, 76(1). Doi: 10.1177/00224871251325083.
  13. Ding A. C. E., Shi L., Yang H., & Choi I. (2024). Enhancing teacher AI literacy and integration through different types of cases in teacher professional development. Computers and Education Open, 6, 100178.
  14. Du H., Sun Y., Jiang H., Muhammad A., & Liu J. (2024). Exploring the effects of AI literacy in teacher learning: An empirical study. Humanities and Social Sciences Communications, 11, 559.
  15. Ertmer P. A., & Ottenbreit-Leftwich A. T. (2010). Teacher technology change: How knowledge, confidence, beliefs, and culture intersect. Journal of Research on Technology in Education, 42(3), 255-284.
  16. European Commission (2021). Digital Education Action Plan 2021-2027. Publications Office of the European Union.
  17. Grover S., & Pea R. (2013). Computational thinking in K-12: A review of the state of the field. Educational Researcher, 42(1): 38-43.
  18. Holmes W., Bialik M., & Fadel C. (2019). Artificial intelligence in education: Promises and implications for teaching and learning. Center for Curriculum Redesign.
  19. Holmes W., Porayska-Pomsta K., Holstein K., Sutherland E., Baker T., Shum S. B., ... & Koedinger K. R. (2022). Ethics of AI in education: Towards a community-wide framework. International Journal of Artificial Intelligence in Education, 32(3): 504-526.
  20. Kim C., Kim M. K., Lee C., Spector J. M., & DeMeester K. (2013). Teacher beliefs and technology integration. Teaching and Teacher Education, 29: 76-85.
  21. Koehler M. J., Mishra P., Kereluik K., Shin T. S., & Graham C. R. (2014). The technological pedagogical content knowledge framework. In: Handbook of research on educational communications and technology (pp. 101-111). Springer.
  22. Landis J. R., & Koch G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1): 159-174.
  23. Laupichler M. C., Aster A., Schirch J., & Raupach T. (2022). Artificial intelligence literacy in higher and adult education: A scoping literature review. Computers and Education: Artificial Intelligence, 3, 100101.
  24. Lee D., Arnold M., Srivastava A., Plastow K., Strelan P., Ploeckl F., Lekkas D., & Palmer E. (2024). The impact of generative AI on higher education learning and teaching: A study of educators' perspectives. Computers and Education: Artificial Intelligence, 6, 100221.
  25. Lipman M. (2003). Thinking in education (2nd ed.). Cambridge University Press.
  26. Long D., & Magerko B. (2020). What is AI literacy? Competencies and design considerations. In: Proceedings of CHI 2020 (pp. 1-13).
  27. Manning C. D., Raghavan P., & Schütze H. (2008). Introduction to information retrieval. Cambridge University Press.
  28. McConnell T. J., Parker J. M., & Eberhardt J. (2019). Problem-based learning for responsive and transformative teacher professional development. Global Journal of Transformative Education, 1(1), 18-25.
  29. Mezirow J. (1991). Transformative dimensions of adult learning. Jossey-Bass.
  30. Mishra P., & Koehler M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6): 1017-1054.
  31. Moon J. A. (2006). Learning journals: A handbook for reflective practice and professional development (2nd ed.). Routledge.
  32. Ng D. T. K., Leung J. K. L., Chu K. W. S., & Qiao M. S. (2021). AI literacy: Definition, teaching, evaluation and ethical issues. In: Proceedings of ITiCSE 2021 (pp. 504-509).
  33. Röder M., Both A., & Hinneburg A. (2015). Exploring the space of topic coherence measures. In: Proceedings of WSDM (pp. 399-408).
  34. Su J., Zhong Y., & Ng D. T. K. (2022). A meta-review of literature on educational approaches for teaching AI at the K-12 levels in the Asia-Pacific region. Computers and Education: Artificial Intelligence, 3, 100065.
  35. Tan Q., & Tang X. (2025). Unveiling AI literacy in K-12 education: A systematic literature review of empirical research. Interactive Learning Environments. Doi: 10.1080/10494820.2025.2482586.
  36. Taylor E. W. (2008). Transformative learning theory. New Directions for Adult and Continuing Education, (119): 5-15.
  37. Touretzky D., Gardner-McCune C., Martin F., & Seehorn D. (2019). Envisioning AI for K-12: What should every child know about AI? In: Proceedings of AAAI (Vol. 33, pp. 9795-9799).
  38. Tuomi I. (2018). The impact of artificial intelligence on learning, teaching, and education. Publications Office of the European Union.
  39. UNESCO (2019). Beijing consensus on artificial intelligence and education. United Nations Educational, Scientific and Cultural Organization.
  40. Walter Y. (2024). Embracing the future of Artificial Intelligence in the classroom: The relevance of AI literacy, prompt engineering, and critical thinking in modern education. International Journal of Educational Technology in Higher Education, 21, 15.
  41. Wang B., Rau P. L. P., & Yuan T. (2023). Measuring user competence in using artificial intelligence: Validity and reliability of artificial intelligence literacy scale. Behaviour & Information Technology, 42(9): 1324-1337.
  42. Wing J. M. (2006). Computational thinking. Communications of the ACM, 49(3): 33-35.
  43. Yim I. H. Y. (2024). A critical review of teaching and learning artificial intelligence (AI) literacy: Developing an intelligence-based AI literacy framework for primary school education. Computers and Education Open, 5, 100162.
  44. Yim I. H. Y., & Wegerif . (2024). Teachers’ perceptions, attitudes, and acceptance of artificial intelligence (AI) educational learning tools: An exploratory study on AI literacy for young students. Future in Educational Research, 2(1), e65.
  45. Zawacki-Richter O., Marín V. I., Bond M., & Gouverneur F. (2019). Systematic review of research on artificial intelligence applications in higher education. International Journal of Educational Technology in Higher Education, 16(1): 1-27.
  46. Zipf G. K. (1949). Human behavior and the principle of least effort. Addison-Wesley.

Gabriele Biagini, Maria Ranieri, Artificial intelligence literacy development in preservice teachers through metacognitive reflection a mixed methods study in "EXCELLENCE AND INNOVATION IN LEARNING AND TEACHING" 1/2026, pp 5-24, DOI: 10.3280/exioa1-2026oa23214