The Effectiveness of Virtual Digital Human Pedagogical Agents in Virtual Reality Learning Environments: A Meta-Analysis of 36 Empirical Studies

Chuan Zhang; Lan Mo

doi:10.71204/6p1ny979

Authors

Chuan Zhang Guangzhou University of Science and Technology Author
Lan Mo Guangzhou University of Science and Technology; Universiti Teknologi MARA Author

DOI:

https://doi.org/10.71204/6p1ny979

Keywords:

Virtual Reality, Virtual Digital Human, Pedagogical Agent, Meta-analysis, Learning Outcomes, Cognitive Load, Social Presence, Educational Technology

Abstract

Virtual Digital Human Pedagogical Agents (VDHPAs) have emerged as vital tools in online and blended learning environments. However, the extent to which they improve learning outcomes in Virtual Reality (VR) environments remains inconclusive. This meta-analysis synthesizes findings from 36 empirical studies conducted between 2013 and 2023 to examine the effectiveness of VDHPAs in VR-based learning. Specifically, we analysed two process-oriented variables—cognitive load and social presence—and three outcome-oriented variables—retention, transfer, and other assessment types. The results indicate that while VDHPAs do not significantly reduce cognitive load (g = -0.084), they significantly enhance learners’ social presence (g = 0.402). Additionally, VDHPAs were found to improve retention (g = 0.451), transfer (g = 0.288), and other test scores (g = 0.423). Moderator analyses revealed that the effects vary depending on agent design features (e.g., gestures, voice, facial expression), content characteristics (e.g., subject domain, knowledge type), and learner attributes (e.g., education level, prior knowledge). This review further discusses the implications of agent embodiment, the "uncanny valley" in affective response, and challenges in long-term outcome assessments. The study contributes to a deeper understanding of how to optimise VDHPAs for immersive learning experiences and highlights directions for future interdisciplinary research in educational technology and digital arts.

Author Biography

Lan Mo, Guangzhou University of Science and Technology; Universiti Teknologi MARA

School of Art and Media, Guangzhou University of Science and Technology, Guangzhou 510555, China; molan@gkd.edu.cn

College of Creative Arts, Universiti Teknologi MARA, Shah Alam 40450, Malaysia; molan@gkd.edu.cn

References

Abril, T., Oliveira, J., & Gamito, P. (2022). Construction and effect of relationships with agents in a virtual reality environment. Virtual Reality, 1–14.

Ba, S., Stein, D., Liu, Q., et al. (2021). Examining the effects of a pedagogical agent with dual-channel emotional cues on learner emotions, cognitive load, and knowledge transfer performance. Journal of Educational Computing Research, 59(6), 1114–1134.

Barrett, A., Pack, A., Monteiro, D., et al. (2023). Exploring the influence of audience familiarity on speaker anxiety and performance in virtual reality and real-life presentation contexts. Behaviour & Information Technology, 1–13.

Beege, M., Schneider, S., Nebel, S., et al. (2017). Ageism–Age coherence within learning material fosters learning. Computers in Human Behavior, 75, 510–519.

Bringula, R. P., Fosgate Jr, I. C. O., Garcia, N. P. R., et al. (2018). Effects of pedagogical agents on students’ mathematics performance: A comparison between two versions. Journal of Educational Computing Research, 56(5), 701–722.

Bu, C., Guo, J., Li, Q., et al. (2021). Does teacher presence in online video instruction promote learning? A meta-analysis of 33 empirical studies. Journal of Distance Education, 39(3).

Carlotto, T., & Jaques, P. A. (2016). The effects of animated pedagogical agents in an English-as-a-foreign-language learning environment. International Journal of Human-Computer Studies, 95, 15–26.

Castro-Alonso, J. C., Wong, R. M., Adesope, O. O., et al. (2021). Effectiveness of multimedia pedagogical agents predicted by diverse theories: A meta-analysis. Educational Psychology Review, 33, 989–1015.

Chen, Z. H., & Chen, S. Y. (2014). When educational agents meet surrogate competition: Impacts of competitive educational agents on students' motivation and performance. Computers & Education, 75, 274–281.

Chiquet, S., Martarelli, C. S., Weibel, D., et al. (2023). Learning by teaching in immersive virtual reality–Absorption tendency increases learning outcomes. Learning and Instruction, 84, 101716.

Craig, S. D., & Schroeder, N. L. (2017). Reconsidering the voice effect when learning from a virtual human. Computers & Education, 114, 193–205.

Dai, L., Jung, M. M., Postma, M., et al. (2022). A systematic review of pedagogical agent research: Similarities, differences and unexplored aspects. Computers & Education, 104607.

Davis, R. O., & Vincent, J. (2019). Sometimes more is better: Agent gestures, procedural knowledge and the foreign language learner. British Journal of Educational Technology, 50(6), 3252–3263.

Davis, R., & Antonenko, P. (2017). Effects of pedagogical agent gestures on social acceptance and learning: Virtual real relationships in an elementary foreign language classroom. Journal of Interactive Learning Research, 28(4), 459–480.

Dinçer, S., & Doğanay, A. (2017). The effects of multiple-pedagogical agents on learners’ academic success, motivation, and cognitive load. Computers & Education, 111, 74–100.

Fountoukidou, S., Ham, J., Matzat, U., et al. (2019). Effects of an artificial agent as a behavioral model on motivational and learning outcomes. Computers in Human Behavior, 97, 84–93.

Goldberg, B., & Cannon-Bowers, J. (2015). Feedback source modality effects on training outcomes in a serious game: Pedagogical agents make a difference. Computers in Human Behavior, 52, 1–11.

Grivokostopoulou, F., Kovas, K., & Perikos, I. (2020). The effectiveness of embodied pedagogical agents and their impact on students learning in virtual worlds. Applied Sciences, 10(5), 1739.

Guo, Y. R., & Goh, D. H. L. (2015). Affect in embodied pedagogical agents: Meta-analytic review. Journal of Educational Computing Research, 53(1), 124–149.

Guo, Y. R., & Goh, D. H. L. (2016). Evaluation of affective embodied agents in an information literacy game. Computers & Education, 103, 59–75.

Guo, Y. R., Goh, D. H. L., Luyt, B., et al. (2015). The effectiveness and acceptance of an affective information literacy tutorial. Computers & Education, 87, 368–384.

Horovitz, T., & Mayer, R. E. (2021). Learning with human and virtual instructors who display happy or bored emotions in video lectures. Computers in Human Behavior, 119, 106724.

Huang, X., & Mayer, R. E. (2016). Benefits of adding anxiety-reducing features to a computer-based multimedia lesson on statistics. Computers in Human Behavior, 63, 293–303.

Johnson, A. M., Ozogul, G., & Reisslein, M. (2015). Supporting multimedia learning with visual signalling and animated pedagogical agent: Moderating effects of prior knowledge. Journal of Computer Assisted Learning, 31(2), 97–115.

Johnson, A. M., Ozogul, G., Moreno, R., et al. (2013). Pedagogical agent signaling of multiple visual engineering representations: The case of the young female agent. Journal of Engineering Education, 102(2), 319–337.

Johnson, W. L., & Lester, J. C. (2016). Face-to-face interaction with pedagogical agents, twenty years later. International Journal of Artificial Intelligence in Education, 26, 25–36.

Kwiatkowski, A., Alvarado, E., Kalogeiton, V., et al. (2022). A survey on reinforcement learning methods in character animation. Computer Graphics Forum, 41(2), 613–639.

Kyrlitsias, C., & Michael‐Grigoriou, D. (2018). Asch conformity experiment using immersive virtual reality. Computer Animation and Virtual Worlds, 29(5), e1804.

Li, W., Wang, F., Mayer, R. E., et al. (2019). Getting the point: Which kinds of gestures by pedagogical agents improve multimedia learning? Journal of Educational Psychology, 111(8), 1382.

Liew, T. W., Zin, N. A. M., Sahari, N., et al. (2016). The effects of a pedagogical agent’s smiling expression on the learner’s emotions and motivation in a virtual learning environment. The International Review of Research in Open and Distributed Learning, 17(5), 248-266.

Lin, L., Ginns, P., Wang, T., et al. (2020). Using a pedagogical agent to deliver conversational style instruction: What benefits can you obtain? Computers & Education, 143, 103658.

Makransky, G., Wismer, P., & Mayer, R. E. (2019). A gender matching effect in learning with pedagogical agents in an immersive virtual reality science simulation. Journal of Computer Assisted Learning, 35(3), 349–358.

Martha, A. S. D., & Santoso, H. B. (2019). The design and impact of the pedagogical agent: A systematic literature review. Journal of Educators Online, 16(1), n1.

Mousavinasab, E., Zarifsanaiey, N., Kalhori, S. N., et al. (2021). Intelligent tutoring systems: A systematic review of characteristics, applications, and evaluation methods. Interactive Learning Environments, 29(1), 142–163.

Nielen, T. M. J., Smith, G. G., Sikkema-de Jong, M. T., et al. (2018). Digital guidance for susceptible readers: Effects on fifth graders’ reading motivation and incidental vocabulary learning. Journal of Educational Computing Research, 56(1), 48–73.

Nye, B. D., Pavlik, P. I., Windsor, A., et al. (2018). SKOPE-IT (Shareable Knowledge Objects as Portable Intelligent Tutors): Overlaying natural language tutoring on an adaptive learning system for mathematics. International Journal of STEM Education, 5, 1–20.

Osman, K., & Lee, T. T. (2014). Impact of interactive multimedia module with pedagogical agents on students’ understanding and motivation in the learning of electrochemistry. International Journal of Science and Mathematics Education, 12, 395–421.

Ospina-Bohórquez, A., Rodríguez-González, S., & Vergara-Rodríguez, D. (2021). On the synergy between virtual reality and multi-agent systems. Sustainability, 13(8), 4326.

Schmidt, S., Bruder, G., & Steinicke, F. (2019). Effects of virtual agent and object representation on experiencing exhibited artifacts. Computers & Graphics, 83, 1–10.

Schroeder, N. L., & Gotch, C. M. (2015). Persisting issues in pedagogical agent research. Journal of Educational Computing Research, 53(2), 183–204.

Schroeder, N. L., Adesope, O. O., & Gilbert, R. B. (2013). How effective are pedagogical agents for learning? A meta-analytic review. Journal of Educational Computing Research, 49(1), 1–39.

Schroeder, N. L., Romine, W. L., & Craig, S. D. (2017). Measuring pedagogical agent persona and the influence of agent persona on learning. Computers & Education, 109, 176–186.

Seymour, M., Yuan, L. I., Dennis, A., et al. (2021). Have we crossed the uncanny valley? Understanding affinity, trustworthiness, and preference for realistic digital humans in immersive environments. Journal of the Association for Information Systems, 22(3), 9.

Shalmani, H. B., & Branch, R. (2021). On the comparison of the effects of conventional and agent-based multimedia instruction on the learning of English speech acts among Iranian EFL learners. Computer Assisted Language Learning, 22(1), 128–157.

Sinatra, A. M., Pollard, K. A., Files, B. T., et al. (2021). Social fidelity in virtual agents: Impacts on presence and learning. Computers in Human Behavior, 114, 106562.

Soliman, M. (2022). Pedagogical intelligence in virtual reality environments. In Learning with Technologies and Technologies in Learning: Experience, Trends and Challenges in Higher Education (pp. 285–302). Springer International Publishing.

Soliman, M., & Guetl, C. (2010). Intelligent pedagogical agents in immersive virtual learning environments: A review. In The 33rd International Convention MIPRO (pp. 827–832). IEEE.

Tai, T. Y., Chen, H. H. J., & Todd, G. (2022). The impact of a virtual reality app on adolescent EFL learners’ vocabulary learning. Computer Assisted Language Learning, 35(4), 892–917.

Tegos, S., & Demetriadis, S. (2017). Conversational agents improve peer learning through building on prior knowledge. Journal of Educational Technology & Society, 20(1), 99–111.

Vaughan, N., Gabrys, B., & Dubey, V. N. (2016). An overview of self-adaptive technologies within virtual reality training. Computer Science Review, 22, 65–87.

Wang, F., Li, W., Xie, H., & Liu, H. (2017). Are pedagogical agents effective in multimedia learning? A meta-analysis. Advances in Psychological Science, 25(1), 12–28.

Wang, X., Qiao, Y., Wang, Y., Cheng, Y., & Li, H. (2022). How do educational agents influence learners’ emotions and learning outcomes? A meta-analysis of 39 experimental studies. Modern Educational Technology, 32(8), 59–66.

Ward, W., Cole, R., Bolaños, D., et al. (2013). My science tutor: A conversational multimedia virtual tutor. Journal of Educational Psychology, 105(4), 1115.

Webster, R. (2016). Declarative knowledge acquisition in immersive virtual learning environments. Interactive Learning Environments, 24(6), 1319–1333.

Xie, H., Wang, F., Zhou, Z., & Wu, P. (2016). A meta-analysis of cueing effects in multimedia learning. Acta Psychologica Sinica, 48(5), 540–555.

Yan, M., Shi, L., Zhang, D., et al. (2020). White paper on the development of virtual digital humans [White paper]. China Artificial Intelligence Industry Development Alliance & Digital Human Working Committee, Zhongguancun Smart Intelligence Alliance.