Exploring Language Pedagogy with Virtual Reality and Artificial Intelligence

Abstract

Copyright © 2023 The Authors. Virtual Reality (VR) is a highly immersive and interactive experience that renders users to be engrossed in a 3D virtual environment. The recent technological advancements with high-resolution headset display, and accurate tracking of six degrees of freedom paired with controllers allow life-like renditions of real-world scenarios as well as fictional scenarios without potential environmental risks. This paper explores the usage of Virtual Reality in education by incorporating current pedagogical approaches into an interactive 3D virtual environment. The focus of this study revolves around language pedagogy, in specific, the tool developed allows teach users fundamental Mandarin Chinese. This educational VR application enables users to practice their reading and writing skills through a calligraphy lesson and engages users in a listening and speaking lesson through natural conversation. To achieve an organic dialogue, phrases spoken by the user in a lesson are validated immediately through an intuitive phrase recognition system developed using machine learning. The developed prototype has undergone testing to ensure its efficacy. An initial investigation into this prototype found that the majority of participants were supportive of this concept and believe that it would improve the engagement of digital education.