STUDENTS who have never been to a mangrove swamp may find it hard to imagine what it’s like to be there. While images and videos are helpful, they are not very effective.
“Wouldn’t it be better if you had the chance to go to the mangrove swamp, look up and see the leaf patterns of the canopy, look down to see the crabs scampering off, and select the animals you wish to see — n the kingfisher or the butterfly?
“Virtual reality can bring this immersive environment into the classroom. It is more cost effective than going on an actual trip,” said Associate Professor Dr Dorothy Dewitt from Universiti Malaya’s Faculty of Education.
Virtual Reality (VR) and Augmented Reality (AR) are among the technology pillars of Industry 4.0. While they are often used interchangeably, there are significant differences between the two.
VR removes users from the real world into a simulated one, Dewitt said.
“The sense of presence in the environment makes learning with VR a realistic experience. This is highlighted by stereoscopic visuals, sound and haptic feedback for a sensory immersion. Through actional immersion, students are empowered to perform actions which may not be possible in the real world, such as teleporting to Mars.”
It can also trigger symbolic immersion, she added.
“Psychological associations such as fear and empathy can be drawn from the learners’ beliefs, emotions, and values about the real world. For example, users can develop empathy by experiencing being in a refugee camp or a war zone.
“The immersive nature of VR also enables transduction, where students can interact with virtual objects in a micro-world, such as in the cells of the body,” said Dewitt.
While VR replaces one’s vision, AR adds to it. AR combines real and virtual objects in a real environment in real time, according to Monash University Malaysia Jeffrey Cheah School of Medicine and Health Sciences (JCSMHS) academic director Associate Professor Dr Arkendu Sen.
Dr Sen, who holds a doctorate in Technology Enhanced Learning and a Master of Surgery in Anatomy said: “In the realm of AR, artificial intelligence is applied for its recognition of the ‘real world’ to project the virtual information, allowing user interaction.
“Three dimensional (3D) objects are rendered interactively and manipulated digitally by the user. This creates the magic of immersive learning experiences.”
He said that AR has the ability to enhance the real environment and perceptually enrich the user’s experience. “This makes it an enticing technology for educators to use as a tool since the real environment gets annotated and informative. For example, we can use AR to overlay a virtual biology model of a heart on a real human.”
Higher Ed looks at how VR and AR are enhancing education in the medical and engineering fields.
Transforming the classroom
There is a need for educators to shift gears to embrace 21st century learning, said Associate Professor Dr Lakshmi Selvaratnam from Monash University Malaysia JCSMHS.
“Our typical classrooms are often vestigial products of a bygone 20th century era suited to facilitate standardised mass education and disregarding the need for individualised learning. With the advent of VR and AR, we need to selectively implement these assistive technologies and harness their potential to support immersive teaching and learning.
“I strongly believe that any education technology implementation must be based on sound pedagogical principles,” said Dr Lakshmi.
Pedagogically, VR and AR allow users to have multiple perspectives, said Dewitt.
“The learner can construct his knowledge at his own pace. Forming connections and describing his understanding of the concepts in his own words involves higher order thinking.These technologies can also improve higher order thinking by encouraging cognitive flexibility, inductive and deductive thinking skills, better perception of sequence and time, and analogical thinking.
“Research has shown that the use of VR and AR highly motivates students to engage in the learning process,” she said.
Dewitt trains future educators to use AR and VR as teaching resources.
“An example of AR is by using marker images on posters developed by students. The markers trigger explanations in the form of video overlays to elaborate definitions related to a topic.
“In a Masters of Curriculum course, my students learn to create AR overlays to describe the uniqueness of an Asean country’s curriculum. Abstract knowledge and understanding of the curriculum are needed to design the overlays,” said Dewitt, adding that VR is a great tool in educational psychology. “It can play a role in influencing emotions. Emotional experiences can be simulated through devices like Oculus Go, Google glasses and mobile apps like In Mind.”
Meanwhile, many STEM education areas emphasise on situated learning, said Dr Sen.
Students are more inclined to learn by actively participating in a learning experience that is embedded within real social and physical contexts. “AR applications can increase students’ focus and engagement. Students are able to construct new meaningful situational experiences with these augmented and virtual objects that are brought to life.
“For a generation of learners more technologically fluent than ever, AR applications offer a very efficient method to deliver bite-sized learning contents,” he said.
Dr Sen added that AR can satisfy the criteria of smart learning environments (SLE).
“SLE is characterised by being context-aware, adaptive and personalised. Students are given guidance and feedback based upon their situation or topic of interest.”
CLINICAL SIMULATION
Monash University Malaysia JCSMHS Emergency Medicine lecturer Dr Najnin Ahmed has extensive experience in conducting simulation workshops for medical undergraduates.
“Simulation has a great impact on clinical teaching, especially for emergency cases like life support workshops and trauma cases. We conduct simulations on various clinical scenarios on high fidelity manikins, also known as patient simulators.”
Medical training faces limitations when it comes to emergency and acute care, she explained.
“Adequate hands-on exposure for undergraduate levels were continuously seen as a challenge as we need to ensure the patients’ safety. This is how immersive technologies are reshaping medical and surgical training.
“In bridging the gap between theory and practical knowledge, tools like gamifications can make real wonders here. VR and AR can give students close to real-life exposure, which is needed in the clinical setup without risking patient care.
“VR can shield the weaknesses of simulation by adding the human factors so students can experience a more genuine feeling of the scenario and develop empathy for patients. Hands on ‘one to one’ clinical teaching can also be transformed into ‘one to many’ through this technology,” said Dr Najnin.
For better exploration of the human anatomy, Dr Sen and his team developed the Monash Augmented Reality Anatomy Learning Objects (Maralo).
“Similar to virtual museums, the repository of anatomy learning objects in the laboratory is designed as an AR museum for medical students to explore. When the embedded ‘triggers’ or ‘markers’ are scanned, high-resolution 3D images of cadaveric and pathological AR content will be displayed.”
He added that AR can superimpose real clinical or laboratory environments into the classroom activity, thus aiding situational learning. “It allows students to be engaged in real life clinical situations like drawing blood, conducting an electrocardiogram (ECG) or skin suturing which could be also be personalised depending upon their existing knowledge of a skill. They will also receive appropriate feedback.”
He added: “Such an advantage is critical, as only limited actual clinical experience can exist for students in the early years of medical education.”
According to Dr Lakshmi, VR and AR features are also being built into clinical procedural simulators, endoscopy equipment and robotic-guided surgery.
“It is important that these tools are used by medical students at appropriate time points in the curriculum where simulation training is helpful, prior to switching over to working hands-on with real patients during the later clinical years. However, students may get too engrossed in the limited experiential learning provided by the controlled simulation technology that they cannot relate to the real world. For instance, the sights, smells, sounds and touch of the human patient.
“That is where educators should step in to ensure there is a fine balance between learning from technological tools and from real patients,” said Dr Lakshmi.
VIRTUAL EXPERIENCE
For petroleum engineering students, visits to offshore rigs are not easy due to their remoteness and harsh environmental conditions.
With the help of AR and VR, students can experience highly inaccessible sites and locations, said Associate Professor Dr Mazura Jusoh from Universiti Teknologi Malaysia (UTM) Engineering Faculty.
“Students will have an experience of riding a helicopter to the platform as well as a virtual visit to the floors equipped with the essential equipment in a realistic-looking setup.
“This will enhance their understanding of the setting of an oil and gas platform. The same can be said for chemical or nuclear plants, where an actual visit can be dangerous,” said Mazura.
Dr Ajune Wanis Ismail, a senior lecturer from UTM’s Computing Faculty has been working on AR and VR technology for over a decade. Her PhD research was supervised by the pioneer of ARToolkit Professor Dr Mark Billinghurst.
“Students now have the ability to alter time to make experiments faster, or go back in time to fix a mistake. AR and VR have long been touted for their potential to revolutionise education and the technologies are becoming more versatile and affordable,” said Ajune Wanis.
After undergoing the virtual experience, engineering students would be better equipped to train in real conditions. “VR has the ability to enhance engagement and improve retention, as students get to learn through experience. The technology also comes with an integrated haptic system that would generate the sensations of touch and tactile resistance, creating the illusion of performing real procedures,” added Ajune Wanis.
However, the process of developing the simulation was not an easy feat, she added.
“We collected extensive information, gathered all subject matter experts including lecturers to bring 3D simulation of offshore assets and operations to life. The most challenging thing was to ensure the realism of a real-time simulation, so students will feel its authenticity.”
AR is also used in construction engineering to plan construction sites and improve site safety.
Monash University Engineering lecturer Dr Ali Rashidi developed a mixed reality game prototypes for Architecture, Engineering and Construction (AEC) Education.
“Using unique design principles, we merged engagement factors with effective educational criteria to advance the capability training and learning assessment of the AEC workforce.The game provides a hazard-free and low-risk environment compared to the actual dangerous, dirty and difficult on-site training process. This facilitates users to learn critical job-site skills and safety challenges.”
Ali said that students have an opportunity to interact with augmented 3D objects which fosters experiential learning. “It can motivate them to pursue an engineering career more by engaging them throughout the learning and teaching process.”
AR and VR can also influence the students’ spatial ability for real-time visualisation, which are crucial skills in the engineering field.
“Students can explore the use of advanced holographic technologies to bring virtual 3D building components to life. Physical activities can be integrated with the virtual environment in order to promote hybrid training.
“I believe that VR and AR have a definite future to encapsulate the essential capability training needs in moving towards the sustainable value creation in the education sector.These technologies will allow students to prosper as more hands-on and experienced future engineers within a shorter span of time,” said Ali.
BARRIERS
To be at the cutting edge of new technologies, universities may face real barriers in the implementation of VR and AR.
According to Ali, there are many infrastructural challenges that need to be addressed.
“For example, providing digital gadgets and creating the VR and AR educational contents are costly and require higher-order skills in teaching and training prototype development.In addition, there is a need for a standard method and system to translate different conventional training curriculum into appropriate and effective learning mechanisms.”
Dewitt said: “Lecturers may lack technical skills or feel afraid to explore VR and AR. Confidence and skills need to be developed.
“There is also the need to invest in resources and technical equipment. But, you can start with cheaper alternatives and small mobile devices first.”
She stressed that pedagogy is important, not just technology. “There should always be an introduction and a debriefing session to discuss and reflect on what the students have seen,” said Dewitt.
Ajune Wanis added: “A transition period to get accustomed to new formats and platforms is necessary. Immersive and interactive classroom technology is no exception.”
Some of the programming aids are not user-friendly for the academics, said Dr Sen.
“Although open libraries and software development kits are freely available, most of them are based upon mid-to-low-level programming languages. Some AR objects are 3D models superimposed on the real environment, and this 3D modelling requires skills in design software applications.” The degree of realism is still limited in some way, he added.
“Accommodation-vergence conflicts in the eye, low resolution, and inaccurate lighting can cause the virtual object to appear further away. Further development in graphics is needed to simulate better match depth and real distances as in a human eye.”