As societies evolve and take in a greater number of distinct cultures, histories, and traditions, the ability to empathise with each other becomes vital, for all our sakes. In an effort to get as close as possible to seeing life through another’s eyes, researchers from the University of Malta are creating a virtual reality experience that allows users to step into someone else’s shoes.
Words by Dr Vanessa Camilleri.
From a young age we are often taught that if we want to understand someone else’s perspective, we must first walk a mile in their shoes. This ability to place ourselves in another’s position is what we call empathy. This component of emotional intelligence is known to increase prosocial behaviour and reduce individualistic traits, meaning that it can lead to a better quality of life where practiced, whether at home, in the workplace, or any other environment.
Empathy is particularly important in the classroom, where comfort and a sense of belonging can improve the engagement and learning prospects of children from different backgrounds. To promote the inclusion of migrant children, teachers and pupils need to understand how different growing up from another cultural perspective can be. This made us question—is there a technology that can be exploited to elicit empathy by allowing users to walk in other people’s shoes, albeit temporarily?
As it turns out, there is. Virtual reality.
Going down the rabbit hole
At the University of Malta, our inter-faculty initiative between the Department of Artificial Intelligence at Faculty of ICT and the Department of Digital Arts at the Faculty of Media and Knowledge Sciences is now developing a VR experience to understand how migrant children perceive daily life at school. The project, titled ‘If you aren’t experiencing it, you’re not understanding it!’, places the participating teachers in the virtual point-of-view of children coming from migrant backgrounds. We use seamless implementation of footage and sound in VR technology together with a strong narrative to create an immersive experience. We think of immersion as the degree to which a VR system projects sensory stimuli in users, that are matching, vivid, and interactive.
The collaboration was born from the realisation that creating a VR experience requires various expertise including story writing, narrative building, audiovisual techniques, and photography, as well as technical development of the actual VR app. The group was comprised of four research students under the supervision of four academics, sharing knowledge and resources.
How does it work?
This technology will see teachers transported into a virtual world by wearing a visual display headset supported by a surround-sound system that provides three-dimensional audio stimulation. The ability to experience their surroundings through multiple senses will provide context to the occurrence, whilst interactions with virtual people will create a sense of presence built around the illusions of space, self-embodiment, and physical interaction.
The experiences that are triggered and then projected depend on the context of the scenario in which the virtual child finds themselves. What the participant sees and hears in the 360-degree
immersive experience is taken from the perspective of a child negotiating daily school encounters, whilst also carrying mental and emotional baggage due to experiences arising from family interactions and narratives. These narratives are shown as a series of flashbacks triggered by what we may consider to be trivial actions or behaviours, but which may carry deeper meaning to those coming from different cultural backgrounds.
A long journey
Developing a project such as this one doesn’t come without hurdles. Our researchers not only had to deal with the technical development aspect of a VR experience, but with integrating audio in a way that augments the visual experiences being projected. When choosing equipment, both budget limitations and technological capabilities were taken into account. The equipment needed to be unobtrusive, yet powerful enough to capture high-quality footage and pristine three-dimensional sound: both paramount for the success of this project. We ultimately decided on a Samsung Gear 360 camera, a binaural mic, Samsung Gear VR headset, and Samsung Phones. Filming in 3D involves novel challenges, from the way the camera has to project motion without enhancing the motion sickness of first-time VR users, to the filming techniques used for a 360-degree projection on a small VR screen.
The ability to experience their surroundings through multiple senses will provide context to the occurrence, whilst interactions with virtual people will create a sense of presence built around the illusions of space, self-embodiment, project such as this one and physical interaction.
Once the footage was collected, the Unity game engine was used to develop the software required to digitally represent the teaching environment. However, recreating life experiences as though they were a reflection of the user’s perceived reality and achieving a sense of presence inside the virtual reality means VR must go beyond mere technology-induced immersion. To do this, we began by identifying scenes that would provide insight into the life experiences of migrant children. The script was written to support the narrative, which needed to unfold in a way that would ensure a seamless experience for the teacher immersed within it. Actors were recruited, and a school granted help, support, and the use of its premises for filming of the classroom scenes. We couldn’t have done this without the assistance of the Lija-Balzan-Iklin Primary School, St Theresa’s College.
Involving children and children’s perspectives meant that ethics was one of the major challenges involved in the project. To solve this problem, we chose to collect data from third-country national adults who volunteered to share some of their life perspective and may have had experience in Maltese schools. We decided on this option for data collection as we believe that children may not yet have the necessary insights to communicate their feelings and emotions in a way that can be accurately represented in the VR experience. Communication was essential, and for this project we enlisted the support and help of the Foundation for Shelter and Support to Migrants to assist in interviewing and data collection.
Responses included participants falling on the floor, jumping in water, giggling and touching empty air, as they reached out to the children they were seeing in the virtual world.
This experience, currently in its beta phase, has been tested with around 300 people from all walks of life and ages ranging from six and 60. Among the initial test volunteers were those coming from migrant backgrounds. The experience was well received by all who tried it, and their varying reactions were enlightening.
During the stormy scene, where the users on top of a rough sea were surrounded by thunder and lightning, several of the younger individuals tried to jump into the water. A Czech woman let out a tiny scream as the floor gave way to ocean, and her hands floundered in the attempt to get her bearings in this new virtual world. In a VR classroom scene, users were giggling and touching empty air as they tried to touch the digital schoolchildren. Some individuals did not immediately realise the scale of the simulation. As it dawned on them that they could look all around as in real life, they began spinning to see behind them, and then in a full circle. Initial reactions such as uneasy excitement are all very normal responses to a first venture into a virtual world, though it is surprising how many returning users tend to be carried away by virtual reality. This suggests that the narrative element of this VR technology would be strong and immersive.
Our next step is to investigate the effects that such experiences have on participants’ real world behaviour. Do they become more empathic? In this phase, we plan to evaluate the human-computer interaction strengths and identify weaknesses when working with the teacher target group. This will be done by expanding our inter-faculty collaboration to include the Department for Inclusion and Access to Learning at the Faculty of Education.
Once proven effective, investment, research, and development of this technology could see it put to service in evoking empathy in many different social situations, from schools to the workplace and beyond. Furthermore, its power need not be limited to one particular society, culture or target group. VR could be the key to addressing multiculturalism’s issues of assimilation and integration, promoting understanding and acceptance between different groups of people, and ultimately maintaining civility and cohesion throughout society.