Human Behavioral Control in Virtual Worlds

The extension of reality through digital worlds – from augmented reality (AR) to virtual reality (VR) – is becoming increasingly important during digital transformation and offers new learning opportunities in numerous application areas. The use of VR training tools promises great added value, especially when it enables repeated and intensive experience in situations that would be associated with high costs or risks in the real world. In a professional context, such opportunities arise, for example, in the psychological context using VR in psychotherapeutic confrontation or desensitization therapies, in sports through VR simulations to prepare for the next opponent, and in medical training through the simulation of complex emergency scenarios in VR simulations. What all these applications have in common is that the value of VR technologies increases if the experiences gained in virtual space can be effectively transferred to the real world – especially in view of the typically resource-intensive implementation. However, the conditions for maximum transfer are still largely unknown. To make the most of VR's potential, an in-depth, problem-oriented understanding of the sensorimotor mechanisms that may limit transfer is of central importance.

In the MidT profiling project "Human Behavioral Control in Virtual Worlds", VR transfer mechanisms are systematically investigated in a series of experimental studies at the Institute of Psychology (IfP, led by Prof. Dr. Matthias Ertl) and the Institute of Sports Science (ISPW, led by Dr. Stephan Zahno). Building on the basic research-oriented studies, interdisciplinary and integrative translational studies are planned with associated project partners from emergency medicine (Visel, head: Prof. Dr. Thomas Sauter).

At the ISPW, we have developed a VR gait coordination task to investigate VR transfer mechanisms. In this experimental task, inspired by Tango Argentino, the subject assumes the role of a "follower" and must coordinate forward and backward steps with a "leader", who is either virtual or physically present. As a measure of coordination, we compare the time differences between the peaks of the velocity and acceleration curves of the leader and the follower. This task allows us to investigate how subjects use different sensory information (visual, auditory, haptic) in interpersonal coordination and how this affects learning in VR and its transfer to the real world.

At the IfP, we investigate fundamental perceptual mechanisms in virtual environments that differ from the real world. The focus is on depth perception, potentially influencing the transfer of learning from goal-directed motion. The design of commercially available head-mounted displays (HMDs) decouples the natural interplay between accommodation and vergence. In contrast to the real world, the eyes in VR constantly focus on a virtual display (accommodation), while only the angular position of the eyes (vergence) adapts to the perceived distance to the object, which is based on artificially generated, disparate image information. These differences between the virtual and the real world may affect learning transfer, as we will investigate experimentally using motor tasks.

Kontakt

Dr. Stephan Zahno (Institute of Sport Science)

Prof. Dr. Matthias Ertl (Institute of Psychology)

This research project is supported by the Digitization Commission of the University of Bern (DigiK).