A new article from RHUL researchers, Clare Mutzenich, Professor Polly Dalton and Dr. Szonya Durant together with Dr Shaun Helman (TRL), discusses how to improve situation awareness in remote operators of automated vehicles
Virtual Reality Driving Simulator by Nan Palmero, licensed under CC BY 2.0
Problems can arise in automated vehicles (AVs) that would require a human operator to remotely assess and instrumentally correct or direct the automation. In this article, we highlight the need to address the human factor challenges that will arise in situations where self-driving cars are controlled by a remote human operator. ROs are likely to require longer exposure times to gain sufficient situation awareness (SA); they face latency and perception issues and may have difficulty achieving a sense of embodiment in the remote vehicle.
Situation awareness in driving contexts
Situation awareness refers to knowing what is going on around you. An RO who has been alerted by an AV will first need to acquire SA of the remote scene, yet there are significant variations between SA in normal driving contexts and SA as it might occur in remote driving operations. The task of developing SA from a remote location is likely to be made more difficult by the operator’s physical absence. SA is likely to develop differently for an RO compared with that of a driver who is present within the vehicle as they will be ‘out-of-the-loop’. Also, they are likely, until 5G is nationwide, to receive all visual and auditory information with at least some level of time-lag. For an RO to be able to drive the vehicle in real time the time-lag between the signal and response of the car must be minimal otherwise turning, accelerating, and braking will all be delayed.
An RO, not being physically present, is likely to have a limited sense of embodiment; they cannot feel the seat beneath them or the pull of the wheel in their hands, they are in no personal danger. This may result in a sense of detachment or reduced perception of risk (UNECE, 2020). A lack of embodiment may also have a negative effect on speed perception; without force feedback pushing you back into the seat or information from the tyre friction on the road, it is difficult to accurate judge how fast you are driving and to remain engaged in the driving task (Tang, Kurkowski, & Lienkamp, 2013).
Suggested approaches for improving remote SA
In remote environments, it may be beneficial to make an RO’s driving experience as realistic as possible whereby visual, auditory and haptic cues are provided as if they were in-situ. A naturalistic experience of driving even in remote contexts could be supplied by using a virtual display headset, allowing the operator to control their field of view just by moving
their head (Almeida, Patrao, Menezes, & Dias, 2014). This would avoid the need for multiple 2D monitors showing different camera feeds, which are likely to increase workload demand (Ricaud et al., 2017). Virtual Reality (VR) creates an illusion that the user is viewing the remote environment from an egocentric perspective, which seems likely to improve an ROs sense of embodiment in the scene. Their SA can also be enhanced by VR to provide a RO with a 360 view of the surrounding environment by combining LIDAR data with visual information from cameras and AR presented in a headset (Hosseini & Lienkamp, 2016).
However, there are motion sickness issues with VR which develop as the vestibular input does not match the visual motion experience of the user (Hosseini & Lienkamp, 2016). Current technological developments under design to address motion sickness include presenting a view of a fake steering wheel or the operator wearing VR gloves which show the drivers hands in the VR environment which aligns visual and motion cues (Manus, 2020). It is important to reiterate that many of these suggestions for improving SA in ROs involve presenting additional information. Their benefits must therefore be carefully weighed against the additional workload that they will impose. Future empirical research will be essential in exploring these issues to develop user interfaces that pull together concepts from research in other, related areas and applies them to the specific case of ROs of AVs. This work is currently underway in the Attention laboratory at Royal Holloway.
A longer version of this article is available fully open access in the journal Cognitive Research: Mutzenich, C., Durant, S., Helman, S. Dalton, P. Updating our understanding of situation awareness in relation to remote operators of autonomous vehicles. Cogn. Research 6, 9 (2021). https://doi.org/10.1186/s41235-021-00271-8
