As I want to use XR (extended reality includes AR, VR & MR) for my master thesis and since the current topic is prosopagnosia and I want to influence their lives for the better, I thought it would be useful to research if and how much VR or MR can influence our emotions. Actually there exist a few various studies on the subject by now and I looked into a few of them. But first it is important to clarify the terms mood and emotion because the two are not the same. Mood does not have to have a specific cause, but is long-term and has low intensity where emotions are triggered by specific things, are short-lived and much more intense.
Emotions can be differently measured or recognized because each person responds differently based on their lifestyle and culture. However, the main characteristics to be able to measure emotions are in most cases subjective evaluation and perception, actions, facial or vocal expressions, heart rate, skin reactions and a few others. This is where user experience comes into play, especially in VR. In order to ensure a good UX some areas need to be included. This includes the feeling of presence within the environment or immersion, the system’s ability to interact with user input, and the user’s involvement in the virtual environment. It can be both passive and active. However, the user must always be a part of the environment or able to do something with it.
But back to the studies.
The Virtual Counselling Environment
The first study deals with the extent to which virtual people can have an influence on the emotions of the user.
To investigate this, a computer-generated counselling environment was created. Furnishings were oriented on a normal therapy room, which means sofas, paintings, books, a table and a chair on which the user can sit down. The user should feel as if he is in this virtual world and sitting on a real chair. The patient is a virtual human called ‘Justin’. The therapist is the user wearing the headset. So a training environment has been set up where virtual patients can be treated by novice clinicians and those therapists can practice challenging situations. To make it more realistic, it is important that the complete process is included in the virtual environment. This includes coming in, reactions, statements and leaving the room. The virtual human has basic movements, human facial features and a human voice. The whole system works only if speech recognition is built in, because only then a positive response is possible and the user is not frustrated.
Surprisingly, the virtual human could simulate human emotions so well that the user could recognize them.
Study results
- VR has a strong influence on the emotions of the user
- A natural human computer interaction has an impact on the user
- Speech recognition is important, because only then it seems more natural and the emotional level is much stronger
- The use by controllers has less influence on the emotional response
Virtual Park Scenarios
The next study is about whether a scenario of a fictional VR park can evoke joy, sadness, boredom, anger and anxiety.
The Velten Mood Induction Procedure method was chosen, which means that under controlled conditions images, film clips or music are shown which evoke temporary emotional states. This method tests effects of emotional states, memory or change in information processing.
In addition, electrodermal activity (EDA) was applied for this study.
„Electrodermal activity (EDA) refers to the variation of the electrical properties of the skin in response to sweat secretion. By applying a low constant voltage, the change in skin conductance (SC) can be measured non-invasively (Fowles et al., 1981).“[3]
To be able to evoke emotions, the feeling of presence must be considered above all. Presence is often defined as the sense of being there in a Virtual Environment and because of this feeling, an emotion can occur. So, a greater presence would create a much greater reaction and the user would behave, feel or think as if he or she were in a similar situation, because it does not feel as if what he or she is experiencing is only from technology. Frame Rate actually affects the feeling of being present as well.
For the study, 120 students were selected to first of all relax for 5 minutes and then explore the park in the VE environment in first-person view for 5 minutes.
- Joy: sunny, calm, quit, daytime scenario with chirping birds, non-playing characters
- Anger: constant, unnerving sound of heavy construction work
- Boredom: dull and boring scenario with sound of distant traffic
- Anxiety: gloomy night-time, dim light, owl sound, non-playing characters silhouettes
- Sadness: grey, rainy day, rain dripping sound, non-playing characters carrying umbrellas and walking hastily
Study results
- Joy, anger, anxiety and boredom have triggered the planned emotional states
- The Park vor sadness failed because boredom and sadness are often accompanied by one another
- Most negative emotions are accompanied to a small degree by other negative emotions
Feelings of fear and anger in VR
The purpose of this study was to find out to what extent the user’s emotions depend on the medium used, in this case VR and computer screen. To be even more specific, fear and anger were selected and compared with each other. Because especially from these two negative-valance emotions it is possible to see clear differences. The willingness to take risks increases with anger, while it decreases with fear. Both emotions have in common that they are negative and that we are strongly aroused by both. The differences are not only the willingness to take risks but also the control over the body and the awareness of the emotion.
To trigger anger, a short film clip from the movie ‘My Bodyguard’ was chosen, in which a group of teenagers bully weaker people. For the emotion fear, the subjects had to play the VR horror game ‘Play with me’. Both emotions were shown to the subjects on a computer screen as well as in VR.
Study results
- Decisions can change depending on the emotional state in which the person is at the time
- People take more risk when they are angry
- VR has a much higher impact on decision-making behavior compared to the use of a computer
Emotional reactions are intensified much more by VR
To get to the bottom of this, the GAPED picture database was used. GAPED stands for Geneva Affective Picture Database. In this database 730 pictures were collected, which are supposed to trigger different states of mind. These included negatively affected pictures such as snakes or pictures about mistreatment (injured animals or butchering), positive ones such as puppies, nature scenes or laughing faces, and neutral pictures such as non-living objects (bicycle spokes).
Participants had to answer three questions and rate their emotional state on a scale after viewing 24 selected images for 4 seconds.
Study results
- For all images, the arousal was much higher with VR than with a computer screen
- Photos with snakes or spiders where associated with phobias so the arousal was even higher
- By diving into the world and blocking out the reality, the much higher arousal of the user could be explained
- The attention span could be much higher when virtual reality is used
Sources:
- The effect of two different types of human-computer interactions on user’s emotion in virtual counseling environment, Tu Ziqi, Weng Dongdong, Cheng Dewen, Shen Ruiying, Fang,Hui, Bao Yihua (10.2019), https://booksc.org/book/80763010/fcb00e
- Is virtual reality emotionally arousing? Investigating five emotion inducing virtual park scenarios, Anna Felnhofera, Oswald D.Kothgassnera, Mareike Schmidt, Anna-Katharina Heinzle, Leon Beutl, Helmut HlavacsbIls, Kryspin-Exner (10.2020), https://www.sciencedirect.com/science/article/abs/pii/S1071581915000981
- A continuous measure of phasic electrodermal activity, Mathias Benedek, Christian Kaernbach (30.06.2010), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2892750/
- The Feeling is Real: Emotion Elicitation in Virtual Reality, Sahinya Susindar, Mahnoosh Sadeghi, Lea Huntington, Andrew Singer, Thomas K. Ferris (11.2019), https://journals.sagepub.com/doi/pdf/10.1177/1071181319631509
- Can Virtual Reality Increase Emotional Responses (Arousal and Valence)? A Pilot Study, Sergio Estupiñán, Francisco Rebelo, Paulo Noriega, Carlos Ferreira, Emília Duarte (2014), https://link.springer.com/chapter/10.1007/978-3-319-07626-3_51
- Physiological Measures of Presence in Stressful Virtual Environments, Frederick Phillips Brooks Jr., Michael Meehan, Brent Insko, Mary C. Whitton (08.2002), https://www.researchgate.net/publication/2529722_Physiological_Measures_of_Presence_in_Stressful_Virtual_Environments
