Mental health is a rising concern that is taking its toll in the workplace and student environments alike. A team of animators are creating a VR experience to alleviate the associated pressures, and require my assistance for their audio assets. Follow my creative process as I explore the usefulness of sound as a function of mindfulness...
Mental Health Australia (MHA) is the peak, national non-government organisation representing and promoting the interests of the Australian mental health sector, and are committed to achieving better mental health for all Australians. It was established in 1997 as the first independent peak body in Australia to represent the full spectrum of mental health stakeholders and issues (MHA, 2019). Amazingly, in 2016-2017, it was reported that $9.1 billion was spent on mental health in Australia. That is a staggering fact to contemplate. Then, in 2017-2018, 4.2 million people received mental health-related prescriptions (AIHW, 2019). In response, both Mental Health Australia and the Australian Institute of Health and Welfare (AIHW) suggest mindful practices could contribute to improving people's headspace, and assist in mitigating this modern-day crisis.
Headspace.org suggests that mindful practices include:
Taking mini breaks throughout the day
Going for a nature walk
Avoid doing too many things at once
The effects of small and frequent breaks in the workplace (and study environments) are also often linked to boosted productivity and creativeness (Randolph, 2016)(Mansi, 2019). Consequently, using VR technology in conjunction with meditative practices, Mindful Escapes will be a tool used to help students and/or employees to improve their headspace and boost their creativity in the workplace.
Click here for a full outline of the project scope as provided to me by the concept's developers; Jack Stedman, Spencer Gibson, & Cayden Finn. Working with fellow sonic sorcerer Jordan Bayne, we have been appointed the task of delivering appropriate audio assets for the various locations in the experience, before assisting the team to weave them in with their final product using Unreal Engine. Communicating via a Slack channel and sharing information, documents, and other files through Google Drive, we were quickly brought up to speed on the project. A contract was signed and we were ready to get down to business.
Initially Jordan and I met with the group and gained an understanding of their vision. The team were very open to suggestions from an audio perspective, and ultimately left us a great deal of creative freedom to explore. Jordan and I went away from the meeting, and after much brainstorming and a little research, decided that we would aim to incorporate a few different elements for each area in the VR experience:
Binaural: beats for optimum relaxation
Ambience: stereo track representative of the location
Music: Soothing synth following a calming chord progression
BINAURAL THEORY
Binaural beats: it's a terminology I had heard before, but never truly grasped the concept. The technique has been around for a while, but recently it is getting a lot of attention for its ability to lower stress levels, improve sleep, and improve cognitive performance. It is an interesting technology that harnesses the brain's responsiveness to sound to move the listener into a deeper state of relaxation (Breus, 2018).
It is a simple technique to produce, requiring only a signal generator and some panning. The goal is to expose your brain to two different frequencies at the same time, one in each ear. By creating two sine waves (obviously a saw-wave won't be very relaxing) a few frequencies apart and panning them hard left and right, the listener's brain perceives a single tone that is actually the difference between the two. It is suggested that your brain "tunes" to this new frequency when listening through headphones.
For example: if your left ear is receiving a 250Hz tone, with a 260Hz tone in your right ear, your brain will process and "absorb" a 10Hz tone. Although that is an extremely low-frequency soundwave, below our hearing capabilities, research suggests it will subtly affect your brain; more specifically it creates changes in the brain's degree of arousal.
To understand how binaural beats may help relaxation, mood, mental performance, and sleep, you need to know a little bit about brain waves and what they indicate about our state of consciousness, emotion, and mental activity. Brainwaves are created from the pulses of electrical activity our neurons exhibit as they 'communicate with each other' (Breus, 2018). There are several types of brainwaves (Smith, 2019):
Beta [15-20Hz]: Associated with high levels of alertness and arousal. These indicate focus and concentration, enabling the ability to make decisions and think analytically. At higher levels it begins to indicate anxiety.
Alpha [9-14Hz]: Associated with a state of wakeful relaxation, or in beginning stages of sleep. They indicate someone who is calm and relaxed. Often they are present during meditation.
Theta [5-8Hz]: Associated with deep relaxation and with some stages of light sleep (non-REM; interestingly, REM sleep is mostly composed of beta wave, similar to an alert, waking brain). Deep meditation produces theta waves, and it is considered useful for improved creativity.
Delta [1.5-4Hz]: Associated with deep, dreamless (non-REM) sleep. According to electroencephalogram (EEG) brain scan results, studies indicate that patients receiving delta pattern frequencies during sleep enter a deeper sleep.
It becomes obvious then, that the faster the brainwave pattern, the higher the frequency, and the greater your state of arousal. Consequently, to flip that, scientists are suggesting that the lower the frequency your brain is 'tuning' to, the deeper your state of relaxation (or sleep). Tuning your brain is the simplest explanation for a process called Entrainment. Essentially, when exposed to sound waves at certain frequencies, brainwave patterns adjust to align with those frequencies.
It seemed highly appropriate to include in Mindful Escapes, and fortunately, not very difficult. Since our purpose was to increase relaxation, and boost creativity, it was decided that Theta waves would be most appropriate.
AMBIENCE
At a week 6 update meeting with the team, I got to experience an apparently raw version of Mindful Escapes for the first time. The basic structure of the space had been created, and I was able to do a walkthrough. The scale of the virtual space surprised me, I ended up getting lost, and in the end looked around for almost half an hour. In my adventures I made some mental notes of each of the areas. There was a:
Forest
Waterfall
Hidden cave behind the waterfall (easter egg)
Stream running from the waterfall to the edge of the map
Separate island with another forest-like space
Another island with an open space and a 'zen temple'
Flower field
Stepping stones leading vertically up to a raised (apparently floating) rock in the sky
Jordan agreed to hire out a Zoom H6 field recorder and capture some real-life stereo ambience to mix in with the binaural beats. We then brainstormed potential locations and landed on an idea to head to the Sunshine Coast. First stop was Buderim Falls, where we planned to get ambience for the waterfall, stream, cave, and one of the forests.
Between us we captured 70 different sound sources and locations, aiming to record a minimum of 2-3 minutes at each. This would allow us enough time to select at minimum the clearest minute, and loop it so that if the end-user remains in one location for five minutes they will more-than-likely not recognise the looped playback. Ideally, 5 minutes of continuous ambience for each space would be perfect, though we were somewhat limited with visitors and tourists continuously walking past. I also noticed the insane number of mosquitos in some areas, that would fly past the mic, or even land on it. I can only imagine how un-relaxing that would be during a mindful escape.
Perhaps my favourite piece of ingenuity for the day was coming up with ideas to organically capture a waterfall for use in the cave. Ultimately, it would be possible to roll off the high-end by placing a LPF in post-production, but I would rather play with capturing the real thing. Consequently, Jordan found a little hole near the waterfall, and aimed the X-Y configuration both in and out of it to see what results would yield. Meanwhile, I attempted to capture it by putting the H6 inside my bag and resting it against the rock wall beside/behind the waterfall:
Next up we directed ourselves to Mt. Coot-Tha and grabbed a half hour recording in two separate locations to be used for forest ambience. We also brought with us a 3Dio Freespace Pro II Binaural Microphone, to experiment with possibilities of realism. This required to XLR cables, one for each "ear", to provide a more realistic stereo recording.
During these sessions we kept an eye out for an appropriate location to record some ambience for the open spaces, but none seemed to match the serenity of what we imagined in the VR experience. All had either animals, bugs, traffic nearby, planes flying overhead, other extraneous noises. Consequently, after some discussion, we decided to take the H6 out again at a later date and hunt for a space far away from everything. We also decided that due to the other-worldly nature of the floating rock zen space, it might be more appropriate to fabricate ambience by combining a series of breeze or wind WAV files I already had in my possession and applying appropriate EQ.
In the end however, our deadline was running tight, and instead we went through the tedious process of finding sections of each recording that could be cut and crossfaded together to provide the 5 minute length piece of ambient audio we were aiming for. This process also included layering different environments on top of each other, to mask the sometimes obvious crossfades.
MUSIC
Jordan had access to a very useful plugin which we utilised for designing unique, calming synth chords and pad sounds to mix in with the binaural beats and the ambient noise. After a lot of trial and error, we came up with several relaxing sounds and utilised our creative side to compose some chords and notes that suited what we had so far. This was a very organic process, and ultimately captured a moment in time for us at peace in the studio.
A tribal drum was also incorporated for inclusion, so that when a participant decided to climb the treehouse in the simulation, they would be incorporated for added effect. We had to make sure that the notes were in key, to ensure a subtle fade in that blended with the remainder of our pieces.
Similarly, we had to ensure all of the different area's assets were in key, so that when crossing between different locations on the map, they would blend appropriately.
Have a listen to the completed area asset for the main forest where the participant spawns below (the background image is also a screenshot of the space):
INTEGRATION
Unreal Engine was the software that the team was working with to integrate the audio assets with the map, and it was a very rewarding experience to sit with the team and understand in better detail how it all worked. Essentially I learned that drawing out guidelines for each area enabled a particular WAV file to be assigned to that area. Included in the program was the option to fade in, which we decided could be set to 5 seconds for a subtle blend. The areas were set not only in the X-Y dimension, but also vertically, so that we could include assets such as the tribal drums for the treehouses, and a change in wind tone as the player ascends a rock formation leading to a meditative zone above a flower field. Once again this would be faded in for smoothness.
The first issue we encountered however, was that there was no option in Unreal Engine to crossfade between loops of the sound file, as we were originally informed. Consequently, it was back to the drawing board to update our assets so that they were perfect loops, and to ensure no pops were heard when they did so. After much failed research, I obtained some assistance from another audio practitioner, and the solution was actually quite simple. By cutting the file in the middle somewhere appropriate, we were able to bring the first half to follow the second half, ensuring that the end of the track and the beginning of the track would perfectly follow on from each other. It was important not to have a fade in or out, as this would be obvious each loop. Then it was merely a matter of finding an appropriate location to crossfade the new 'middle' of the track, blending it together as a cohesive whole. On a couple of occasions this involved me layering yet another sound and adding another synthesized note during the crossfade, to blend it in subtly.
After this, it was a matter of offering suggestions from an audio perspective to the integration experts themselves, such as minimising the level of the waterfall sample, due to its loud nature, so that it seemed in realistic balance to the rest. This was achieved using the Volume Adjustment parameter shown in the top image. I also offered the suggestion of ensuring that the waterfall sound would have a low-pass filter steadily activated as the player enters the cave behind the waterfall. It was actually quite interesting setting up the waterfall, as this was a sound within the 'island' which already had a location sound. Here the team assigned the WAV file to a particular location and set the roll-off distance appropriately, so that it would gradually increase in volume as you approached it. As seen in the above images, this rolloff could be set not only to a distance, but to a particular Attenuation Shape. For the waterfall we decided a cone was appropriate, angling it away from the waterfall, and away from the cave sitting behind.
Similarly, setting the sound of the stream to the bridge crossing nearby required some adjustments, to make it as realistic as possible. This required adjusting the distance from the bridge that the stream could be heard using a spherical shape. We also had to pay particular attention to the audible 'direction' of the stream, assigning it appropriately so that it would appear natural as the participant tilts or rotates their head. I noticed during the integration process that there was an option for this to be binaurally assessed, but apparently an extra "plug-in" (to use an audio term) was required for this processing. Instead the WAV file was panned into the left and right ears and the virtual head was turned.
Stay Tuned!
- TA
REFERENCES
Australian Institute of Health and Welfare. (2019). Mental health services: In brief 2019 [PDF]. Retrieved from https://www.aihw.gov.au/getmedia/f7395726-55e6-4e0a-9c1c-01f3ab67c193/aihw-hse-228-in-brief.pdf.aspx?inline=true
Australian Institute of Health and Welfare. (2019). Mental Health Services in Australia. Retrieved from https://www.aihw.gov.au/reports/mental-health-services/mental-health-services-in-australia/report-contents/summary-of-mental-health-services-in-australia
Breus, M. (2018) How Can Binaural Beats Help You Sleep Better?. Retrieved from https://www.psychologytoday.com/au/blog/sleep-newzzz/201810/how-can-binaural-beats-help-you-sleep-better
Mansi, M. (2019). Meditation and Workplace Creativity: A Potential Relationship (Ph.D). RMIT.
Mental Health Australia. (2019). About Us. Retrieved from https://mhaustralia.org/about-us
Randolph, S. (2016). The Importance of Employee Breaks. Workplace Health & Safety, 64(7), 344-344. doi: 10.1177/2165079916653416
Science Direct. (2019). Entrainment. Retrieved from https://www.sciencedirect.com/topics/psychology/entrainment
Smith, L. (2019). Binaural beats therapy. Retrieved from https://www.medicalnewstoday.com/articles/320019.php
Comments