Acoustic Virtual Reality
The interaction of human beings with their environment happens with a multitude of sensory stimuli. Besides the optical, haptic and tactile perception acoustics provides important information about the environment. One gains an impression of the environment, about external events and in particular feedback on one's own actions (proprioception = perceiving oneself). To achieve an optimal immersion in a virtual simulated environment the simulated stimuli for all senses must be consistent. Sound propagates through structures and via the air, is reflected by objects as for example walls in a room, and eventually reaches the place of the recipient. The human being then evaluates different properties of these sound cues. These are amongst others the volume, the quality of sound, the incidence direction of direct sound and the perceived distance. In addition, conclusions with regard to the environment in which the person finds himself can be drawn, in a room for example with regard to its size and shape. This happens because the fine structure of the cue response between sender and recipient which consists of superimpositions of the direct sound and its reflections is analyzed in a time-related manner. So a system was identified consisting of three series-connected components, the generation of sound, its propagation up to the recipient and finally the perception processes of the recipient can which are researched in the specialized area of psycho acoustics. Psycho acoustics defines in particular the quality requirements which have to be met by the components. In addition, for a VR system sound reproduction has to be considered in order to render the acoustic cues in a suitable manner to the ear.
The generation of auditory stimuli is called auralization or "acoustic rendering". This should be understood in the same way as "visualization" The basis for auralization is the calculation of acoustic cue responses by means of suitable simulation technology. Nowadays there are a number of methods in numerics, geometrical as well as wave-based procedures. The challenge for research and development with regard to auralization and therefore for virtual acoustics depends on the requirements concerning real time processing. The generation of acoustic scenarios (rendering) and their reproduction must be effected as quickly as possible, so that there are no delays (latencies) or other artefacts.
Parallel to the progress made in spatial acoustic simulation technology complex models were developed for simulating vibro-acoustic issues as in case of sound absorption in buildings and with regard to transfer path analysis and synthesis in the automotive industry. The diversity of applications of virtual acoustics is very wide and goes far beyond listening to music or the quality assessment of concert halls. Sound design and researching sounds and noise in building acoustics, vehicle acoustics, noise control and psycho acoustics are further interesting application areas.
Video - Examples
In the following, example videos regarding the topic are presented. The respective audio samples are rendered using our software Virtual Acoustics (VA) . For a full list of our videos, it is referred to our YouTube channel .
Offline visualization and auralization using physics-based room acoustic simulation
Virtual Acoustics (VA) and the aixCAVE
Virtual Acoustics (VA) and Unity with HMD
The following videos have been created by Maurice Andreas during his Bachelor thesis on physics-based real-time auralization with the VR and gaming environment Unity. The material is licensed under Creative Commons Version 2.0.
Interactive park situation demonstrating the use of Unity and Virtual Acoustics (VA) for an audio-visual user study. The fictive task is to locate the chirping Robin in one of the trees with the pointing device, while audio source and (hard to find) visual prop may be dislocated. Hypothesis would be, which modality is dominating.
For an increased immersive experience, the binaural signal is not played back with headphones but with a transaural loudspeaker setup using dynamic multi-channel cross-talk cancellation.
Situation in a virtual room with a band playing music and some small-scale buildings on posts to demonstrate interaction with acoustics. By sticking the head into the models, the reverberation time is changed accordingly using a binaural artificial reverberation simulation. For movement, portal navigation is available using the hand controller. To interact with the auralization rendering modules, a menu called SoundPallette has been implemented that can change the auralization modes (e.g. DirectSound, EarlyReflections, DiffuseDecay, DopplerShifts, SpreadingLoss and others). Note: menu interaction demonstration by the user is missing in this clip.
For an increased immersive experience, the binaural signal is not played back with headphones but with a transaural loudspeaker setup using dynamic multi-channel cross-talk cancellation.
Building Acoustics Auralization
Building acoustics auralization technology extended towards real-time interactive audio-visual technology, i.e. VR technology, to introduce more realism and, hence, more contextual features into future psychoacoustic experiments. Having achieved progress in VR building acoustics auralization a new test paradigm is designed, which help to better analyse and interpret the noise impact in building situations depending on the actual activity such as living, working, learning, and rest.
Virtual Reality Framework: Building acoustics auralization with contextual and interactive features
This video demonstrates a listening experiment for outdoor moving noise sources in a virtual reality framework and explores the perceptual localization capabilities of human in real as well as in virtual environments. This experiment is an example of exploring outdoor moving sound sources noise effects under building acoustics condition where a binaural sound stimuli of moving sound source is presented and the task is to judge whether the passing-by source is moving from right to left or from left to right. This way the user feels fully immersed in the situation and perceive sound insulation of the dwellings from where the evaluation of building performance can be performed.
Offline auralization of aircraft flyover using Atmospheric Ray Tracing (ART) in Virtual Acoustics (VA)
Virtual aircraft flyover after take-off using physics-based simulation of atmospheric sound propagation and binaural reproduction. The material is licensed under Creative Commons Version 4.0 (CC BY 4.0). To get a proper spatial impression, using headphones is advised..
The Atmospheric Ray Tracing (ART) algorithm considers advection and refraction due to wind and inhomogeneity of the atmosphere. It finds the curved propagation paths connecting the aircraft (source) with the listener (receiver) as shown in the video. Based on these paths, acoustic effects such as air attenuation, spreading loss, propagation delay and Doppler-shift are modelled and applied to the audio signal.
Auralization of psychoacoustically optimized aircraft
This video demonstrates how Virtual Acoustics (VA) can be used to compare the sound of a today's state-of-the-art aircraft („Reference“) with a psychoacoustically optimized aircraft design („Engine over wing“).
The noise of aircraft is composed of different sound contributions coming from the engine and the turbulent flow around the aircraft. The computational modeling of these aeroacoustic sound sources concerning generation and radiation in combination with the simulation of its propagation through the atmosphere makes it possible to listen how a virtual prototype sounds like.
In the search for quieter aircraft concepts to reduce their environmental impact, one possibility is to position the engine above the wings. This is intended to reflect the so-called fan noise on the wing surface and deflect to the upper hemisphere.
Auralization of virtual aircrafts in real scenes
This video shows how Virtual Acoustics (VA) can be used to experience the noise emissions of an aircraft in the context of an existing situation.
The sound field emitted by aircraft was embedded in an outdoor scene at the institute building. The real environment was first recorded with a 3D camera and an HOA microphone, then transferred to a virtual environment. The background sound remains constant in the video, with the virtual aircraft and its sound immission being mounted into the scene.