Hybrid 3-D reproduction methods for acoustic virtual environments

Key Info

Basic Information

Funding Program:
Research Area:
Acoustic Virtual Reality


The acoustic environment, like noise etc., has a big influence on our performance in school, work and everyday life. To investigate and evaluate this influence in laboratory environments, different acoustics scenes have to be presented as realistic as possible. These scenes have to react to the user’s behavior (e.g. change of the user’s position, raise of voice) in real-time, allowing a natural interaction between user and environment. Nowadays, real-time filter exchange and optical tracking systems provide the required processing for such a system. Scientists from psychology or occupational medicine should be able to use such system for investigation, yet they are using offline rendered and non-interactive stimuli. This excludes an acoustically moderated investigation of people in their natural behavior (like learning, working, reading, communicating, relaxing, etc.). It is expected to gain greater insight into influence of noise on people by using interactive listening tests and overcoming the issues of the classical approach. The development of modern visual presentation methods in laboratory environments is more and more substituted by 3-D technology like those that can be found in cinema, home cinema or virtual reality (VR) applications like head-mounted displays (HMD) and CAVE system (Cave with Automated Virtual Environment). Especially the VR systems strongly depend on perceiving immersion. This requires, besides visual presentation, a spatial perception of the acoustic scene. As body-attached devices distract the user from a perception of immersion a loudspeaker-based reproduction is preferred instead of a headphone-based one. Known techniques are the reproduction of binaural signals using crosstalk-cancellation, physical based techniques like wave field synthesis (WFS) and vector-base amplitude panning, which uses the psychoacoustics effect of perceiving phantom sources. Another technique is Ambisonics, which in its original formulation reproduces a sound field, but, depending on its decoding, is also based on psychoacoustics effects. Each reproduction method has different strengths in presenting certain aspects of a reproduction. To present a sound source in a VR system, the acoustic characteristics of the virtual environment (e.g. classroom, work environment) have to be considered by providing room impulse responses (RIR). These can be calculated by using modern approaches of room acoustic simulation based on a combined approach of Ray Tracing and Image Source approach to synthesize a realistic reverberation field. The perception of different time slots of a RIR differs. Direct sound and early reflection contribute to localization performance and provide information needed to estimate the size and loudness of a source. Later arriving reflections cannot be perceived separately and provide a perception of auditory spaciousness. The listener can estimate the room size and some of its characteristics using this information. These are relevant components of a room simulation which have to be simulated and reproduced individually according to their perception, computational requirements and update rate for a real-time system. This project investigates the effects of combining the individual parts of the rendering with different 3-D audio reproduction methods to satisfy the different aspects of perception. E.g. CTC systems, known for providing a good localization performance, can be used to present the early part of a RIR while Ambisonics with its spacious, immersive reproduction is more suitable to present the later part of a RIR, even though it is less detailed in terms of localization. To ensure in inaudible transition between different reproduction techniques loudness and latency adaption have to be developed. Additionally to gaining simulation and measurement data subjective perception is a main factor to evaluate the overall performance of such a system. To avoid ambiguity the Spatial Auditory Quality Inventory (SAQI) will be used. The influence of the reproduction room will be investigated to adapt the systems to non-ideal environments. Finally the system will be evaluated on the basis of experiment evaluating the influence of noise on the subjective ability to concentrate and perform.