Ambisonic Sound Design for Theatre with Virtual Reality Demonstration - A Case Study (original) (raw)
Related papers
Towards Generating Ambisonics Using Audio-visual Cue for Virtual Reality
ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019
Ambisonics i.e., a full-sphere surround sound, is quintessential with 360 • visual content to provide a realistic virtual reality (VR) experience. While 360 • visual content capture gained a tremendous boost recently, the estimation of corresponding spatial sound is still challenging due to the required sound-field microphones or information about the sound-source locations. In this paper, we introduce a novel problem of generating Ambisonics in 360 • videos using the audiovisual cue. With this aim, firstly, a novel 360 • audiovisual video dataset of 265 videos is introduced with annotated sound-source locations. Secondly, a pipeline is designed for an automatic Ambisonic estimation problem. Benefiting from the deep learning based audiovisual feature-embedding and prediction modules, our pipeline estimates the 3D sound-source locations and further use such locations to encode to the B-format. To benchmark our dataset and pipeline, we additionally propose evaluation criteria to investigate the performance using different 360 • input representations. Our results demonstrate the efficacy of the proposed pipeline and open up a new area of research in 360 • audiovisual analysis for future investigations.
The immersive cinematic sound space: Audience perspectives
2016
The changing materiality of moving images and picture sources is a crucial aspect of the space in which screen stories are told. Technologies that capture and present moving images are responsible for our understanding of what we see as audiences; and as makers, how we create reality on screen. The immersive cinematic experience is influenced by many factors, including, but not limited to, the production, narrative, technology and screening environment of the film. Since the introduction of digital cinema, directors have provided increased immersion through 3-D imagery and immersive 3-D sound technologies. Despite excitement in the production industry, new immersive technologies are being introduced into the cinema exhibition space with caution. Cinema managers are not rushing to upgrade to new technologies despite increased production of 3-D and immersive sound films. The cost of upgrading equipment, and with many new technologies not yet standardised, makes any technological inves...
Designs
In the last few years, the immersive theater has become a new trend for modern performances. Venues increasingly utilize widely available computer technologies, such as virtual/augmented reality and spatial sound, to help facilitate the realization of different ideas. Motivated by this current trend, a prototype platform has been developed that enables the design and implementation of an augmented theatrical experience based on spatial sound immersion. This paper describes the implementation of the platform and, through several use case scenarios, its evaluation. The paper concludes with a discussion of the results and offers some thoughts on future developments.
Designing Spatial Sound: Adapting Contemporary Screen Sound Design Practices for Virtual Reality
SMPTE17: Embracing Connective Media, 2015
The authors are solely responsible for the content of this technical presentation. The technical presentation does not necessarily reflect the official position of the Society of Motion Picture and Television Engineers (SMPTE), and its printing and distribution does not constitute an endorsement of views which may be expressed. This technical presentation is subject to a formal peer-review process by the SMPTE Board of Editors, upon completion of the conference.
2019
This paper analyzes the limitations of binaural sound that is reproduced with headphones based on Ambisonics for Virtual Reality (VR) audio. VR audio can be provided with binaural sound that compensates head rotation of a listener. Ambisonics is widely used for recording and reproducing ambient sound fields around a listener in VR audio, and the First order Ambisonics (FOA) is still being used for VR audio because of its simplicity. However, the maximum frequencies with this order is too low to perfectly reproduce ear signals, and thus the binaural reproduction has inherent limitations in terms of spectrum and sound localization. This paper investigates these limitations by comparing the signals arrived at ear positions in the reference field and the reproduced field. An incidence wave is defined as a reference field, and reproduced over virtual loudspeakers. Frequency responses, inter-aural level differences, and inter-aural phase differences are compared. The results show, above t...
Novel approaches to production and post-production of immersive VR/360 audio-visual experiences
2017
Key aspirations for VR and 360 Audio-Visual productions are immersion and presence. Yet, these are impossible to achieve without close spatial synchronisation between fields of vision and the soundscape. Audio has been recognised by many VR/360 practitioners as the most difficult aspect to control when included within the user experience. In an attempt to overcome this there is much current research into object-based and scene-based audio paradigms that deploy complex rendering and spatialisation techniques via game-engines such as Unity and proprietary software. There has been some success with these approaches, most notably BBC R&D’s landmark VR experience, The Turning Forest. However, many VR/360 productions may not require such complex workflows in order to render an engaging user experience, particularly if content delivery is linear. Scene-based audio using Ambisonic principles coupled with simple head-tracking technology offers a more manageable workflow that is ideal for sho...
EBU Technical Review Paper, 2019
Scene Based Audio is a set of technologies for 3D audio that is based on Higher Order Ambisonics. HOA is a technology that allows for accurate capturing, efficient delivery, and compelling reproduction of 3D audio sound fields on any device, such as headphones, arbitrary loudspeaker configurations, or soundbars. We introduce SBA and we describe the workflows for production, transport and reproduction of 3D audio using HOA. The efficient transport of HOA is made possible by state-of-the-art compression technologies contained in the MPEG-H Audio standard. We discuss how SBA and HOA can be used to successfully implement Next Generation Audio systems, and to deliver any combination of TV, VR, and 360° video experiences using a single audio workflow.
Ambisonic sound in virtual environments and applications for blind people
1998
To date there has been much more effort directed to generating credible presentations of virtual worlds in a visual medium than in reproducing corresponding or even self contained worlds with synthesised or recorded sound. There is thus today a disparity in the relative fidelity of these 2 modes in most VR systems. While much work has been done in hi-fidelity and dimensional sound reproduction in psycho-acoustic research and applications for audiophiles this has rarely been taken onboard by the VR community. This paper describes work ongoing to apply Ambisonic techniques to the generation of audio virtual worlds and environments. Firstly Ambisonics is briefly outlined then principles behind its implementation in this context described. The design of the implementations to date is described and the results of trials discussed. The strengths and limitations of the approach are discussed in the light of this practical experience. There is a range of applications envisaged for this tech...
Designing a virtual theatrical listening space
2000
The research at CRM has been oriented in the last years, at the study of “space” as new parameter for music composition and hearing processes. This allows to conceive concerts, and multimedia events in which the “acoustic space” is being designed coherently to the artistic criteria of the pieces presented. The paper will illustrate some artistic projects realized in the last two years. In particular, production of multimedia and technological events for which “acoustic space” has been designed coherently to the artistic criteria proposed.