Joshua D Reiss - Academia.edu (original) (raw)

Papers by Joshua D Reiss

The main focus of this article is to explore and investigate the fundamental constraints that sho... more The main focus of this article is to explore and investigate the fundamental constraints that should be at the basis of algorithm development in intelligent audio production systems. Through mix analysis and grounded theory strategies, a best-practices framework on the craft of mixing is sought out. Findings, while not to be taken as dogmatic, give a clear indication of preferred implementation strategies, and show what still needs to be done to fully understand the technical choices that audio mixing has incorporated throughout its history.

Applied Acoustics, 2016

This paper presents an experiment where participants were asked to adjust,

Applied Sciences, 2016

Audio equalization is a vast and active research area. The extent of research means that one ofte... more Audio equalization is a vast and active research area. The extent of research means that one often cannot identify the preferred technique for a particular problem. This review paper bridges those gaps, systemically providing a deep understanding of the problems and approaches in audio equalization, their relative merits and applications. Digital signal processing techniques for modifying the spectral balance in audio signals and applications of these techniques are reviewed, ranging from classic equalizers to emerging designs based on new advances in signal processing and machine learning. Emphasis is placed on putting the range of approaches within a common mathematical and conceptual framework. The application areas discussed herein are diverse, and include well-defined, solvable problems of filter design subject to constraints, as well as newly emerging challenges that touch on problems in semantics, perception and human computer interaction. Case studies are given in order to illustrate key concepts and how they are applied in practice. We also recommend preferred signal processing approaches for important audio equalization problems. Finally, we discuss current challenges and the uncharted frontiers in this field. The source code for methods discussed in this paper is made available at https://code.soundsoftware.ac.uk/projects/allaboutaudioeq.

IFAC Proceedings Volumes, 2006

Sigma delta modulation is a popular form of A/D and D/A conversion. This nonlinear device exhibit... more Sigma delta modulation is a popular form of A/D and D/A conversion. This nonlinear device exhibits a high degree of complex nonlinear behaviour, including chaotic dynamics. One of the main unsolved problems in the theory of sigma delta modulation concerns the ability to analytically derive conditions for the boundedness of solutions of a high order sigma delta modulator (SDM). In this work, we describe how a sigma delta modulator may be rephrased within the context of systems theory. We present several theoretical results concerning bounded solutions of general high order SDMs, including necessary and sufficient conditions for the lack of a finite escape time, necessary conditions for bounded solutions based on the nature of the output sequences, and topological properties of the solutions, which are a precursor to the study of chaotic solutions of SDMs.

AIP Conference Proceedings, 2009

ABSTRACT A skew symmetric, fully conservative compact finite difference scheme for the compressib... more ABSTRACT A skew symmetric, fully conservative compact finite difference scheme for the compressible Navier-Stokes Equa­ tions is presented. The construction of this scheme builds on preserving structures of the continuous equations, in particular the skew symmetry of the derivative operator. This is done by utilising the Galerkin ansatz for the linear and the non-linear terms. This approach allows systematic construction of such fully conservative discretisations for different needs. We show how to use this freedom to create a numerical efficient compact high order scheme. The scheme is of 6th oder and has good resolution properties in wave number space even for the nonlinear case. Since the conservation of momentum within skew symmetric schemes is not generally guaranteed, we pay special attention to this point and derive an easy to use criterion for this property in our approach. Finally numerical examples are presented. Although not presented here we emphasise its usefulness for LES. Skew symmetric schemes are schemes which preserve the skew symmetry of differential operators in the discrete case, and thereby respect conservation properties. These schemes were first introduced by Feiereisen [2] and Tadmor [8]. While Feiersisen was interested on numerical simulations Tadmor concentrated on analytical aspects. Later it was used by [7, 3] and [10] among others. We use the term skew symmetric scheme for a scheme which preserves the correct skew symmetry or symmetry of the different terms. Skew symmetry refers here to the symmetry of an operator in the scalar product, in the continuous case given by (M, V) = J u{x)v{x)dx. An operator is said to be skew symmetric or skew adjoint if

Many digital sound archives still suffer from tremendous problems concerning access. Materials ar... more Many digital sound archives still suffer from tremendous problems concerning access. Materials are often in different formats, with related media in separate collections, and with non-standard, specialist, incomplete or even erroneous metadata. Thus, the end user is unable to discover the full value of the archived material. EASAIER addresses these issues with the development of an innovative remote access system

In this work the techniques of chaotic time series analysis are applied to music. The audio strea... more In this work the techniques of chaotic time series analysis are applied to music. The audio stream from musical recordings are treated as representing experimental data from a dynamical system. Several performance of well-known classical pieces are analysed using recurrence analysis, stationarity measures, information metrics, and other time series based approaches. The benefits of such analysis are reported.

A new approach for automatically equalizing an audio signal towards a target frequency spectrum i... more A new approach for automatically equalizing an audio signal towards a target frequency spectrum is presented. The algorithm is based on the Yule-Walker method and designs recursive IIR digital filters using Least-Squares fitting to any desired frequency response. The target equalization curve is obtained from the spectral distribution analysis of a large dataset of popular commercial recordings. A real-time C++ VST plug-in and an off-line Matlab implementation have been created. Straightforward objective evaluation is provided, where the output frequency spectra are compared against the target equalization curve and the ones produced by an alternative equalization method.

Music Information Retrieval may be perceived as part of the larger Multimedia Information Retriev... more Music Information Retrieval may be perceived as part of the larger Multimedia Information Retrieval research area. However, many researchers in Music Information Retrieval are unaware that the problems they deal with have analogous problems in image and video retrieval. Many issues concerning the creation of testbed digital libraries and effective benchmarking of information retrieval systems are common to all multimedia retrieval systems. We examine the approaches used in the image and video communities and show how they are applicable to testbed creation and information retrieval system evaluation when the media is music.

A framework is presented which addresses the issues related to the real-time implementation of sy... more A framework is presented which addresses the issues related to the real-time implementation of synchronised video and audio time-scale and pitch-scale modification algorithms. It allows for seamless real-time transition between continually varying, independent time-scale and pitch-scale parameters arising as a result of manual or automatic intervention. We illuminate the problems which arise in a real-time context as well as provide novel solutions to prevent artefacts, minimise latency, and improve synchronisation. The time and pitch scaling approach is based on a modified phase vocoder with optional phase locking and an integrated transient detector which enables high quality transient preservation in real-time. A novel method for audio/visual synchronisation was implemented in order to ensure no perceptible latency between audio and video while real-time time scaling and pitch shifting is applied. Evaluation results are reported which demonstrate both high audio quality and minimal synchronisation error.

Journal of the Audio Engineering Society, Jan 15, 2008

Page 1. J. Audio Eng. Soc., Vol. 56, No. 1/2, 2008 January/February 49 By Joshua D. Reiss, AES Me... more Page 1. J. Audio Eng. Soc., Vol. 56, No. 1/2, 2008 January/February 49 By Joshua D. Reiss, AES Member INTRODUCTION Sigma–delta modulation (SDM) is per-haps best understood by comparison with traditional pulse-code modulation (PCM). ...

We introduce the Open Multitrack Testbed, an online repository of multitrack audio, mixes or proc... more We introduce the Open Multitrack Testbed, an online repository of multitrack audio, mixes or processed versions thereof, and corresponding mix settings or process parameters such as DAW files. Multitrack audio is a much sought after resource for audio researchers, students and content producers, and while some online resources exist, few are large and reusable, and none allow querying audio fulfilling specific criteria. The test bed we present contains a semantic database of metadata corresponding with the songs and individual tracks, enabling users to retrieve all pop songs featuring an accordion, or all tracks recorded in reverberant spaces. The open character is made possible by requiring that the contributions, mainly from educational institutions and individuals, have a Creative Commons license.

The main focus of this article is to explore and investigate the fundamental constraints that sho... more The main focus of this article is to explore and investigate the fundamental constraints that should be at the basis of algorithm development in intelligent audio production systems. Through mix analysis and grounded theory strategies, a best-practices framework on the craft of mixing is sought out. Findings, while not to be taken as dogmatic, give a clear indication of preferred implementation strategies, and show what still needs to be done to fully understand the technical choices that audio mixing has incorporated throughout its history.

Applied Acoustics, 2016

This paper presents an experiment where participants were asked to adjust,

Applied Sciences, 2016

Audio equalization is a vast and active research area. The extent of research means that one ofte... more Audio equalization is a vast and active research area. The extent of research means that one often cannot identify the preferred technique for a particular problem. This review paper bridges those gaps, systemically providing a deep understanding of the problems and approaches in audio equalization, their relative merits and applications. Digital signal processing techniques for modifying the spectral balance in audio signals and applications of these techniques are reviewed, ranging from classic equalizers to emerging designs based on new advances in signal processing and machine learning. Emphasis is placed on putting the range of approaches within a common mathematical and conceptual framework. The application areas discussed herein are diverse, and include well-defined, solvable problems of filter design subject to constraints, as well as newly emerging challenges that touch on problems in semantics, perception and human computer interaction. Case studies are given in order to illustrate key concepts and how they are applied in practice. We also recommend preferred signal processing approaches for important audio equalization problems. Finally, we discuss current challenges and the uncharted frontiers in this field. The source code for methods discussed in this paper is made available at https://code.soundsoftware.ac.uk/projects/allaboutaudioeq.

IFAC Proceedings Volumes, 2006

Sigma delta modulation is a popular form of A/D and D/A conversion. This nonlinear device exhibit... more Sigma delta modulation is a popular form of A/D and D/A conversion. This nonlinear device exhibits a high degree of complex nonlinear behaviour, including chaotic dynamics. One of the main unsolved problems in the theory of sigma delta modulation concerns the ability to analytically derive conditions for the boundedness of solutions of a high order sigma delta modulator (SDM). In this work, we describe how a sigma delta modulator may be rephrased within the context of systems theory. We present several theoretical results concerning bounded solutions of general high order SDMs, including necessary and sufficient conditions for the lack of a finite escape time, necessary conditions for bounded solutions based on the nature of the output sequences, and topological properties of the solutions, which are a precursor to the study of chaotic solutions of SDMs.

AIP Conference Proceedings, 2009

ABSTRACT A skew symmetric, fully conservative compact finite difference scheme for the compressib... more ABSTRACT A skew symmetric, fully conservative compact finite difference scheme for the compressible Navier-Stokes Equa­ tions is presented. The construction of this scheme builds on preserving structures of the continuous equations, in particular the skew symmetry of the derivative operator. This is done by utilising the Galerkin ansatz for the linear and the non-linear terms. This approach allows systematic construction of such fully conservative discretisations for different needs. We show how to use this freedom to create a numerical efficient compact high order scheme. The scheme is of 6th oder and has good resolution properties in wave number space even for the nonlinear case. Since the conservation of momentum within skew symmetric schemes is not generally guaranteed, we pay special attention to this point and derive an easy to use criterion for this property in our approach. Finally numerical examples are presented. Although not presented here we emphasise its usefulness for LES. Skew symmetric schemes are schemes which preserve the skew symmetry of differential operators in the discrete case, and thereby respect conservation properties. These schemes were first introduced by Feiereisen [2] and Tadmor [8]. While Feiersisen was interested on numerical simulations Tadmor concentrated on analytical aspects. Later it was used by [7, 3] and [10] among others. We use the term skew symmetric scheme for a scheme which preserves the correct skew symmetry or symmetry of the different terms. Skew symmetry refers here to the symmetry of an operator in the scalar product, in the continuous case given by (M, V) = J u{x)v{x)dx. An operator is said to be skew symmetric or skew adjoint if

Many digital sound archives still suffer from tremendous problems concerning access. Materials ar... more Many digital sound archives still suffer from tremendous problems concerning access. Materials are often in different formats, with related media in separate collections, and with non-standard, specialist, incomplete or even erroneous metadata. Thus, the end user is unable to discover the full value of the archived material. EASAIER addresses these issues with the development of an innovative remote access system

In this work the techniques of chaotic time series analysis are applied to music. The audio strea... more In this work the techniques of chaotic time series analysis are applied to music. The audio stream from musical recordings are treated as representing experimental data from a dynamical system. Several performance of well-known classical pieces are analysed using recurrence analysis, stationarity measures, information metrics, and other time series based approaches. The benefits of such analysis are reported.

A new approach for automatically equalizing an audio signal towards a target frequency spectrum i... more A new approach for automatically equalizing an audio signal towards a target frequency spectrum is presented. The algorithm is based on the Yule-Walker method and designs recursive IIR digital filters using Least-Squares fitting to any desired frequency response. The target equalization curve is obtained from the spectral distribution analysis of a large dataset of popular commercial recordings. A real-time C++ VST plug-in and an off-line Matlab implementation have been created. Straightforward objective evaluation is provided, where the output frequency spectra are compared against the target equalization curve and the ones produced by an alternative equalization method.

Music Information Retrieval may be perceived as part of the larger Multimedia Information Retriev... more Music Information Retrieval may be perceived as part of the larger Multimedia Information Retrieval research area. However, many researchers in Music Information Retrieval are unaware that the problems they deal with have analogous problems in image and video retrieval. Many issues concerning the creation of testbed digital libraries and effective benchmarking of information retrieval systems are common to all multimedia retrieval systems. We examine the approaches used in the image and video communities and show how they are applicable to testbed creation and information retrieval system evaluation when the media is music.

A framework is presented which addresses the issues related to the real-time implementation of sy... more A framework is presented which addresses the issues related to the real-time implementation of synchronised video and audio time-scale and pitch-scale modification algorithms. It allows for seamless real-time transition between continually varying, independent time-scale and pitch-scale parameters arising as a result of manual or automatic intervention. We illuminate the problems which arise in a real-time context as well as provide novel solutions to prevent artefacts, minimise latency, and improve synchronisation. The time and pitch scaling approach is based on a modified phase vocoder with optional phase locking and an integrated transient detector which enables high quality transient preservation in real-time. A novel method for audio/visual synchronisation was implemented in order to ensure no perceptible latency between audio and video while real-time time scaling and pitch shifting is applied. Evaluation results are reported which demonstrate both high audio quality and minimal synchronisation error.

Journal of the Audio Engineering Society, Jan 15, 2008

Page 1. J. Audio Eng. Soc., Vol. 56, No. 1/2, 2008 January/February 49 By Joshua D. Reiss, AES Me... more Page 1. J. Audio Eng. Soc., Vol. 56, No. 1/2, 2008 January/February 49 By Joshua D. Reiss, AES Member INTRODUCTION Sigma–delta modulation (SDM) is per-haps best understood by comparison with traditional pulse-code modulation (PCM). ...

We introduce the Open Multitrack Testbed, an online repository of multitrack audio, mixes or proc... more We introduce the Open Multitrack Testbed, an online repository of multitrack audio, mixes or processed versions thereof, and corresponding mix settings or process parameters such as DAW files. Multitrack audio is a much sought after resource for audio researchers, students and content producers, and while some online resources exist, few are large and reusable, and none allow querying audio fulfilling specific criteria. The test bed we present contains a semantic database of metadata corresponding with the songs and individual tracks, enabling users to retrieve all pop songs featuring an accordion, or all tracks recorded in reverberant spaces. The open character is made possible by requiring that the contributions, mainly from educational institutions and individuals, have a Creative Commons license.

The majority of Digital Audio Workstation designs represent mix data using a channel strip metaph... more The majority of Digital Audio Workstation designs represent mix data using a channel strip metaphor. While this is a familiar design based on physical mixing desk layout, it can lead to a visually complex interface incorporating a large number of User Interface objects which can increase the need for navigation and disrupt the mixing workflow. Within other areas of data visualisation, multi-variate data objects such as glyphs are used to simultaneously represent a number of parameters within one graph-ical object by assigning data to specific visual variables. This can reduce screen clutter, enhance visual search and support visual analysis and interpretation of data. This paper reports on two subjective evaluation studies that investigate the efficacy of different design strategies to visually encode mix information (volume, pan, reverb and delay) within a stage metaphor mixer using multivar-iate data objects and a channel strip design using faders and dials. The analysis of the data suggest that compared to channel strip designs, multivariate objects can lead to quicker visual search without any subsequent reduction in search accuracy.