AUFX-O: Novel Methods for the Representation of Audio Processing Workflows (original) (raw)

Novel methods in information management for advanced audio workflows

2009

This paper discusses architectural aspects of a software library for unified metadata management in audio processing applications. The data incorporates editorial, production, acoustical and musicological features for a variety of use cases, ranging from adaptive audio effects to alternative metadata based visualisation. Our system is designed to capture information, prescribed by modular ontology schema. This advocates the development of intelligent user interfaces and advanced media workflows in music production environments. In an effort to reach these goals, we argue for the need of modularity and interoperable semantics in representing information. We discuss the advantages of extensible Semantic Web ontologies as opposed to using specialised but disharmonious metadata formats. Concepts and techniques permitting seamless integration with existing audio production software are described in detail.

Semantic and perceptual management of sound effects in production systems

2004

Main professional sound effects (SFX) providers offer their collections using standard text-retrieval technologies. SFX cataloging is an error-prone and labor consuming task. The vagueness of the query specification, normally one or two words, together with the ambiguity and informality of natural languages affects the quality of the search: Some relevant sounds are not retrieved and some irrelevant ones are presented to the user. The use of ontologies alleviates some of the ambiguity problems inherent to natural languages, yet they pose others. It is very complicated to devise and maintain an ontology that account for the level of detail needed in a production-size sound effect management system. To address this problem we use WordNet, an ontology that organizes real world knowledge: e.g.: it relates doors to locks, to wood and to the actions of knocking. However a fundamental issue remains: sounds without caption are invisible to the users. Content-based audio tools offer perceptual ways of navigating the audio collections, like "find similar sounds", even if unlabeled, or query-byexample. We describe the integration of semantically-enhanced management of metadata using WordNet together with content-based methods in a commercial sound effect management system.

An Ontology For Audio Features

2016

A plurality of audio feature extraction toolsets and feature datasets are used by the MIR community. Their different conceptual organisation of features and output formats however present difficulties in exchanging or comparing data, while very limited means are provided to link features with content and provenance. These issues are hindering research reproducibility and the use of multiple tools in combination. We propose novel Semantic Web ontologies (1) to provide a common structure for feature data formats and (2) to represent computational workflows of audio features facilitating their comparison. The Audio Feature Ontology provides a descriptive framework for expressing different conceptualisations of and designing linked data formats for content-based audio features. To accommodate different views in organising features, the ontology does not impose a strict hierarchical structure, leaving this open to task and tool specific ontologies that derive from a common vocabulary. The ontologies are based on the analysis of existing feature extraction tools and the MIR literature, which was instrumental in guiding the design process. They are harmonised into a library of modular interlinked ontologies that describe the different entities and activities involved in music creation, production and consumption.

Sound effects taxonomy management in production environments

Proc. AES 25th Int. Conf …, 2004

Categories or classification schemes offer ways of navigating and higher control over the search and retrieval of audio content. The MPEG-7 standard provides description mechanisms and ontology management tools for multimedia documents. We have implemented a classification scheme for sound effects management inspired on the MPEG-7 standard on top of an existing lexical network, WordNet. WordNet is a semantic network that organizes over 100.000 concepts of the real world with links among them. We show how to extend WordNet with the concepts of the specific domain of sound effects. We review some of the taxonomies to describe acoustically sounds. Mining legacy metadata from sound effects libraries further supplies us with terms. The extended semantic network includes the semantic, perceptual and sound effects specific terms in an unambiguous way. We show the usefulness of the approach easing the task for the librarian and providing higher control on the search and retrieval for the user.

How to Understand Digital Studio Outputs: The Case of Digital Music Production

IFIP Advances in Information and Communication Technology, 2014

Digital studios trace a great amount of processes and artifacts. The important flow of these traces calls for a system to support their interpretation and understanding. We present our design and development of such a system in the digital music production context, within the Gamelan research project. This trace-based system is structured in three main layers: track production process, interpret collected traces according to a dedicated domain ontology, help querying and visualizing to foster production understanding. We conclude by discussing some hypotheses about trace-based knowledge engineering and digital music production understanding.

Ontological Representation of Audio Features

Lecture Notes in Computer Science, 2016

Feature extraction algorithms in Music Informatics aim at deriving statistical and semantic information directly from audio signals. These may be ranging from energies in several frequency bands to musical information such as key, chords or rhythm. There is an increasing diversity and complexity of features and algorithms in this domain and applications call for a common structured representation to facilitate interoperability, reproducibility and machine interpretability. We propose a solution relying on Semantic Web technologies that is designed to serve a dual purpose (1) to represent computational workflows of audio features and (2) to provide a common structure for feature data to enable the use of Open Linked Data principles and technologies in Music Informatics. The Audio Feature Ontology is based on the analysis of existing tools and music informatics literature, which was instrumental in guiding the ontology engineering process. The ontology provides a descriptive framework for expressing different conceptualisations of the audio feature extraction domain and enables designing linked data formats for representing feature data. In this paper, we discuss important modelling decisions and introduce a harmonised ontology library consisting of modular interlinked ontologies that describe the different entities and activities involved in music creation, production and publishing.

Composition with sound web services and workflows

2007

Sound synthesizer programs such as Csound, Max/MSP, PureData and Supercollider offer the composer a powerful stand-alone environment for the creation of sounds, instruments and complete performances, but sharing, publishing and collaborating are difficult for largely technical reasons, such as platform and configuration. The enforced isolation also necessarily leads to reinvention rather than development and refinement. Service-oriented architectures built on web services are an emerging technology that aims to help interoperation and increase flexibility by decoupling consumer and provider as long as the user is prepared to pay the price of more complex protocols and bear network latency. The potentially high computational load of some synthetic instruments, their sensitivity to the runtime environment and the possibly not unreasonable desire to maintain direct control over a delicate entity, makes the case for investigating the application of web services to the delivery of sound synthesis. Furthermore, the tools that have been developed for joining together web services, into so-called workflows, mimics the patch mechanisms in some synthesizer graphical user interfaces. We report upon the construction of a prototype system for the deployment of sound synthesis software as web services in which we used the Triana and Taverna workflow systems to combine such services. We conclude with an outline of how our related research into the semantic description of mathematical web services and matchmaking and brokerage of such services could be adapted for the domain of sound, opening the way to utilising the entire web as a source of synthetic instruments and computational power to generate music.