MarsyasX (original) (raw)

MarsyasX: multimedia dataflow processing with implicit patching

2008

Abstract The design and implementation of multimedia signal processing systems is challenging especially when efficiency and real-time performance is desired. In many modern applications, software systems must be able to handle multiple flows of various types of multimedia data such as audio and video. Researchers frequently have to rely on a combination of different software tools for each modality to assemble proof-of-concept systems that are inefficient, brittle and hard to maintain.

Marsyas: A framework for audio analysis

2000

Existing audio tools handle the increasing amount of computer audio data inadequately. The typical tape-recorder paradigm for audio interfaces is inflexible and time consuming, especially for large data sets. On the other hand, completely automatic audio analysis and annotation is impossible using current techniques. Alternative solutions are semi-automatic user interfaces that let users interact with sound in flexible ways based on content.

Flexible scheduling for dataflow audio processing

Proc. ICMC, 2006

The notions of audio and control rate have been a pervasive feature of audio programming languages and environments. Real-time computer music systems depend on schedulers to coordinate and order the execution of many tasks over the course of time. In this paper we describe the scheduling infrastructure of Marsyas-0.2, an open source framework for audio analysis and synthesis. We describe how to support multiple, simultaneous, dynamic control rates while retaining the efficiency of block audio ...

An open architecture for real-time audio processing software

PREPRINTS-AUDIO ENGINEERING …, 1999

OSW, or "Open Sound World," allows development of audio applications using patching, C++, high-level specifications and scripting. In OSW, components called "transforms" are dynamically configured into larger units called "patches." New components can be expressed using familiar mathematical definitions without deep knowledge of C++. High-level specifications of transforms are created using the "Externalizer," and are compiled and loaded into a running OSW environment. The data used by transforms can have any valid C++ type. OSW uses a reactive real-time scheduler that safely and efficiently handles multiple processors, time sources and synchronous dataflows.

Faustine: a Vector Faust Interpreter Test Bed for Multimedia Signal Processing System Description

2015

Abstract. Faustine is the first interpreter for the digital audio sig-nal processing language Faust and its vector extension. This domain-specific language for sample-based audio is highly expressive and can be efficiently compiled. Faustine has been designed and implemented, in OCaml, to validate the Faust multirate vector extension proposed in the literature, without having to modify the sophisticated Faust scalar com-piler. Moving to frame-based algorithms such as FFT is of paramount importance in the audio field and, more broadly, in the multimedia signal processing domain. Via the actual implementation of multidimensional FFT and morphological image processing operations, Faustine, although unable to process data in real time, illustrates the possible advantages and shortcomings of this vector extension as a language design proposal. More generally, our paper provides a new use case for the vision of inter-preters as lightweight software platforms within which language design a...

A dataflow pattern catalog for sound and music computing

Proceedings of the 2006 conference on Pattern languages of programs - PLoP '06, 2006

This article describes a set of patterns the authors have seen emerging during years of experience developing assorted applications in the sound and music domain and receiving influences from theoretical models, existing systems, and colleagues.

Platforms for handling and development of audiovisual data

2012

Everywhere around us, digital is replacing analogue. Such is especially true in the audiovisual: be it in consumer or professional market, the advantages of computer-based media have quickly persuaded investors. To choose in which technologies, software should be based, proper understanding of the major aspects behind digital media is essential. An overview of the state of the art regarding the compression of digital video, the existing container formats and the multimedia frameworks (that enable easier playback and editing of audiovisual content) will be given, preceded by an introduction of the main topics. The professional video market is particularly focused on this document, due to the context of its elaboration, predominantly MPEG-2, MXF and DirectShow. In order to obtain more accurate results, fruit of the practical contact with the technology, a hands-on programming project was accomplished using DirectShow to playback both local and remote WMV files. Compression algorithms are continuously being improved and new ones invented, the designers of container formats are paying increased attention to metadata and some serious alternatives to DirectShow are emerging. In a continuously mutating environment, light is shed at what has, what is and what will be happening in the upcoming years on the digital video scene. Results show that the choice of a particular video coding format or multimedia framework depends heavily on the purpose of its application and that there is no universal solution. Various use scenarios are given an answer as to which is the best course of action concerning which technologies to apply. The software developed was ultimately given a positive feedback by the client.

An audio virtual DSP for multimedia frameworks

… 2001. Proceedings.(ICASSP'01). 2001 IEEE …, 2001

The new MPEG-4 Audio standard provides two toolsets for synthetic Audio generation, Audio processing and multimedia content description called Structured Audio (SA) and BInary Format for Scenes (BIFS).

A framework for automatic generation of audio processing applications on a dual-core system

In this paper we describe an open framework for programming a dual-core system whose architecture is designed for ultra low-power audio processing applications. We show how the system's architecture leads to a model of computation, i.e. a formalism that describes the execution, scheduling and interaction between components in a system. We also show how applications can automatically be created from a set of components that are specifically architected for the dual-core system in terms of resource usage and performance using a set of complexity metrics associated with the model of computation.