The Analysis & Resynthesis Sound Spectrograph (original) (raw)

The ARSS

About

The Analysis & Resynthesis Sound Spectrograph (formerly known as the Analysis & Reconstruction Sound Engine), or ARSS, is a program that analyses a sound file into a spectrogram and is able to synthesise this spectrogram, or any other user-created image, back into a sound.

ARSS is now superseded by Photosounder, which makes use of most of the techniques offered by ARSS in a simple to use and powerful graphical user interface and built in editor.

What it can do

Store all types of sounds as images and reproduce them as sounds
Time stretching, pitch shifting, interval stretching
Filter sound features using an image-editing program
Image transmission over sound
Create sounds from hand-drawn spectrograms such as speech, drum beats or even techno-ish music
Create sounds from real-world images
Synthesise spectrograms created from other spectrographs, such as printed spectrograms of bird calls in books
Reproduce traditional sound effects (such as filtering, equalization, echo, flanger, chorus, phaser, wah-wah, noise reduction, pitch correction) or new variants of effects (such as time-frequency compression) with an image-editing program
Who knows what else, there's a lot of room for experimentation

News
RSS feed

December 17th, 2008 : Photosounder 1.0 released. Photosounder is a new program that I've been working on for the last 3 monthes which incorporates the core of the ARSS into an easy-to-use graphical program. It can load any type of image, OGG and WAV sounds, and allows you to experiment with image-sounds quickly and easily thanks to various controls, the spray tools to edit the images and the faster noise synthesis.

September 15th, 2008 : Alex Ferro made this AppleScript front end for the ARSS on Mac OS X. Also a RSS feed has been added to this site so you can keep track of future updates.

July 29th, 2008 : Thank you to Alex Ferro who sent me this Mac OS X Universal Binary of the ARSS 0.2.3! It contains an installer that you just have to doucle-click. Alex also tells me that anyone who wants to compile the ARSS in Mac OS X should run the command 'gcc *.c -o arss -lm -lfftw3 -O2'.

May 30th, 2008 : The ARSS 0.2.3 released. Changes since version 0.2.2 :

Fixed the logarithmic base system so that it produces the expected results when using a different base than 2
Implemented linear frequency scaling
Fixed the volume of noise bands in noise synthesis, although it may not be perfect and may be way off for logarithmic bases other than 2 and linear frequency scale
Implemented bounds checking when reading and writing in the lower and upper frequencies
Fixed the brightness setting input which would only take integer values
Added a bit more information on the progress display line

May 9th, 2008 : The ARSS 0.2.2 released. Changed the formula used in the filtering function which reduces the visible time domain ripples which can mainly be seen on the lower part of spectrograms, hence increasing the quality of produced spectrograms.

May 5th, 2008 : The ARSS version 0.2 (final version) has just been released. It's the first feature-complete version since 0.1, which means noise synthesis is back in a rewritten and faster form, although not yet as fast as I'm working to make it. Here's a list of significant changes since the previous release :

Noise synthesis has been re-implemented
Full command-line support, meaning the ARSS can now be scripted, that you get define settings you couldn't directly modify before, but also that front-ends can easily be made for it
Major reorganisation in the non-DSP code into easing the re-use of functions and the implementation of image/sound I/O functions by third-party developers
Improved various aspects of IO functions, mainly regarding and few bugs and PowerPC compatibility
Added brightness correction, a parameter similar to gamma correction. Makes spectrograms brighter, which allows for a better visibility when editing a spectrogram but also better sounding results

Technical Description

The ARSS consists in two main parts, a spectrograph with a base-2 logarithmic frequency scale, and a spectrogram synthesiser.

Unlike most spectrographs which are based on STFTs (which perform the analysis by cutting the signal into small time slices to analyse these slices in the frequency domain), the ARSS is based on a filter bank followed by envelope detection, which means that the signal is cut into small frequency-domain slices, and then analysed in the time domain, in a manner quite similar to how analog spectrographs do.

The filter bank is, as of now, made up with overlapping logarithmic-scale frequency-domain Hann windows. Once the original signal is filtered with the filter bank, each resulting signal is sent to envelope detection. The technique used for envelope detection consists in obtaining the magnitude of the analytic signal. The resulting envelope for each frequency band makes the horizontal lines of the image representing the spectrogram. The amplitude of the envelopes translate linearly into intensity in the image.

The spectrogram synthesiser is based on modulation using horizontal lines of the image as envelopes. Each horizontal line is upsampled to the sampling rate of the desired final signal's sampling rate, and is then modulated with, depending on the synthesis mode chosen by the user, sines matching to the central frequency each horizontal line represents, or noise filtered through the filter bank.

This site is in hiatus. Last updated on February 23rd, 2009