Add support for 24 bit output devices on Windows · Issue #85 · processing/processing-sound (original) (raw)

Many audio interfaces currently throw a javax.sound.sampled.LineUnavailableException: line with format PCM_SIGNED 44100.0 Hz, 16 bit, stereo, 4 bytes/frame, little-endian not supported on Windows because 16 bit resolution is hard-coded in JSyn's JavaSoundAudioDevice (or don't list any audio devices to begin with, such as the Motu Ultralite mk5).

This might be fixable by bundling the JPortAudio bindings for Windows and making use of the JPortAudioDevice instead.

Work-in-progress

It turns out that getting (and) keeping the engine and synth in a valid state between device manager/synth switches is actually quite tricky. It goes something like this:

• when the Engine singleton is first instantiated it should silently create a synth on:
  • a PortAudioDeviceManager if the first call to it is MultiChannel.usePortAudio() (need to make it clear in the documentation that this absolutely needs to be the first Sound command in the sketch)
  • the default JavaSoundDeviceManager otherwise
• it should then try to start the synth on:
  • if the first call to the library is an explicit Sound.outputDevice(...), the given output device
  • the default output device of the audio device manager otherwise
    * it should be noted that the JavaSound 'default device' number is not actually very informative: https://github.com/philburk/jsyn/blob/06f9a9a4d6aa4ddabde81f77878826a62e5d79ab/src/main/java/com/jsyn/devices/javasound/JavaSoundAudioDevice.java#L86-L89

Assuming we are on JavaSound:

• if the default output device is causing trouble (see comment above), the library should opaquely try to switch to the PortAudioDeviceManager and start a synth on its (possibly more informative?) default output device instead
  • the JavaSoundAudioDevice suppresses the informative LineUnavailableException that indicates we need PortAudio as well as clogging up the console (https://github.com/philburk/jsyn/blob/06f9a9a4d6aa4ddabde81f77878826a62e5d79ab/src/main/java/com/jsyn/devices/javasound/JavaSoundAudioDevice.java#L176-L185), so the Sound library needs to do low-level probing itself (note that AudioSystem.isLineSupported(info) does not seem to be very informative, the proof of the pudding lies in opening the line and provoking an exception)

    private void probeDeviceOutputLine(int deviceId, int sampleRate) throws LineUnavailableException {

After this first (silent) startup, there is a running SynthesisEngine with volume and output nodes.

• if there is more than one device with outputs, should the selected device info be printed to the console?

Whenever the user selects a different input or output device, do the following:

• if we're still on JavaSound: try to switch to PortAudio and, if successful, find the input and output devices with the same or similar names (this might need several attempts, again MME...), printing a note about changed device id's/names ultimately selected. This should also happen if Sound.outputDevice() was the first call to the library!
  • changing device manager requires completely purging the existing SynthesisEngine and all of its associated nodes. Check the Engine's playingUnits tracker for anything already instantiated and warn the user about it!
  • note that when switching audio device managers they need to be instantiated directly, JSyn's AudioDeviceManagerFactory only ever returns whichever device manager it instantiated first
• if we're already on PortAudio: restart the synth on whatever devices given (as long as they have appropriate channels)
  • If there is a problem with opening a line (often with MME devices), either let the AudioDeviceManager print the error to the console, or probe and throw an exception to stop the sketch?
• an open question is who/where should execute System.loadLibrary() for the appropriate native libraries. When the JPortAudioDevice is first instantiated it runs this static import block, not sure how it would raise a failure https://github.com/philburk/portaudio-java/blob/2ec5cc47d6f8abe85ddb09c34e69342bfe72c60b/src/main/java/com/portaudio/PortAudio.java#L101-L121 either way System.out would need to be diverted as this is happening, to suppress JPortAudio output in the console, like so:

  static {
  PrintStream originalStream = System.out;
  System.setOut(new PrintStream(new OutputStream(){
  public void write(int b) { }
  }));
  try {
  System.loadLibrary("portaudio_x64");
  } catch (UnsatisfiedLinkError e) {
  // System.loadLibrary("jportaudio_0_1_0");
  }
  System.setOut(originalStream);
  }

• including (completely optional) PortAudio support on Mac might be desirable, e.g. when using some Bluetooth devices with strange line constraints (such as the Sony WH-CH510) with JavaSound the entire system audio seems to be forced into a low fidelity 16 bit mode, with the Processing audio output reminiscent of what is described here: https://www.reddit.com/r/processing/comments/qtgb1q/audio_quality_is_slow_buzzy_and_distorted/
  • OSX first refuses to load the jnilib, instructions on giving permission in the system preferences would need to be added to the console (added in 843cacf)

Two outstanding glitches/corner cases:

• when JPortAudio finds no output devices, it currently displays a rather uninformative "-1, possibly no available default device" exception
• still need to test whether the following code in selectOutputDevice() also automatically switches to PortAudio as expected when called with a device id that (erroneously) shows 0 output channels on JavaSound:

  // TODO does this also work as expected if the device is currently
  // listed as having 0 output channels?
  // if (this.synth.getAudioDeviceManager().getMaxOutputChannels(deviceId) == 0) {
  // Engine.printMessage(...);
  // } else {
  this.probeDeviceOutputLine(deviceId, this.sampleRate);