Sound presentation and temporal precision with PyAudio and APIs like ASIO

ohavlicek

Hi everyone,
I am doing an experiment where temporal precision of sound presentation is quite important. I am switching to OpenSesame from E-Prime, where one can select in the GUI which sound API/device to use to present the sound, which can make a huge difference in terms of latency and jitter. E-Prime also has a utility called SoundTester, where one can list and test the APIs and devices. In the last few weeks I was trying to figure out how to do this in OpenSesame and I think I have succeeded, at least for my purposes. I'd be happy if I can help someone else struggling with a similar problem, so if you are interested, please have a look at my post here: ondrejhavlicek.com/2016/10/playing-sound-with-low-latency-and-jitter-in-opensesame/

I'm looking forward to hearing your comments, suggestions.. Could it be perhaps possible to make a GUI support for this?

All the best,
Ondrej

Daniel

Hi Ondrej,

This is definitely interesting and some impressive work. We've been trying to get the latency of sound playback down, but never found the opportunity to really dive into this, like you have. This may be interesting stuff for @Jarik too, who is occasionally challenged with experiments requiring low sound latency. The pygame mixer sound playback has been a thorn in our (or at least my) side for a while now, and we have been hoping to replace it with a better alternative that is also able to play compressed formats such as ogg, or even mp3. This is something that pyaudio isn't able to do on its own, so we haven't really looked into this library as a viable alternative. It is of course possible to decode the stream with another library (such as moviepy) and then pass the sound data to pyaudio, but this might introduce some extra latency again.

That being said, have you also had a look at the very impressive (but little known) library sounddevice. It is also based on PortAudio, like pyaudio, but I have found it to be easier to install (it might come bundled with ASIO support, but I haven't checked), have more robust performance across platforms (PyAudio has given me some headaches in the past on Windows), and it looks more frequently maintained than pyaudio at the moment. I am curious to how this library performs, as it has a frequent emphasis on latency in its documentation, so it seems that the library's designer also found this aspect very important.

Maybe you would be willing to convert your project into OpenSesame plugins: a pyaudio based sampler and synth item? @sebastiaan, would you be interested in this?

ohavlicek

Hi Daniel,

Thanks:) No, I didn't know about sounddevice. I just went trough the documentation webpage - it looks very similar to PyAudio, except: it has a nicer documentation, there doesn't seem to be option for setting buffer size, but there is parameter "latency" (can be set to "low" or "high"), there are special settings for ASIO and WASAPI (the "exclusive" parameter looks interesting: "Exclusive mode allows to deliver audio data directly to hardware bypassing software mixing").

It also needs raw sound data as input, but perhaps converting a sound file (mp3 etc) using some other library in the Prepare phase might not be too much of a problem?

Creating the plugins would be nice indeed, as well as playing around with sounddevice. However at the moment I am afraid I have other duties and also am still a beginner in Python.. Or is there some simple way how to turn the code I have posted into the two plugins? Plus a sound tester utility, maybe similar to the PsychoPy monitor center, would be nice, so that normal users know what sound device and what settings to use..

Daniel

Hi Ondrej,
Creating a plugin is relatively simple, and usually you can reuse most of your code, so it shouldn't be that much extra work. Of course, as it is with all things around OpenSesame, you should only work on this if you have the time (and feel like working on it of course ). If you are a beginner in Python, it is a nice learning step, as you learn how to work with a GUI (you only scratch the surface so it won't get technical yet). You can find more information on how to create a plug-in here, and thanks for your work till now!