Detecting voice onsets (voicekey)
Lately, there have been a few requests for a script to detect voice onsets (i.e. a voicekey script). The original example script is no longer available, so below is a new script, written for OpenSesame 3.0.
You can insert this script into the run phase of an
inline_script, and use it like a
keyboard_response. It creates the following variables:
responsewhich is either 'timeout' (when a timeout occurred) or 'detected' (when an onset was detected)
response_timewhich is the response time of the voice onset, which is equal to the timeout when a timeout occurred
loudnesswhich is the loudness of the voice onset that triggered the response, or
Nonewhen a timeout occurred.
You need to tweak (at least) the following properties in the script:
sound_threshold, which determines how sensitive the detection is. The appropriate setting varies a lot from system to system.
timeoutis the maximum response time, after which a timeout is triggered.
Important disclaimer: This code should work, but is based on a very crude algorithm. In most cases, it's probably better to record the voices, and detect the onsets afterwards, rather than doing this online with a voicekey. Also, the code uses
pyaudio, which is sometimes a bit unstable.
import pyaudio import struct import math # A low threshold increases sensitivity, a high threshold # reduces it. sound_threshold = 0.5 # Maximum response time timeout = 5000 FORMAT = pyaudio.paInt16 SHORT_NORMALIZE = (1.0/32768.0) CHANNELS = 2 RATE = 44100 INPUT_BLOCK_TIME = 0.01 INPUT_FRAMES_PER_BLOCK = int(RATE*INPUT_BLOCK_TIME) chunk=1024 def get_rms(block): """Get root mean square as a measure of loudness""" count = len(block)/2 format = "%dh" % (count) shorts = struct.unpack( format, block ) sum_squares = 0.0 for sample in shorts: n = sample * SHORT_NORMALIZE sum_squares += n*n return math.sqrt( sum_squares / count ) # Open the mic stream = pyaudio.PyAudio().open( format=FORMAT, channels=CHANNELS, rate=RATE, input=True, input_device_index=0, frames_per_buffer=INPUT_FRAMES_PER_BLOCK ) # Listen for sounds until a sound is detected or a timeout occurs. start_time = clock.time() while True: if clock.time() - start_time >= timeout: response_time = timeout response = u'timeout' var.loudness = None break try: block = stream.read(chunk) except IOError as e: print e loudness = get_rms(block) if loudness > sound_threshold: response_time = clock.time() - start_time response = u'detected' var.loudness = loudness break # Process the response set_response(response=response, response_time=response_time) # Close the audio stream stream.close() pyaudio.PyAudio().terminate()
There's much bigger issues in the world, I know. But I first have to take care of the world I know.