Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

voice key recording

Dear all,

I need help implementing a voice key as response. Does this function still exist?

Many thanks!


Katharina

«1

Comments

  • Hi Katherina,

    There is no "official" voice key function in Opensesame, so you probably have to some coding yourself. Nevertheless, there are a couple of versions floating around the forum. How about you give it a browse?

    For example, this one here might be a good starting point: https://forum.cogsci.nl/discussion/5397/issue-with-response-times-and-loudness-in-a-reading-aloud-task-with-voice-key

    Eduard

    Buy Me A Coffee

  • Hi Katherina,

    If you use psychopy for the backend, you can use its official voice recording module:

    https://www.psychopy.org/api/voicekey.html

    You will need to have to do some scripting then though, and you need a very recent version of OpenSesame as this functionality has only recently been added to psychopy too.

    Buy Me A Coffee

  • Thank you very much for the answers. I use Expyriment as the backend.

    I remember there was a sund_start_recording plugin and even an element for that.


    I did not find it anymore.

  • @eduard I am sorry, I did not see your reply from March.

    In fact, what I need is just set an event (reply given) by recording a sound, so that I can anylize the data response locked. Is there a simpler way to do it maybe without having to code a huge inline? Thanks

  • I have now browsed some examples on this forum, also the suggested one, but unfortunately, I did not help. I must confess, I am not skilled at programming though. Therefore I have several questions:

    1. Do I have to install something for the voice recording?
    2. Where exactly is the inline, assuming that this is the code?
    all=[]
    var.A_Response_time = timeout
    start_time = clock.time()
    
    # Open the mic
    stream = pyaudio.PyAudio().open(
    	format=FORMAT,
    	channels=CHANNELS,
    	rate=RATE,
    	input=True,
    	input_device_index=0,
    	frames_per_buffer=INPUT_FRAMES_PER_BLOCK
    	)
    
    
    # Listen for sounds until a sound is detected or a timeout occurs.
    while True:
    	if clock.time() - start_time >= timeout:
    #		var.loudness = None
    		break   
    	try:
    		block = stream.read(chunk)
    		all.append(block)
    		
    	except IOError as e:
    		print e
    	loudness = get_rms(block)
    	
    	if loudness > sound_threshold:
    		var.A_Response_time = clock.time() - start_time
    		var.A_loudness = loudness
    		var.A_clocktime = clock.time()
    		var.A_starttime = start_time
    
    		# clean the canvas
    		canvas.clear()
    		canvas.show()
    		
    		#break
    
    # Close the audio stream
    stream.close()
    pyaudio.PyAudio().terminate()
    
    
    # write data to WAVE file
    data = ''.join(all)
    wf = wave.open(WAVE_OUTPUT_FILENAME, 'wb')
    wf.setnchannels(CHANNELS)
    wf.setsampwidth(pyaudio.PyAudio().get_sample_size(FORMAT))
    wf.setframerate(RATE)
    wf.writeframes(data)
    wf.close()
    

    3.As far as I understand it "listens" to the sound and if its louder than the threshhold it records.

    if loudness > sound_threshold:
    		var.A_Response_time = clock.time() - start_time
    

    I don't know how this would work in my setup: I present a sound (50 ms) every 2 seconds and participants should say a word right after they hear a sound. Theoretically, they can already start saying while the sound is playing. I need the voice response onset and offset times rather then the reaction times. I measure their grip force while they do this contineously. The onset time will be used to lock the grip force data to it in the analysis (similar as in EEG for example). I cant see how this would work. Maybe I do not understand something.

    I really appreciate your help and guidance.

  • You also wrote that its is better to record speech with some other device. Could you suggest some? Is this http://www.cog.brown.edu/people/mertus/BlissDownLoad.shtml something you meant?


    Thanks a lot!

  • Hi Katherina,

    If timing is important, voice key could be a problem as it is not particular reliable/accurate (just imagine all the unrelated sounds/soft voices/ coughs/ etc that could cause misses and false alarms). So, maybe it might even to make more sense to use some other response measure. I haven't tried it, but the code that you posted looks like something that can be used to measure response onset (you understand correctly, the response is triggered if the volume exceeds a threshold). You have to put it in an inline_script item probably in the run phase and place this inline_script in the sequence after the item that you want your participants to respond to.

    Hope this helps a little,

    Eduard

    Buy Me A Coffee

  • Thank you for your quick reply. I do need to use a verbal response.

    I was suggested to you this device as voice key https://cedrus.com/support/sv1/index.htm

    and apparantly it can be integrated using python code

    but I have no experience of doing this. Do you have any suggestings where to start?

    Many thanks.

  • Thank you for your quick reply. I do need to use a verbal response.

    I was suggested to you this device as voice key https://cedrus.com/support/sv1/index.htm

    and apparantly it can be integrated using python code

    but I have no experience of doing this. Do you have any suggestings where to start?

    Many thanks.

  • Hi Katharina,

    Given that you want to run the study online, it might be better to start with javascript right away, rather than Python. But anyway, on Cedrus, they publish example python code how to connect the response box. YOu can find it here: https://github.com/cedrus-opensource/pyxid

    A starting point would be to try to understand and reproduce the code and eventually to adapt it to your needs.

    As for javascript, there doesn't seem to be code published by them, but maybe it is possible to access the protocols that they list with javascript, but unfortunately, I wouldn't know how.


    Eduard

    Buy Me A Coffee

  • Thank you for your quick reply. I do need to use a verbal response.

    I was suggested to you this device as voice key https://cedrus.com/support/sv1/index.htm

    and apparantly it can be integrated using python code

    but I have no experience of doing this. Do you have any suggestings where to start?

    Many thanks.

  • Thank you, @eduard


    This is an other study I want to run online, this one will be normal offline desktop study.

    I will try the code posted here.

    https://forum.cogsci.nl/index.php?p=/discussion/1772/

  • edited June 2020

    Sorry to bother you again.


    Can you explain when the clock.time starts to count in this setup?


    # Listen for sounds until a sound is detected or a timeout occurs.
    start_time = clock.time()
    while True:
    	if clock.time() - start_time >= timeout:
    		response_time = timeout
    		response = u'timeout'
    		var.loudness = None
    		break  
    	try:
    		block = stream.read(chunk)
    	except IOError as e:
    		print e
    	loudness = get_rms(block)
    	if loudness > sound_threshold:
    		response_time = clock.time() - start_time
    		response = u'detected'
    		var.loudness = loudness
    		break
    
    # Process the response  
    set_response(response=response, response_time=response_time)
    


    What I need is actually the start_time (and Rt as well but start_time is more important)


    many thanks!

  • in parallel, I would also like to save the voice data recordings in a wav file. Itried to import wave but it didnt work.

  • Hi Katharina,

    Sorry for the late reply.

    Start time of what? The trial or the voice? if you want to measure if from the start of the trial, that the way you do it, looks good to me. Provided that before the line start_time=clock.time() there is nothing else and the code is put in the run phase of the inline_script, start_time would correspond to the moment Opensesame finishes with the sampler item. Depending on the settings of its duration parameter this can either be after the sound finished playing (sound), or after initializing playing the sound.

    Aside of that, voice keys are not really good RT measures (a lot of background noise, etc. ), so don't put to much hope to get as accurate results as with keyboards, let alone button boxes.

    What packages do you use to record the voice? Pyaudio? Information on how to export recorded sounds are probably explained there. This link looks also promising.

    Hope this helps,

    Eduard

    Buy Me A Coffee

  • Thank you for your answer.


    I have now tried


    import sounddevice as sd

    from scipy.io.wavfile import write


    fs = 44100 # Sample rate

    seconds = 1 # Duration of recording


    myrecording = sd.rec(int(seconds * fs), samplerate=fs, channels=2)

    sd.wait() # Wait until recording is finished

    write('output.wav', fs, myrecording) # Save as WAV file 


    in the beginning of the inline with voice key


    import pyaudio

    import struct

    import math 


    import sounddevice as sd

    from scipy.io.wavfile import write


    fs = 44100 # Sample rate

    seconds = 1 # Duration of recording


    myrecording = sd.rec(int(seconds * fs), samplerate=fs, channels=2)

    sd.wait() # Wait until recording is finished

    write('output.wav', fs, myrecording) # Save as WAV file 



    # A low threshold increases sensitivity, a high threshold

    # reduces it. We need to play around with it. 

    sound_threshold = 0.05

    # Maximum response time 

    timeout = 945


    but I get an error


    IOError: [Errno 13] Permission denied: 'output.wav'


    What do you think this could be? I use Expyriment as background

  • Hi Katherina,

    You don't seem to have permission to write files where you are trying to write the files to. Can you specify a full pah to your desktop?

    I don't know what exactly it is (also depending on your operating system, but something like: C://katherina/Desktop/output.wav?

    Eduard

    Buy Me A Coffee

  • thanks for a speedy reply @eduard


    It's


    C:\Users\Katharina Kühne\Desktop

  • So? Does it work if you specify that path?

    Buy Me A Coffee

  • edited June 2020

    Still get an error

    IOError: [Errno 2] No such file or directory: 'C:\\Users\\Katharina Kuehne\\Desktop\\output.wav'
    
    Maybe I placed it in a wrong way
    
    import pyaudio
    import struct
    import math 
    
    import sounddevice as sd
    from scipy.io.wavfile import write
    
    fs = 44100 # Sample rate
    seconds = 3 # Duration of recording
    
    
    myrecording = sd.rec(int(seconds * fs), samplerate=fs, channels=2)
    sd.wait() # Wait until recording is finished
    write('C:\Users\Katharina Kuehne\Desktop\output.wav', fs, myrecording) # Save as WAV file 
    
    
    # A low threshold increases sensitivity, a high threshold
    # reduces it. We need to play around with it. 
    sound_threshold = 0.05
    # Maximum response time 
    timeout = 945
    
    FORMAT = pyaudio.paInt16 
    SHORT_NORMALIZE = (1.0/32768.0)
    CHANNELS = 2
    RATE = 44100  
    INPUT_BLOCK_TIME = 0.01
    INPUT_FRAMES_PER_BLOCK = int(RATE*INPUT_BLOCK_TIME)
    chunk=1024
    
    
    
    def get_rms(block):
    
    	"""Get root mean square as a measure of loudness"""
    
    	count = len(block)/2
    	format = "%dh" % (count)
    	shorts = struct.unpack( format, block )
    	sum_squares = 0.0
    	for sample in shorts:
    		n = sample * SHORT_NORMALIZE
    		sum_squares += n*n
    	return math.sqrt( sum_squares / count )
    
    
    # Open the mic
    stream = pyaudio.PyAudio().open(
    	format=FORMAT,
    	channels=CHANNELS,
    	rate=RATE,
    	input=True,
    	input_device_index=0,
    	frames_per_buffer=INPUT_FRAMES_PER_BLOCK
    	)
    
    # Listen for sounds until a sound is detected or a timeout occurs.
    start_time = clock.time()
    while True:
    	if clock.time() - start_time >= timeout:
    		response_time = timeout
    		response = u'timeout'
    		var.loudness = None
    		break  
    	try:
    		block = stream.read(chunk)
    	except IOError as e:
    		print e
    	loudness = get_rms(block)
    	if loudness > sound_threshold:
    		response_time = clock.time() - start_time
    		response = u'detected'
    		var.loudness = loudness
    		var.voice_onset = clock.time()
    		var.starttime = start_time
    		break
    
    # Process the response  
    set_response(response=response, response_time=response_time)
    
    
    
    
    # Close the audio stream
    stream.close()
    pyaudio.PyAudio().terminate()
    
  • Not sure, but I think a way is by using the soundFile library. In any case, you can check out this documentation here: https://python-sounddevice.readthedocs.io/en/0.3.12/examples.html

    in particular (recording with arbitrary duration).

    It probably also works with plain python methods, but I wouldn't know fromthe top of my head, what the problem is with your code.

    Eduard

    Buy Me A Coffee

  • I would be all right when it just records 1 s, the responses are around 500 ms, I can cut the rest if I want but it's more of a document WHAT they say not how.


    I believe I am placing that in a wrong place


    import sounddevice as sd
    from scipy.io.wavfile import write
    
    fs = 44100 # Sample rate
    seconds = 3 # Duration of recording
    
    
    myrecording = sd.rec(int(seconds * fs), samplerate=fs, channels=2)
    sd.wait() # Wait until recording is finished
    write('C:\Users\Katharina Kuehne\Desktop\output.wav', fs, myrecording) # Save as WAV file 
    

    sorry for the naive questions

  • Hi Katherina,

    I just checked the code and it works fine. It writes a file after a file is recorded for 3 seconds. However, it does not produce a voice key, so there is nothing happening once you start speaking.

    For that you would indeed need a voice key plugin, like the one provided in psychopy (which you can import in Opensesame and use).

    Sorry for asking again, but can you summarize again, what exactly it is that you need, and what it is that doesn't work.

    • a voice key, that is, marking a response, every time a participant says something?
    • a recorded wave file that you can analyse offline?
    • recorded sound that you can do operations on during the experiment?

    As far as I understand, you want in any case a wave file. That should be easily possible with the sounddevice code. If it doesn't work, then I need more information on what the problem is. Are there error messages? Is a file produced?


    Thanks,

    Eduard

    Buy Me A Coffee

  • Thanks a lot!


    I always get

    IOError: [Errno 13] Permission denied: 'output.wav'

    I specified the path and everything but still. Having reponse a s voice key I finally managed :) thanks to you all

    but now it would be nice to have a wave file.

    Do you have anything special installed on your machine?


    Many thanks!

  • aha, okay. So, only the write file issue left.

    The error message is probably misleading. You probably have the rights to write files to your desktop, but the problem is related to the wrong paths or folders that don't exist. It is hard to really clearly tell you what to do. You can check some online resources that might help addressing the error: for example:

    I hope this helps,

    Eduard

    Buy Me A Coffee

  • Thanks a lot! I realized that this very error was because of my account name with ü, changing the account did not change the path. I recorded it on a neutral USB stick and it worked. But! It only records the very last trial.

    Moreover, it somehow enormly enlarged the presentation time - from 2 to 4 ms!

    Thank you for your patience and help.

  • I mean, seconds of course :)

  • Yeah, Umlaute (as well as white space) is not ideal for file names and directories. I'd change that to something simpler (e.g. katkue?). But nice that it is fixed.

    The delay might come from running it on the USB. However, a doubling of the presentation time is unlikely to come from a device lag. Probably, something else is happening. Can you try to change the presentation time to other values (0.5, 1, 4 seconds) and check whether the actual times are still double each time? If so, then somewhere in your code there is probably some accidental doubling happening, e.g. calling one thing twice. If you share your code or experiment, I could have a look then.


    Eduard

    Buy Me A Coffee

  • edited July 2020

    Many thanks!


    import pyaudio
    import struct
    import math 
    
    # A low threshold increases sensitivity, a high threshold
    # reduces it. We need to play around with it. 
    sound_threshold = 0.02
    # Maximum response time 
    timeout = 990
    
    FORMAT = pyaudio.paInt16 
    SHORT_NORMALIZE = (1.0/32768.0)
    CHANNELS = 2
    RATE = 44100  
    INPUT_BLOCK_TIME = 0.01
    INPUT_FRAMES_PER_BLOCK = int(RATE*INPUT_BLOCK_TIME)
    chunk=1024
    
    import sounddevice as sd
    from scipy.io.wavfile import write
    
    fs = 44100 # Sample rate
    seconds = 1 # Duration of recording
    
    myrecording = sd.rec(int(seconds * fs), samplerate=fs, channels=2)
    sd.wait() # Wait until recording is finished
    write('D:\myselftest\output.wav', fs, myrecording) # Save as WAV file 
    
    def get_rms(block):
    
    	"""Get root mean square as a measure of loudness"""
    
    	count = len(block)/2
    	format = "%dh" % (count)
    	shorts = struct.unpack( format, block )
    	sum_squares = 0.0
    	for sample in shorts:
    		n = sample * SHORT_NORMALIZE
    		sum_squares += n*n
    	return math.sqrt( sum_squares / count )
    
    
    # Open the mic
    stream = pyaudio.PyAudio().open(
    	format=FORMAT,
    	channels=CHANNELS,
    	rate=RATE,
    	input=True,
    	input_device_index=0,
    	frames_per_buffer=INPUT_FRAMES_PER_BLOCK
    	)
    
    # Listen for sounds until a sound is detected or a timeout occurs.
    start_time = clock.time()
    while True:
    	if clock.time() - start_time >= timeout:
    		response_time = timeout
    		response = u'timeout'
    		var.loudness = None
    		break  
    	try:
    		block = stream.read(chunk)
    	except IOError as e:
    		print e
    	loudness = get_rms(block)
    	if loudness > sound_threshold:
    		response_time = clock.time() - start_time + prepare_duration
    		response = u'detected'
    		var.loudness = loudness
    		var.voice_onset = clock.time()
    		var.starttime = start_time
    		break
    
    # Process the response  
    set_response(response=response, response_time=response_time)
    
    # Close the audio stream
    stream.close()
    pyaudio.PyAudio().terminate()
    


    The file is included.

    It's actually a mega easy experiment, with a mentronome sound coming every (ideally) 2 seconds, and participants having to say smth after the sound. This sould be recorded as voice_onset time and the wav.file

  • have you checked the consistency of the delay as I suggested?

    Buy Me A Coffee

Sign In or Register to comment.

agen judi bola , sportbook, casino, togel, number game, singapore, tangkas, basket, slot, poker, dominoqq, agen bola. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 50.000 ,- bonus cashback hingga 10% , diskon togel hingga 66% bisa bermain di android dan IOS kapanpun dan dimana pun. poker , bandarq , aduq, domino qq , dominobet. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 10.000 ,- bonus turnover 0.5% dan bonus referral 20%. Bonus - bonus yang dihadirkan bisa terbilang cukup tinggi dan memuaskan, anda hanya perlu memasang pada situs yang memberikan bursa pasaran terbaik yaitu http://45.77.173.118/ Bola168. Situs penyedia segala jenis permainan poker online kini semakin banyak ditemukan di Internet, salah satunya TahunQQ merupakan situs Agen Judi Domino66 Dan BandarQ Terpercaya yang mampu memberikan banyak provit bagi bettornya. Permainan Yang Di Sediakan Dewi365 Juga sangat banyak Dan menarik dan Peluang untuk memenangkan Taruhan Judi online ini juga sangat mudah . Mainkan Segera Taruhan Sportbook anda bersama Agen Judi Bola Bersama Dewi365 Kemenangan Anda Berapa pun akan Terbayarkan. Tersedia 9 macam permainan seru yang bisa kamu mainkan hanya di dalam 1 ID saja. Permainan seru yang tersedia seperti Poker, Domino QQ Dan juga BandarQ Online. Semuanya tersedia lengkap hanya di ABGQQ. Situs ABGQQ sangat mudah dimenangkan, kamu juga akan mendapatkan mega bonus dan setiap pemain berhak mendapatkan cashback mingguan. ABGQQ juga telah diakui sebagai Bandar Domino Online yang menjamin sistem FAIR PLAY disetiap permainan yang bisa dimainkan dengan deposit minimal hanya Rp.25.000. DEWI365 adalah Bandar Judi Bola Terpercaya & resmi dan terpercaya di indonesia. Situs judi bola ini menyediakan fasilitas bagi anda untuk dapat bermain memainkan permainan judi bola. Didalam situs ini memiliki berbagai permainan taruhan bola terlengkap seperti Sbobet, yang membuat DEWI365 menjadi situs judi bola terbaik dan terpercaya di Indonesia. Tentunya sebagai situs yang bertugas sebagai Bandar Poker Online pastinya akan berusaha untuk menjaga semua informasi dan keamanan yang terdapat di POKERQQ13. Kotakqq adalah situs Judi Poker Online Terpercayayang menyediakan 9 jenis permainan sakong online, dominoqq, domino99, bandarq, bandar ceme, aduq, poker online, bandar poker, balak66, perang baccarat, dan capsa susun. Dengan minimal deposit withdraw 15.000 Anda sudah bisa memainkan semua permaina pkv games di situs kami. Jackpot besar,Win rate tinggi, Fair play, PKV Games