voice key and recording

Comments

• eduard

  August 2023

  Hi,

  It is a bit hard to judge because the indentation in your code is incorrect, but I suppose the problem is this part:

  if loudness > sound_threshold:
      var.response_time = clock.time() - start_time
      var.response = u'detected'
      var.in_clock_time = clock.time()
      var.start = start_time
      var.loudness = loudness
      break
  

  Once you hit his if statement, you will leave the while loop (bc the break statement), and continue with writing the file. What you want is probable either to wait until the loudness is again below threshold (the person stopped talking?) and only then stop the recording, or wait a fixed amount of time before you leave the while loop. Do you see what I mean?

  If I try to automatically log all the variables in a logger

  Not sure what you mean with automatically logging all the variables. In any case, try to not log all variables and find out which variable is the problem. Once you know you might be able to figure out why it is a problem and whether and how you can fix it.

  Hope this helps,

  Eduard

  
• cltsson

  August 2023

  Ah thanks for answering!

  You mean adding a time.sleep() before the break statement ? I'll give that a go !
• eduard

  August 2023

  That just by itself won't do. You will postpone the breaking the loop, but you will also completely halt your experiment, so nothing else will be recorded. You want your loop to normally proceed, until a certain amount of time has passed. So instead of breaking the loop, you start a timer, that you will check on every iteration until some waiting period has elapsed, and only then, break the loop.

  Does this make sense?

  Eduard

  
• cltsson

  August 2023

  Thanks for this, it got me on the right track (well, I think). I think I get what I want now, however I am not sure it's the most optimal way to accomplish this :)

  I removed the calculation from the variables from the loop and did it "offline", in between trials. Does that make sense ?

  import pyaudio
  import struct
  import math
  import wave
  import pandas as pd
  
  
  
  # A low threshold increases sensitivity, a high threshold
  # reduces it.
  sound_threshold = 0.0005
  # Maximum response time
  timeout = 2000
  
  
  
  
  FORMAT = pyaudio.paInt16
  SHORT_NORMALIZE = (1.0/32768.0)
  CHANNELS = 2
  RATE = 44100
  INPUT_BLOCK_TIME = 0.01
  INPUT_FRAMES_PER_BLOCK = int(RATE*INPUT_BLOCK_TIME)
  filename = "C://Users/kokor/Desktop/"+ str(target) + ".wav"
  chunk=1024
  
  p = pyaudio.PyAudio()
  def get_rms(block):
  """Get root mean square as a measure of loudness"""
  
  count = len(block)/2
  format = "%dh" % (count)
  shorts = struct.unpack( format, block )
  sum_squares = 0.0
  for sample in shorts:
  n = sample * SHORT_NORMALIZE
  sum_squares += n*n
  return math.sqrt( sum_squares / count )
  
  # Open the mic
  stream = p.open(
  format=FORMAT,
  channels=CHANNELS,
  rate=RATE,
  input=True,
  input_device_index=0,
  frames_per_buffer=INPUT_FRAMES_PER_BLOCK
  )
  
  # Listen for sounds until a sound is detected or a timeout occurs.
  print("* recording")
  frames = []
  list_ldn = []
  list_ct = []
  
  start_time = clock.time()
  while clock.time() - start_time <= timeout:
  try:
  block = stream.read(chunk)
  frames.append(block)
  except IOError as e:
  print(e)
  loudness = get_rms(block)
  list_ldn.append(loudness)
  list_ct.append(clock.time())
  
  df = pd.DataFrame({'clock_time':list_ct,'loudness':list_ldn})
  try:
  var.test = (df.loc[df['loudness'] > 0.003, 'clock_time'].iloc[0]) - start_time
  except IndexError:
  var.test = timeout
  
  # Close the audio stream
  stream.stop_stream()
  stream.close()
  p.terminate()
  
  wf = wave.open(filename, 'wb')
  wf.setnchannels(CHANNELS)
  wf.setsampwidth(p.get_sample_size(FORMAT))
  wf.setframerate(RATE)
  wf.writeframes(b''.join(frames))
  wf.close()
  
• cltsson

  August 2023

  import pyaudio
  import struct
  import math
  import wave
  import pandas as pd
  
  
  # A low threshold increases sensitivity, a high threshold
  # reduces it.
  sound_threshold = 0.0005
  # Maximum response time
  timeout = 2000
  
  
  FORMAT = pyaudio.paInt16
  SHORT_NORMALIZE = (1.0/32768.0)
  CHANNELS = 2
  RATE = 44100
  INPUT_BLOCK_TIME = 0.01
  INPUT_FRAMES_PER_BLOCK = int(RATE*INPUT_BLOCK_TIME)
  filename = "C://Users/kokor/Desktop/"+ str(target) + ".wav"
  chunk=1024
  
  
  p = pyaudio.PyAudio()
  def get_rms(block):
  """Get root mean square as a measure of loudness"""
   count = len(block)/2
   format = "%dh" % (count)
   shorts = struct.unpack( format, block )
   sum_squares = 0.0
   for sample in shorts:
   n = sample * SHORT_NORMALIZE
   sum_squares += n*n
   return math.sqrt( sum_squares / count )
  
  
  # Open the mic
  stream = p.open(
   format=FORMAT,
   channels=CHANNELS,
   rate=RATE,
   input=True,
   input_device_index=0,
   frames_per_buffer=INPUT_FRAMES_PER_BLOCK
   )
  
  # Listen for sounds until a sound is detected or a timeout occurs.
  print("* recording")
  frames = []
  list_ldn = []
  list_ct = []
  
  # Start a timer until timeout and compute loudness/clocktime for each block - append to lists
  start_time = clock.time()
  while clock.time() - start_time <= timeout:
   try:
    block = stream.read(chunk)
    frames.append(block)
   except IOError as e:
    print(e)
   loudness = get_rms(block)
   list_ldn.append(loudness)
   list_ct.append(clock.time())
  
  
  # Close the audio stream
  stream.stop_stream()
  stream.close()
  p.terminate()
  
  
  wf = wave.open(filename, 'wb')
  wf.setnchannels(CHANNELS)
  wf.setsampwidth(p.get_sample_size(FORMAT))
  wf.setframerate(RATE)
  wf.writeframes(b''.join(frames))
  wf.close()
  
  
  # merge the 2 lists to a df and compute response_time & response
  df = pd.DataFrame({'clock_time':list_ct,'loudness':list_ldn})
  try:
   var.response_time = (df.loc[df['loudness'] > sound_threshold, 'clock_time'].iloc[0]) - start_time
   var.response = u'detected'
  # use except to deal with errors when no response is detected
  except IndexError:
   var.response_time = timeout
   var.response = u'timeout'
  
• eduard

  August 2023

  Hi,

  optimality is overrated. As long as the code does what it is supposed to and doesn't cause unbearable delays, you're good to go I would say!

  Good luck,

  Eduard

  
• cltsson

  August 2024