Audio recording in parallel with an RSVP task

eliza92

Hello everyone! In my current experiment, I have an RSVP task and I want to synchronously record the voice of the participants. (There is no specific duration in the RSVP task, as there is a random number of elements in every trial).

I have written the following code but I have two main problems:

1_ I cannot synchronously have the audio recording and RSVP inline scripts. Either the recording happens before, or after the RSVP task.

2_ If i don't define the seconds for the audiorecording, I'm having wav. files of zero duration. However, I don't want the recording to be time specific, but rather lasts as long as the RSVP does. That is why I am using the example of seconds = 3. However, it suceeds or preceeds the RSVP script

import wave

import os
import random



chunk = 1024 # Record in chunks of 1024 samples

sample_format = pyaudio.paInt16 # 16 bits per sample

channels = 1

seconds = 3

fs = 44100 # Record at 44100 samples per second

output_folder = 'C:\\Users\\vassilis\\Desktop\\test_file'

filename = os.path.join(output_folder, f"{trial_count}.wav")


p = pyaudio.PyAudio() # Create an interface to PortAudio


# Open the mic

stream = p.open(format=sample_format,

channels=channels,

rate=fs,

frames_per_buffer=chunk,

input=True)

frames = [] # Initialize array to store frames


# Store data in chunks for 3 seconds

for i in range(0, int(fs / chunk * seconds)):

data = stream.read(chunk)

frames.append(data)

# ################################################
# THE RSVP TASK


var.T_color = "rgb(255, 255, 255)"

var.letter_dur = 15

var.isi = random.randint(75,86)




var.T_pos = random.randint(7,15)

var.max_lag = 8

var.stream_len = var.T_pos + var.max_lag + 1




letter = ["Α","Β","Γ","Δ","Ε","Ζ","Η","Θ","Ι","Κ","Λ","Μ","Ν","Ξ","Ο","Π","Ρ","Σ","Τ","Υ","Φ","Χ","Ψ","Ω"]

stim_list = random.sample(letter, var.stream_len)




letter_canvas_list = []




for i, stim in enumerate(stim_list):

letter_canvas = Canvas()

if i == var.T_pos:

letter_canvas.color=u'rgb(255, 255, 255)'




letter_canvas.text(stim)

letter_canvas_list.append(letter_canvas)

black_canvas = Canvas()

var.T = stim_list[var.T_pos]

for letter_canvas in letter_canvas_list:

letter_canvas.show()

clock.sleep(var.letter_dur)

black_canvas.show()

clock.sleep(var.isi)

# ########################################################



# Stop and close the stream

stream.stop_stream()

stream.close()

# Terminate the PortAudio interface

p.terminate()




# Save the recorded data as a WAV file

wf = wave.open(filename, 'wb')

wf.setnchannels(channels)

wf.setsampwidth(p.get_sample_size(sample_format))

wf.setframerate(fs)

wf.writeframes(b''.join(frames))

wf.close()

eduard

Hi Eliza,

2_ If i don't define the seconds for the audiorecording, I'm having wav. files of zero duration. However, I don't want the recording to be time specific, but rather lasts as long as the RSVP does. That is why I am using the example of seconds = 3. However, it suceeds or preceeds the RSVP script

Yeah, the only way to go about this is to integrate the RSVP into the recording. So, start the recording, start the RSVP, and then stop the recording. Even if you don't know the length of the RSVP, you can provide a maximum value, right?

1_ I cannot synchronously have the audio recording and RSVP inline scripts. Either the recording happens before, or after the RSVP task.

I am not sure because your code is not correctly formatted (no indents!), but I think it should be possible to wrap the RSVP into the loop of the stream reader. Not ideal, but possible. Potentially it might even work to run a couple of those loops during the RSVP, but again, I would need to see the code to be (slightly more) certain and you definitely would need to check the timing to be sure.

Eduard

eliza92

Thank you Eduard for your immediate response. I send you the code with the indents. I also have left an example of 3 seconds for recording, however I cannot insert the rsvp task inside the recording script, thus the recording happens either before or after the RSVP task.

import pyaudio
import wave
import os
import random

chunk = 1024 # Record in chunks of 1024 samples
sample_format = pyaudio.paInt16 # 16 bits per sample
channels = 1
seconds = 3
fs = 44100 # Record at 44100 samples per second
output_folder = 'C:\\Users\\vassilis\\Desktop\\test_file' #εδω βαλε το δικο σου path
filename = os.path.join(output_folder, f"{trial_count}.wav")

p = pyaudio.PyAudio() # Create an interface to PortAudio


stream = p.open(format=sample_format,
    channels=channels,
    rate=fs,
    frames_per_buffer=chunk,
    input=)

frames = [] # Initialize array to store frames

for i in range(0, int(fs / chunk * seconds)):
    data = stream.read(chunk)
    frames.append(data)


# #######################################

# RSVP task

var.T_color = "rgb(255, 255, 255)"
var.letter_dur = 15
var.isi = random.randint(75,86)

var.T_pos = random.randint(7,15)
var.max_lag = 8
var.stream_len = var.T_pos + var.max_lag + 1

letter = ["Α","Β","Γ","Δ","Ε","Ζ","Η","Θ","Ι","Κ","Λ","Μ","Ν","Ξ","Ο","Π","Ρ","Σ","Τ","Υ","Φ","Χ","Ψ","Ω"]
stim_list = random.sample(letter, var.stream_len)

letter_canvas_list = []

for i, stim in enumerate(stim_list):
    letter_canvas = Canvas()
    if i == var.T_pos:
        letter_canvas.color=u'rgb(255, 255, 255)'

    letter_canvas.text(stim)
    letter_canvas_list.append(letter_canvas)

black_canvas = Canvas()

var.T = stim_list[var.T_pos]

for letter_canvas in letter_canvas_list:
    letter_canvas.show()
    clock.sleep(var.letter_dur)
    black_canvas.show()
    clock.sleep(var.isi)


stream.stop_stream()
stream.close()
p.terminate()

wf = wave.open(filename, 'wb')
wf.setnchannels(channels)
wf.setsampwidth(p.get_sample_size(sample_format))
wf.setframerate(fs)
wf.writeframes(b''.join(frames))
wf.close()

eduard

Hi Eliza,

I can't get my pyaudio installation to work properly (with the limited amount of time that I have), so I can't really try out and advice things. However, what I had in mind is something like in the second code snippet here. In the while loop stream.is_active() you could present the individual elements from the rsvp, something like:

while stream.is_active():
     canvas_list.pop(0).show()
     clock.sleep(50) 

You know what I mean? Not sure whether it will work like that, but it looks promising to me. That being said, I would recommend you draw every frame in your rsvp on a separate canvas, prepare everything in the prepare phase and add the finished canvasses to a list with all the canvases. So that the processing demands are minimal in the run phase (show is very quick compared to drawing things).

I hope this helps,

Eduard