Continue experiment while recording with camera

AnnikaLinguistics

Hi all,

I am setting up an experiment in which participants perform a task in German or British Sign Language, so I want to film them. I would like to create a new recording for each trial, which is recorded in the background throughout the trial and stopped by the button press that ends the trial.

Right now, my inline script opens the laptop camera and records, but is foregrounded and needs to be ended separately from the trial. Only after having closed the camera, can I continue the experiment. Any ideas on how to fix this?

Thanks!

eduard

Hi Annika,

What is your current approach? I have not much experience with video recording from the webcam, but it sounds like something OpenCV should be able to do. You would need to install and import this Python package. Then you should be able to control the webcam with inline_scripting.

Maybe give it a go, and report back to us once you get stuck (or manage).

Good luck,

Eduard

AnnikaLinguistics

Hi Eduard,

thanks for your response! That is indeed what I'm doing.

I have now found a solution using psychopy's inline_scripting, but I haven't been able to make it work in opensesame.

In case someone else wants to continue solving it in opensesame, this is the code that I'm running. I have marked the bits that are problematic and tried to comment on the structural issues in the code.

# This bit is called once, before the routine starts
import cv2 as cv
    
cap = cv.VideoCapture(0)
if not cap.isOpened():
    print("Cannot open camera")
    exit()
    
# Define the codec and create VideoWriter object
fourcc = cv.VideoWriter_fourcc(*'XVID')
    
out = cv.VideoWriter('video.avi', fourcc, 20.0, (640,  480)) 
# I am supplying the name 'video.avi' from a variable in my actual code, 
# so it changes for every routine.

# ----------- The problematic loop ----------------------   
# Now, this is the problematic bit, as it needs to run during the actual trial/routine:
# In psychopy, I can delete the loop (and everything from cv.imshow() and have the code be run once per frame
while cap.isOpened():

    ret, frame = cap.read()
    if not ret:
        print("Can't receive frame (stream end?).")
        break
    frame = cv.flip(frame, 1)
    # write the flipped frame
    out.write(frame)
    
    # The following is causing the problem: 
    # It opens the video (which I don't want) but also closes the loop (which is necessary)
    cv.imshow('frame', frame)
    if cv.waitKey(1) == ord('q'):
        break

# ---------- Back to normal ----------------
# Finally, this needs to be run at the end of the trial/routine
# Release everything if job is finished
cap.release()
out.release()
cv.destroyAllWindows()

eduard

Hi Annika,

Could you be more explicit in how far it doesn't work in Opensesame? Is there an error message? You can also share the experiment if you want.

Eduard

AnnikaLinguistics

Hi Eduard,

it works in the sense that it opens the camera and starts filming. However, it opens an interface for the camera, showing the video that is being recorded (cv.imshow('frame', frame)), which records until q is pressed. While this is running, the experiment is in the background, but when I click 'q', the camera stops recording but the experiment doesn't move on to the next trial.

If cv.imshow() is deleted, cv.waitKey() doesn't do anything anymore. The camera just keeps recording in the background and only stops if all processes are stopped in the task manager.

So, I guess, one would need to build a different kind of trigger for ending the while-loop into the script. That trigger would need to react to what's happening elsewhere in opensesame (namely the end of the trial in the parallel process) but I have no idea how to implement this.

(But as I said, I have found a solution outside opensesame that works for me, so this is more out of curiosity. I'm sure this can be solved somehow, I just couldn't figure out how.)

Cheers,

Annika

eduard

If cv.imshow() is deleted, cv.waitKey() doesn't do anything anymore. The camera just keeps recording in the background and only stops if all processes are stopped in the task manager.

This sounds like the experiment lost its focus, meaning you would need to click inside the experiment window in order to activate it. Could that be it?

Maybe you can set cv as self.experiment.window, as described here: https://osdoc.cogsci.nl/3.3/manual/backends/#psycho

Can you share the full experiment? Just copying the code doesn't work for me. (The recording doesn't even start.

Thanks,

Eduard