Continue experiment while recording with camera

Comments

• eduard

  September 2020 edited September 2020

  Hi Annika,

  What is your current approach? I have not much experience with video recording from the webcam, but it sounds like something OpenCV should be able to do. You would need to install and import this Python package. Then you should be able to control the webcam with inline_scripting.

  Maybe give it a go, and report back to us once you get stuck (or manage).

  Good luck,

  Eduard

  
• AnnikaLinguistics

  September 2020

  Hi Eduard,

  thanks for your response! That is indeed what I'm doing.

  I have now found a solution using psychopy's inline_scripting, but I haven't been able to make it work in opensesame.

  In case someone else wants to continue solving it in opensesame, this is the code that I'm running. I have marked the bits that are problematic and tried to comment on the structural issues in the code.

  # This bit is called once, before the routine starts
  import cv2 as cv
      
  cap = cv.VideoCapture(0)
  if not cap.isOpened():
      print("Cannot open camera")
      exit()
      
  # Define the codec and create VideoWriter object
  fourcc = cv.VideoWriter_fourcc(*'XVID')
      
  out = cv.VideoWriter('video.avi', fourcc, 20.0, (640,  480)) 
  # I am supplying the name 'video.avi' from a variable in my actual code, 
  # so it changes for every routine.
  
  # ----------- The problematic loop ----------------------   
  # Now, this is the problematic bit, as it needs to run during the actual trial/routine:
  # In psychopy, I can delete the loop (and everything from cv.imshow() and have the code be run once per frame
  while cap.isOpened():
  
      ret, frame = cap.read()
      if not ret:
          print("Can't receive frame (stream end?).")
          break
      frame = cv.flip(frame, 1)
      # write the flipped frame
      out.write(frame)
      
      # The following is causing the problem: 
      # It opens the video (which I don't want) but also closes the loop (which is necessary)
      cv.imshow('frame', frame)
      if cv.waitKey(1) == ord('q'):
          break
  
  # ---------- Back to normal ----------------
  # Finally, this needs to be run at the end of the trial/routine
  # Release everything if job is finished
  cap.release()
  out.release()
  cv.destroyAllWindows()
  

  
  
• eduard

  September 2020

  Hi Annika,

  Could you be more explicit in how far it doesn't work in Opensesame? Is there an error message? You can also share the experiment if you want.

  Eduard

  
• AnnikaLinguistics

  September 2020

  Hi Eduard,

  it works in the sense that it opens the camera and starts filming. However, it opens an interface for the camera, showing the video that is being recorded (cv.imshow('frame', frame)), which records until q is pressed. While this is running, the experiment is in the background, but when I click 'q', the camera stops recording but the experiment doesn't move on to the next trial.

  If cv.imshow() is deleted, cv.waitKey() doesn't do anything anymore. The camera just keeps recording in the background and only stops if all processes are stopped in the task manager.

  So, I guess, one would need to build a different kind of trigger for ending the while-loop into the script. That trigger would need to react to what's happening elsewhere in opensesame (namely the end of the trial in the parallel process) but I have no idea how to implement this.

  (But as I said, I have found a solution outside opensesame that works for me, so this is more out of curiosity. I'm sure this can be solved somehow, I just couldn't figure out how.)

  Cheers,

  Annika
• eduard

  September 2020

  If cv.imshow() is deleted, cv.waitKey() doesn't do anything anymore. The camera just keeps recording in the background and only stops if all processes are stopped in the task manager.

  This sounds like the experiment lost its focus, meaning you would need to click inside the experiment window in order to activate it. Could that be it?

  Maybe you can set cv as self.experiment.window, as described here: https://osdoc.cogsci.nl/3.3/manual/backends/#psycho

  Can you share the full experiment? Just copying the code doesn't work for me. (The recording doesn't even start.

  
  

  Thanks,

  Eduard