Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

Response time during video display

Hi there,

I'm currently converting an old e-prime experiment to Opensesame and I'm struggling to handle response times during video presentations. I'm using the media_player_mpy plugin as suggested here. I found similar threads on the forum but I couldn't make it work.

The trial sequence is as follows: Fixation (keypress) - Video (1100ms; response with a keyrelease) - Blank screen (1000ms).

The video clips are exactly 1100 ms long and in each of these clips, one of two events (let's call them 'event A' and 'event B') can occur precisely at 500 ms. Participants are asked to release '1' or '2' in response to 'event A' or 'event B', respectively.

So, I'd like to record participants' response times to these events (with a timeout of 1500 ms from the 'event' onset); so responses should be possible as soon as the video starts with a timeout that overlaps with the blank screen (hope that's clear). I tried several things (e.g., setting the duration to keypress doesn't seem to record anything). Any hints on this ?

Then, I don't exactly know how to handle this 'keypress'/'keyrelease' issue ... I found this on an existing thread but I don't really know how to implement it properly in the experimental sequence ...

import pygame key = None start_time = self.time() while key == None: time = self.time() for event in pygame.event.get(): if event.type == pygame.KEYUP: key = event.key rt = time - start_time exp.set('keyreleaseRT', rt) print '%s was released after %d ms' % (key, rt)

Any advice would be greatly appreciated ! Thanks a lot !

CL

Comments

  • EDIT:

    Some progress have been made based on this thread

    By adding this in the python code section of the video object, responses are collected as long as they are given during the video playback.

    if event: key = event[1] if var.correct_response == key: correct=1 else: correct=0 responses.add(response=key, correct=correct, response_time=clock.time()-var.time_video) continue_playback = True

    The problem is that responses should also be possible once the video has stopped playing (during the 1000 ms blank screen). As it is, 'nothing' is collected and for those trials where responses are given after the end of the video, the same response times as the trial before are collected. Any ideas ?

    Thanks

  • Hi Cletess,

    Can you print the rest of the code that you use? In general, extending the response collection period, requires only to loop a little longer before proceding with the next trial. And optionally, change the stuff that is presented on the screen during this interval. Once I know how you present everything, I can probably also give you more specific suggestions how you could implement it.

    Eduard

    Buy Me A Coffee

  • edited February 2017

    Hi again and sorry for the delay !

    We made some changes on the paradigm and I took some time to read different threads on this forum. It appears (maybe I'm wrong) that for what I need, I'll have to use openCV to deal with the video presentation. Let me explain:

    • After fixation, we want the videos to be played whenever a key is pressed (key '1' or '2' of the numerical pad).
    • From video onset, we would like to record participants' response time to one of two possible events that can occur in the videos by releasing one of the two key that was used to initiate the video ('1' or '2').
    • Finally, whenever participants make a response in a specific condition (presented in a variable in the trial loop), we would like a colored cue to appear on top of the video. This seems to be possible using the pygame.draw.rect command in opencv.

    I saw that this code was suggested in a previous post:

        import cv2
        import numpy
        import pygame
    
        path = pool[self.get('video')]
        video = cv2.VideoCapture(path)
    'insert wait for keypress here?'
        while go:
            success, frame = video.read()
            if not success:
                break
            frame = numpy.rot90(frame)
            # The video uses BGR colors and PyGame needs RGB
            frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
            # Create a PyGame surface
            surf = pygame.surfarray.make_surface(frame)
            # Now you can draw whatever you want onto the PyGame surface!
            pygame.draw.rect(surf, (255,0,0), (25, 25, 25, 25))
            # Show the PyGame surface!
            surface.blit(surf, (0, 0))
            pygame.display.flip()
    

    For now, the screen just goes black for each trial and I can't display the videos. No specific error message. It seems that the value of 'Success' is always False therefore breaking the loop for some reasons. Also, I'm not sure on how to initiate the loop with a key press and to record the reaction time until it is released (the videos must not be stopped once the key is released). I tried something but I'm not really sure it's even correct ...

    key = None
    while key == None:
       start_time = self.time()
        for event in pygame.event.get():
            if event.type == pygame.KEYDOWN:
                key = event.key
                go = True
    
    'Insert video playback code here'
    
    while key == event.key:
       end_time = self.time()
        for event in pygame.event.get():
            if event.type == pygame.KEYUP:
                key = None
    end_time = self.time()
    rt = end_time- start_time  
    exp.set('keyreleaseRT', rt)
    print '%s was released after %d ms' % (key, rt)
    

    Thanks a lot for your advice

  • Hi,

    First, I haven't worked much with videos and not at all with OpenCV, so I can't help you with that. However, I think some of your questions are more general in nature. So, let's see whether I can help you.

    Also, I'm not sure on how to initiate the loop with a key press and to record the reaction time until it is released

    This is rather simple. Basically, when a key is pressed you have something like video.start() which should also give a time stamp for when exactly this happened. If there is not time stamp sent automatically by calling 'video.start', you can add a line like start_time = clock.time() right before you start the video. Importantly, releasing the key shouldn't stop the video, which won't happen unless you write something like "if key is released, stop video playback". So, whenever the key is actually released, you can take a second time stamp. Later, when the trial is over (after all the loops), you can stop the video, by something like "video.stop()". Below the part of the code that would accomplish that (pretty much the same as what you already have had):

    # note, you don't check for pressing a particular key, any key will do in this example
    key = None
    while key == None:
        for event in pygame.event.get():
            if event.type == pygame.KEYDOWN:
                start_time = self.time()
                key = event.key
                go = True # doesn't do anything in this code snippet
    
    #'Insert video playback code here'
    
    while key == event.key:
        for event in pygame.event.get():
            if event.type == pygame.KEYUP:
                key = None
    end_time = self.time()
    rt = end_time- start_time  
    
    exp.set('keyreleaseRT', rt)
    print '%s was released after %d ms' % (key, rt)
    
    # stop the video
    #'Insert video stopping here'
    

    The other issue, I don't know what is happening exactly. If you provide a working example (Opensesame file, incl. an example video), I'll gladly look into to it.

    Eduard

    Buy Me A Coffee

  • Hi Eduard,

    Many thanks for the prompt response ! This makes perfect sense. However, I noticed two things with this:

    • It requires a key to be pressed to initiate the video (which is what I want) but in order to get the release time, I have to press and release it again during the video playback otherwise I get the following error:

        Error while executing inline script
      
      item-stack: experiment[run].Block[run].trial[run].new_inline_script[run]
      exception type: AttributeError
      exception message: 'Event' object has no attribute 'key'
      item: new_inline_script
      time: Tue Feb 07 16:55:13 2017
      phase: run
      
      Traceback:
        File "C:\Program Files (x86)\OpenSesame\lib\site-packages\libopensesame\inline_script.py", line 102, in run
          self.experiment.python_workspace._exec(self.crun)
        File "C:\Program Files (x86)\OpenSesame\lib\site-packages\libopensesame\python_workspace.py", line 161, in _exec
          exec(bytecode, self._globals)
        File "<string>", line 42, in <module>
      AttributeError: 'Event' object has no attribute 'key'
      

    I'm assuming that the keydown command has to be within the video playback loop right ?

    • As for the second issue, I'm assuming that it might be due to the way my video clip are encoded as it seems to be dependent on the video clip i'm using (working for some videos, not for others. All in .avi). I've read something about ffmpeg being not properly installed or something. Would that be possible ?

    I'll look more into that !

  • EDIT:

    Ok, so it seems to be related to how certain videos are encoded. Using the Xvid video codec package (https://www.xvid.com/) seems to solve the problem. The thing now is to figure out how to adapt the frame rate (videos are way too quick). I've read in another thread that it could be possible to achieve something like that using the self.sleep() thingy somewhere in the loop.

    My question would be then: How accurate can we be in setting the frame rate using that option ? The videos I'm using have a framerate of exactly 20fps (I made them from different pictures using VirtualDub). This allows me to control the timing of specific events in the videos and to relate that to participants' response time. Any advice ?

  • EDIT 2:

    Well, after some violent battles with the script, I think I managed to write something that fulfills my needs.
    I think it could potentially be useful for others having similar problems so I post it here (any comments are greatly appreciated)
    This script deals with video presentation using opencv (as suggested elsewhere on this forum).

    This script will wait for a keypress before playing the video and record participants' reaction time when they release the key. Also, depending on participants' responses, we change the background color during video presentation (as a feedback). As the video playback script did not respect the video files 20Hz frame rate (they were way faster), we used the tick_busy_loop(20) to fix the frame rate at 20 fps. It seems to work that way !

        import cv2
        import numpy
        import pygame
    
        #define background colors
        red = (255,0,0)
        black = (0,0,0)
    
        #create the pygame clock object
        FPSCLOCK=pygame.time.Clock()
    
        #check whether the correct backend is being used
        if self.get('canvas_backend') == 'legacy':
            surface = exp.surface
        elif self.get('canvas_backend') == 'xpyriment':
            surface = exp.window
        else:
            raise Exception('This script requires legacy or xpyriment (no opengl)')
    
    
        # Path to the video file. Get the filename from the 'video' variable
        path = pool[self.get('video')]
        # Open the video and determine the video dimensions
        video = cv2.VideoCapture(path)
    
        #initiate the video with a keypress and start the clock
        key = None
        while key == None:
           for event in pygame.event.get():
              if event.type == pygame.KEYDOWN:
                  start_time = self.time()
                  key = event.key
                  go = True
    
        #initiate the video loop
        while go:
    
            #set the frame rate to 20fps and wait if too fast
            FPSCLOCK.tick_busy_loop(20)
    
            # Get a frame and break the loop if we failed, usually because the video
            # ended.
            success, frame = video.read()
            if not success:
                break
    
            #for some reasons the frames appeared inverted & flipped
            frame = numpy.fliplr(frame)
            frame = numpy.rot90(frame)
    
                # The video uses BGR colors and PyGame needs RGB
            frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    
                # Create a PyGame surface
            surf = pygame.surfarray.make_surface(frame)
    
            # Fill the background color of the PyGame surface whenever a specific key is released + collect key release time!
            for event in pygame.event.get():
                if event.type == pygame.KEYUP:
                    #get pygame key name
                    newKey=pygame.key.name(event.key)
    
                    if  newKey == self.get('contingent_action'):
                        end_time = self.time()
                        background_color = red
                        surface.fill(background_color)
                        pygame.display.update
                    else:
                        end_time = self.time()
                        background_color = black
                        surface.fill(background_color)
                        pygame.display.update
    
            # Show the PyGame surface!
            surface.blit(surf, ((surface.get_rect().centerx-150),(surface.get_rect().centery-100)))
            pygame.display.flip()
    
        #define the experimental variable
        rt = end_time-start_time
        exp.set('keyreleaseRT', rt)
        print '%s was released after %d ms' % (key, rt)
    

    Any comments are greatly appreciated (there might be errors :>) and I hope this will help others

  • Hi cletess,

    Thanks for sharing!
    Using video in an experiment can be (always is) very challenging. The website www.videohelp.com can be informative. And the tool XMedia Recode could be handy to convert all videos to the same format.

    Best,
    Jarik

Sign In or Register to comment.

agen judi bola , sportbook, casino, togel, number game, singapore, tangkas, basket, slot, poker, dominoqq, agen bola. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 50.000 ,- bonus cashback hingga 10% , diskon togel hingga 66% bisa bermain di android dan IOS kapanpun dan dimana pun. poker , bandarq , aduq, domino qq , dominobet. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 10.000 ,- bonus turnover 0.5% dan bonus referral 20%. Bonus - bonus yang dihadirkan bisa terbilang cukup tinggi dan memuaskan, anda hanya perlu memasang pada situs yang memberikan bursa pasaran terbaik yaitu http://45.77.173.118/ Bola168. Situs penyedia segala jenis permainan poker online kini semakin banyak ditemukan di Internet, salah satunya TahunQQ merupakan situs Agen Judi Domino66 Dan BandarQ Terpercaya yang mampu memberikan banyak provit bagi bettornya. Permainan Yang Di Sediakan Dewi365 Juga sangat banyak Dan menarik dan Peluang untuk memenangkan Taruhan Judi online ini juga sangat mudah . Mainkan Segera Taruhan Sportbook anda bersama Agen Judi Bola Bersama Dewi365 Kemenangan Anda Berapa pun akan Terbayarkan. Tersedia 9 macam permainan seru yang bisa kamu mainkan hanya di dalam 1 ID saja. Permainan seru yang tersedia seperti Poker, Domino QQ Dan juga BandarQ Online. Semuanya tersedia lengkap hanya di ABGQQ. Situs ABGQQ sangat mudah dimenangkan, kamu juga akan mendapatkan mega bonus dan setiap pemain berhak mendapatkan cashback mingguan. ABGQQ juga telah diakui sebagai Bandar Domino Online yang menjamin sistem FAIR PLAY disetiap permainan yang bisa dimainkan dengan deposit minimal hanya Rp.25.000. DEWI365 adalah Bandar Judi Bola Terpercaya & resmi dan terpercaya di indonesia. Situs judi bola ini menyediakan fasilitas bagi anda untuk dapat bermain memainkan permainan judi bola. Didalam situs ini memiliki berbagai permainan taruhan bola terlengkap seperti Sbobet, yang membuat DEWI365 menjadi situs judi bola terbaik dan terpercaya di Indonesia. Tentunya sebagai situs yang bertugas sebagai Bandar Poker Online pastinya akan berusaha untuk menjaga semua informasi dan keamanan yang terdapat di POKERQQ13. Kotakqq adalah situs Judi Poker Online Terpercayayang menyediakan 9 jenis permainan sakong online, dominoqq, domino99, bandarq, bandar ceme, aduq, poker online, bandar poker, balak66, perang baccarat, dan capsa susun. Dengan minimal deposit withdraw 15.000 Anda sudah bisa memainkan semua permaina pkv games di situs kami. Jackpot besar,Win rate tinggi, Fair play, PKV Games