Video stimuli and response time

sarita

Hi,

I want to create a simple experiment where a video is played and participants respond each time they see a stimuli while the video keeps running. I will need only response times. My knowledge in this field is very limited (I study linguistics) and failed to create an experiment although I ve been trying for more than 3 months. I will be very grateful and appreciative if you could provide simple steps to create an experiment.

Best wishes,

Fab

Hi @sarita,

I must preamble this reply by saying that I’m no computer programmer either and that I’ve only started exploring OS recently (in fact until today, I had not touched Python code), so what I’m suggesting below might not be the optimal solution to your problem, but hopefully it will be of some help and other users could add their contribution and suggest better ways.

I found some useful hints looking at this previous thread: https://forum.cogsci.nl/discussion/2651/video-playback-and-keyboard-response (credit to @Daniel), as well as in OS’s documentation (especially https://osdoc.cogsci.nl/3.1/manual/python/responses/ and https://osdoc.cogsci.nl/3.3/manual/python/log/).

From your description, I’m not entirely sure what your stimuli are and how they are presented. I’m going to assume that your task simply consists in showing videos and that your stimuli are part of these videos and not some other stimulus that the task must present on top of the video. If my assumption is not correct, then the solution below will not be suitable for you and you'd need something more sophisticated that falls outside my level of expertise at the moment).

So, assuming that you want participants to press keys when some stimulus appears within your video footage, it seems that the way to do this is by using Python code embedded into the media_player_mpy (which, in my example, I called myvideo) object to process responses, and to have that code executed after every frame.

I’ve put together an example. This is its structure:

I built a loop with two trials, each consisting in the presentation of a short video (the file name is stored in the videofile variable, which I’ll later use to call the video in the myvideo object).

In each trial sequence, we have a bit of Python code to set up a little time buffer, then a message indicating which video file is about to be played, and then the media_player_mpy (which I called myvideo) with its embedded Python code.

Let me describe the general idea of the Python code embedded in the media_player_mpy object: this code will be executed after every frame of the video. For this, we have to make sure to select the “after every frame” option of the “Call custom Python code” property (and let's also make sure to read the video file name from the filename variable):

So, what does the code do? Basically this: to detect whether a key has been pressed, and if it has, to collect the response and some of its properties, to output all variables to the log file, and to tell OS to keep playing the video (as opposed to stopping the video when a key press is detected).

Here's the code with annotations:

# code to execute if an event is detected and a certain amount of time has passed since the previous event
# This buffer avoids slugish key presses to be elicit several responses
if event and clock.time()>time_buffer:
# collect the last key pressed
key = event[1]
# adds the key and response time to collection of responses
responses.add(response=key, response_time=clock.time()-var.time_myvideo)
# write data to log
log.write_vars()
# sets up tome buffer
time_buffer = clock.time()+500
# Tell OS to keep playing the video
continue_playback = True

You’ll notice that I define and use a “time buffer”. The reason for this is that the code is executed very fast (after every frame of the video). That means that you easily get the same key press collected twice if you don’t do anything about it. If your video plays with at 24 frames per sec, depending on the frequency with which your computer samples your keyboard, and how long the subject keeps the key down when responding, what constitutes a single response from the participant’s perspective may come out as two responses in very quick succession for the task. So, to avoid that, I define a time buffer that correspond to the time it is when a response is collected (measured from the computer’s internal clock) + some arbitrary amount (here I use a value of 200 and its well on my computer). The Python code will only process key presses if they happen at least 200 time units since the previous response. 

You'll notice that each response time is calculated from the onset of the video (not from response to response). To measure response times from response to response, you could simply open the log file in Excel and use a formula, or you could edit the Python code to change the way the response time is calculated).

Finally, right after the myvideo object, I inserted some Python code simply to output all the responses collected during the task to the console - it does nothing to the task per se (tip: outputting things to the console is a great way of checking what's going on in your task as it runs; can help identifying problems).

You'll find the responses and response times in the log file,. The key pressed will show in the response column, the response times in the response_time column, and you can differentiate between trials by looking at the count variable of any object inserted in the trials_sequence, such as count_msg, for example:

Because we're outputting all variables to the log, you'll find there a lot of additional variables (e.g., the name of the video file that was playing when each response was recorded).

You'll notice that the output to the console is not exactly the same as the content of the log file... This is because to the console, I'm printing all the responses collected during the experiment, which include the keys pressed to skip the welcome and comment sketchpads. Whereas the log file will only contain what I manually send to the log, which is done in the Python code of the myvideo object (which, as you can see from the code, only is executed when a key is pressed while the video is playing). Incidentally, you'll notice that the task contains no logger object! The logging is done manually. If your task is more complex and involves other parts where you collect responses, you'll need to figure out how you want to handle the logging to make sure you collect all the responses you need for your analysis.

You can download my the OS task used in this example here: https://we.tl/t-AlJYzlc1o4

Hope this helps. Good luck!

Fabrice.

sarita

Hi Fabrice,

You can't imagine how you saved me. No words can express how I am grateful for you. Thank you very much.

Just one more thing if you can show me what to write as a code within the inline script (time buffer). I didn't write anything as a code within time buffer so I get the log file with more than 2000 columns.

Kind regards,

Fab

Hi @sarita,

I'm glad I could help you. 👍️

I think that the link provided to download my example had already expired when you read my post. I've uploaded the task to Dropbox, so that you should be able to download it and see all the code. You can find it here: https://www.dropbox.com/s/p7gegzyn5syor0n/Responses%20during%20video%20example.osexp?dl=0

To answer your specific question about the sets_timebuffer code, here it is:

time_buffer = clock.time()+500

You'd want to adjust this value based on your specific design (I'm not sure how frequently the target stimuli appear in your videos). The shorter the time you set, the higher the risk that a response is registered twice instead of once. In my example, a 500 value works well,. I expect it should in your case too, but do feel free to play around with it to check what is best for you.

I'm not sure what you mean about the log and the 2000 columns. the log I get from my example is relatively clean (though it is true that it contains

Normally, you'd want to include a log object in the trial sequence to save all variables (default option) or a manually selected list. However, in this case the logging is executed by directly by code in the script of the myvideo object.

The log.write_vars() will trigger the logging of all variables.

You get the required information and a total of 75 columns, including those that will be critical for your analysis, such as the actual keys pressed, the response_time (measure from the onset of the video), the video being played, or variables that are incremented on every trial and that you could use as a proxy for the trial number.

I think it's always better to have too many variables logged than to risk having some useful one missing once your data collection is done. However, it appears to be possible to specify selectively the list of variables to be logged using by passing it as a parameter of the log_write_vars function (https://osdoc.cogsci.nl/3.2/manual/python/log/). If your task contains many other objects than my example, it may be that your number of columns may get very large and that it creates some small time lags, skipping or freezing during video playback. If this were the case, my advice would be to (1) check whether you still spot that problems after removing the log.write_vars() line (that would be a strong indication that the logging is taking too many resources away from the video playing), and, in that case, (2) use the log.write_vars and specify which variables you want logged exactly.

If useful, you could easily add some trial-specific information in different ways, either declaring variables using code, or more simply, inserting new variables in the loop table:

For example, let's say you have videos belonging to different conditions (say positive, neutral and negative), you could add a variable for the condition or whatever you consider relevant and want to see appear in the data log:

Adding a logging object in the task actually creates a messy output (because, as I mentioned above, we already take care of the logging manually through code), so in this case I'd recommend against it if you don't require it somewhere else in the task.

Hope this helps and that the example will allow you to finish programming your task. In any case, make sure to check and double check everything before you start testing participants. The example I programmed seems to do the job but I'm not aware of all the requirements of your specific experiment.

Best,

Fabrice.

PS: When replying in the forum, you might want to use the handlers so that your correspondent gets notified of your message (otherwise they won't know unless they manually revisit the thread). You just need to type @ followed by the first letters of your correspondent's username and a pull-down menu will pop up. For example, if you type @fa, this will appear:

https://forum.cogsci.nl/uploads/210/HHQ0G4PR8YWN.png

sarita

Hi @Fab,

Really appreciate your detailed explanation. Thank you so much for your time and trouble.

Could you please reload the task to Dropbox? It says that the file has been deleted.

Fab

Hi @sarita,

Apologies for the broken link. Here is a new link that should work: https://www.dropbox.com/s/l7uji6ri46h336y/Responses%20during%20video%20example.osexp?dl=0

Hope this helps,

Fabrice.

sarita

Hi @Fab,

Eveverything works perfectly now. I really hope you always find someone to help you like you did with me. Wish you all the best.

Best wishes,

Fab

Hi @sarita,

Glad I could help you! I started using OS in March and found that helping other users is a good way to learn the program. The forum is a good place to come if you get stuck; I got some great help from the moderators when I needed it.

Good luck with your experiment!

Best wishes,

Fabrice.

sarita

Hi @Fab,

I am having problems with this experiment (there is a link to it 2 messages above). Not sure if it is because I have updated opensesame since it was running perfectly before. Could you please check it?

Kind regards,

Fab

Hi @sarita,

I'm not sure what you mean when you say that you're having problems. What do you mean? What error message comes up? Can you describe?

Best,

Fabrice.

sarita

Hi @Fab

this is the message error:

Errors occurred while opening the file:

No module named 'libqtopensesame.items.media_player_mpy'

This is the experiment you helped me with. It was running perfectly but now it stops and doesn't run at all. Does it still run with you?

https://www.dropbox.com/s/l7uji6ri46h336y/Responses%20during%20video%20example.osexp?dl=0

Fab

Hi @sarita,

It sounds as if something has changed on your computer. Not sure what the issue is, I must say. The task runs fine on mine. Have you tried from a different computer? Perhaps try to reinstall Open Sesame (I saw that a new update has just come out:

Download // OpenSesame documentation (cogsci.nl)

Best,

Fabrice.

sarita

Hi @Fab

If it runs on your computer then the problem is because I have upgraded my macos to big sur. Do you know what to do to run it on big sur?

Fab

Hi @sarita,

I'm not using Mac but I do remember seeing something about Big Sur and a colleague of mine had trouble after upgrading to that version of Mac OS. We managed to get OS to run following @Daniel's solution posted here: https://forum.cogsci.nl/discussion/6688/version-3-3-5-is-incompatible-with-macos-bigsur#Comment_21525

Hopefully it can work for you too.

Best,

Fabrice.