[open] gst player scripting

sharti

Hey,

I am setting up an eye-tracking experiment involving a video presentation and a concurrent auditory response task, which is timed to certain frames. For now I have put an inline-script and the video player in the parallel component. In the script that is executed after a new frame is displayed, the current frame no. is written to an opensesame variable such that I can assess it in the inline-script that is running in parallel and checks if a certain frame no. is reached and if so plays a sound and waits for a keyboard response.

However I wonder if there is a simpler method without using the parallel plugin, but just scripting on the frame rendering loop of the video player? The problem with the existing possibility of executable python code after every frame is that it appears not to share variables with previous inline-scripts, for e.g. component preparation.

Any help is highly appreciated!

Cheers, Steffen

Josh

Hi Steffen,

Yes it is possible to script on the frame rendering loop of the video player (if you're talking about the media_player_vlc item?). Clicking on that item in the overview window, you can set the Call custom python code parameter to "after every frame". Right below that is the box in which you can insert your code.
You have to create a variable with which you keep count of the frames. This is done in an inline_script prior to the video item (with a simple frame=0). In the video item, you insert the line frame += 1. In the same code window you would have to implement code to play the audio stimulus at the desired frame (e.g. if frame== 200: etc.).

Cheers,

Josh

sharti

I am using the media_player_gst item, because I got some problems using the vlc item. Well your idea is quite what I had in mind, but it seems that when I define a variable in a prior inline script it is not accessible in the custom python code. Thats my trouble.

Josh

Ah, that's because you need to 'set' the variable, making it available outside of the inline_script you created it in:

  frame = 0
  exp.set('frame', frame)

And in the custom python code window:

  exp.set('frame', exp.get('frame') += 1)

Hope this helps.

Josh

sharti

Yeah I do that already, however I cannot set e.g. a sound component in an opensesame variable thus I would like to initialise the component in a prior inline script and access it in the custom code. However that seems not possible and I have to use an inline script which runs in parallel and checks the frame no. (set in an opensesame variable in the custom code). Thus I would like to know if I miss something and somehow it is possible to access inline script python variables from within the custom code, or if the whole media_player_gst item code can be accessed in another way like in psychopy?

sharti

Ok I solved the issue by making some modifications to the media_player_gst.py file, i.e. executing the custom python code with the globals of the opsensesame python_workspace:

exec(self.custom_event_code, self.main_player.experiment.python_workspace._globals, locals())

This way I can run the video player without the parallel plugin. However, now taking the keyboard responses without delaying the video causes trouble. Any ideas?

Josh

Hi Steffen,

The only thing I can think of, is to implement the keyboard response inside the custom code window, rather than to use a keyboard_response item (don't know if you did?). The idea would be to check every frame if a key was pressed - and of course, the possiblity to press a key should only become available after the specifically desired frame.

At the desired frame you could start a timer, (start = self.time()), along with the variable stimulus_onset = True. From then on, every frame you could check if a key was pressed (see also: http://osdoc.cogsci.nl/python/keyboard/#function-keyboardflush. The keyboard command would be preceded by an "if stimulus_onset == True:" line, so that it doesn't check for keyboard responses prior to stimulus onset). If a key was pressed, the response time would be self.time() - start.
The only problem with this method would be that the temporal resolution of your response times isn't too great, as you only check for responses every frame.

Cheers,

Josh

sharti

Thanks for your advice! Yes temporal resolution really goes down. I managed to solve the problem now using the ThreadPool class from the Python multiprocess module. It works really fine. However, I have the problem that it does not work in fullscreen mode, the video would at some point stop and the experiment finishes normally. But this is directly related to the media_player_gst implementation I think. I will set up another discussion about this. http://www.cogsci.nl/forum/index.php?p=/discussion/1630/media_player_gst-not-working-in-fullscreen-mode#Item_1