Running PsychoPy code in OpenSesame for eye-tracking

Agata · August 2016

Hi! I wanted to adapt an experiment that is fully written in PsychoPy to Open Sesame for the purpose of running it with Tobii eye-tracker. Can I try to simply copy the full code and paste it in the inline script in Open Sesame and then add the pygaze items to start/stop recording? Or should I add the recording commands as inline script as well?

The experiment shows (in 4 blocks) a series of pictures in different places on the screen and requires keyboard presses as participant's answers. No audio required.

I would be extremely grateful if somebody would know what is the best way to deal with that in OpenSesame or whether there is a better way to run a PsychoPy experiment with Tobii. Thanks in advance!

Agata

sebastiaan · August 2016

Hi Agata,

There are a few questions here.

Can I try to simply copy the full code and paste it in the inline script in Open Sesame

So how you can run your PsychoPy script in OpenSesame? You can simply do this by inserting an inline_script as the only item in your experiment sequence, and then copy-pasting your script in there. At the beginning of your script, you probably create a Window, with something like:

my_window = psychopy.visual.Window()

You have to remove this line, because OpenSesame already creates a window for you, and you have to replace all occurrences of my_window (or whatever you called the Window) by win (which is what OpenSesame calls the Window). After that, your experiment should run fine.

Now I should point out that this is not the normal way to use OpenSesame: Normally, you would build your experiment by connecting many different items, including short inline_scripts. What you're doing now is simply implementing the entire experiment with one enormous script, which works, but is nonstandard.

for the purpose of running it with Tobii eye-tracker.

My understanding is that PyGaze has experimental support for non-EyeX models of Tobii trackers (but I've never used a Tobii myself). If I were you, I would first set up a simple dummy experiment to see if you can get the tracker to work at all. If you do, you can then embed this in the bigger experiment. And if you don't, you won't have lost an enormous amount of time for nothing.

Some discussions to get you started:

Cheers!
Sebastiaan

Edwin · August 2016

Hi Agata,

There are other options that seem like they might fit your problem a bit better:

1) Use iohub to communicate with the tracker; this should be compatible with your script. I'm unfamiliar with the exact interface, and the documentation isn't there yet (although I did find this, which seems a bit less incomplete), but from the actual source code it looks like a good package.

2) Another option that should be compatible with your script, is to use PyGaze (the Python module, not the OpenSesame plug-ins). You can get a handle to the active PsychoPy Window like so:

import pygaze
from pygaze.display import Display
from pygaze.eyetracker import EyeTracker

# Initialise a new Display instance
disp = Display(disptype='psychopy', dispsize=(1920,1080))

# Initialise a new EyeTracker instance
tracker = EyeTracker(disp, trackertype='tobii')

# Get the handle to the active Window
win = pygaze.expdisplay

Now you can use your existing script. Where you referenced the active Window, you can simply use win (which is the reference to the active PsychoPy Window created in PyGaze).

To start recording, you can call tracker.start_recording(), and to pause it, you can call tracker.stop_recording(). To log events, such as trial starts and screen flips, you can use the `tracker.log("example message").

Also, make sure to add the following at the end of your script, to close the connection with the eye tracker, and to close the active Window (this replaces win.close())::

# Close the connection to the EyeTracker
# (This will also close the log files.)
tracker.close()

# Close the active Window
disp.close()

Cheers,
Edwin

Agata · August 2016

Thank you, Sebastiaan and Edwin! It's all very helpful and informative! I will try to play around with both solutions and will see what works best.

Edwin, you might be right that using PyGaze in the Python module could be maybe the easiest option here. As I am completely new to pygaze and don't have so much programming experience I will have some (maybe a bit naive) questions about how to do it step by step:

1) Before I add the code, should I first install pygaze separately from here (http://www.pygaze.org/installation/) and get the tobii SDK package from here (http://www.tobiipro.com/product-listing/tobii-pro-analytics-sdk/#Download) as it says in the pygaze installation tutorial? Is that all I need if I already have PsychoPy?

2) Then I guess I should follow these instructions about the tobii package? "Once you have downloaded the package, unzip it to a location of your choice. Open the unzipped folder and look for a folder called ‘Python27′ (or ‘python27′). You’ll find a folder called ‘Modules’ (or ‘modules’) in there. Open it and copy it’s contents (ALL of it!) to the site-packages directory of your Python installation (Windows: C:\Python27\Lib\site-packages; Linux: usr/lib/pymodules)."

3) When I have pygaze and tobii SDK, should I just add the code lines directly in the PsychoPy coder and it will communicate with pygaze and idenfity the tobii tracker? Or is there some trick that additionally they should all be in the same folder or so?

3) Once I collect the data - is there any way to visualise it overlayed on the stimulus images without too much programming experience? Is there a way to create the Areas of Interest? Should they be pre-defined (as sometimes has to be done in E-Prime) or can I define them post hoc?

And again, thanks a lot for you help! (and there will be probably more questions to come... )

Agata

sebastiaan · August 2016

1) Before I add the code, should I first install pygaze separately from here (http://www.pygaze.org/installation/)

As of 3.1, OpenSesame comes with a full Python environment, based on Anaconda (or actually MiniConda). You'll see the Python interpreter python.exe in the OpenSesame program folder. So even if you're using PyGaze and/ or PsychoPy directly from a script (and not using OpenSesame), you can still do this within OpenSesame's Python environment. And the version of PyGaze that it contains is much more recent than the one that you'll find on the PyGaze website.

Agata · August 2016

Hi again,

I was trying to run the code from PsychoPy importing the pygaze like Edwin suggested but I have encountered some problems already at that stage. I constantly get an ImportError message: no module named pygaze. When I got it for the first time then I thought that maybe I should try with installing pygaze separately but it doesn't work either.

But maybe I shouldn't be running it FROM PsychoPy at all? Does it make a difference? I tried to copy paste the code to the Python interpreter but I got only errors...

Would it be possible to get somewhere a short step by step guide of how to run a PsychoPy/PyGaze with Tobii? I mean, starting at the very installation and what goes where? Is there any user manual for pygaze? I cannot find anything online that would help me, I did look at the example codes etc., but I still don't know how to make it work with PsychoPy (help!).

Agata

Agata · August 2016

Update: I was also trying to run just the example pygaze experiments (e.g. simple_tracker_experiment or simple_experiment) and they give me the same error: no module named pygaze.

Edwin · August 2016

Hi Agata,

To answer your questions (numbers correspond to your questions):

1) You should use an actual installation of Python, and not the PsychoPy Builder. I would recommend using Anaconda (a Python distribution that is very complete). It doesn't contain PsychoPy, pyglet, PyGame, or PyGaze yet. But you can add those by using the following command in the Command Prompt (or a Terminal, if you're on Linux or OS X):

pip install pyglet psychopy python-pygaze

And then install PyGame according to these instructions.

2) Yes, you would need to then add the Tobii library.

3) After doing steps 1 and 2, you should have a Python environment outside of PsychoPy. You shouldn't use the PsychoPy coder (which is restricted to the PyschoPy environment), but for example Spyder. This is a code editor.

4) There isn't a quick-and-easy way of doing it without knowing a bit of coding, no. And you can always define AOIs post-hoc, provided you have the data AND you logged the image names and onsets/offsets to the gaze data!

Your later errors come from point 1: You didn't install PyGaze within the PsychoPy Python environment.

Cheers,
Edwin

Agata · August 2016

Hi again,

I am now using Anaconda as you suggested and Spyder as an editor, and the experiment itself is working fine. However, when I try to add the pygaze lines to the code I get an error about the display (when trying to run (from pygaze.display import Display: No module named display.

Do you have any ideas about how to fix that?

Best,
Agata

Daniel · August 2016

Just as a sidenote to Edwin's post, we have created real anaconda packages for these modules (though for pyglet and psychopy they might not always be the latest versions). You can also simply do

conda install pyglet pygame psychopy python-pygaze -c cogsci

The -c cogsci flag instructs conda to also look for packages on our cogsci channel.

Agata · August 2016

Thanks for that, dschreij!

Also, now the display and eyetracker import seems to work fine but the code stops again when it tries to import Image from libtobii:

File "C:\Anaconda2\lib\site-packages\pygaze_eyetracker\libtobii.py", line 66, in
import Image

ImportError: No module named Image

Why can't it find the Image module?

Best,
Agata

Daniel · August 2016

Hi Agata,
Image is part of the Python Image Library. You should be able to install it with

conda install pillow

Agata · August 2016

Hi,

Yes, the PIL and pillow packages were already there in the site packages (to double check that I ran the install command as you suggested and just got the message that it is already installed).

I get the error when libtobii.py code (below) is running and it stops at the line "import Image":

TobiiController

from tobii.eye_tracking_io.basic import EyetrackerException

import os
import datetime

import tobii.eye_tracking_io.mainloop
import tobii.eye_tracking_io.browsing
import tobii.eye_tracking_io.eyetracker
import tobii.eye_tracking_io.time.clock
import tobii.eye_tracking_io.time.sync

from tobii.eye_tracking_io.types import Point2D, Blob

import psychopy.visual
import psychopy.event

import Image
import ImageDraw

Could it be something with the tobii SDK? Or do I maybe need a different version of PIL or pillow? Maybe it is a stupid question, but does it know that it should import from PIL? Later in the code that comes from my PsychoPy experiment I do have a line: "from PIL import Image", but here (libtobii) it doesn't say from where.

Best,
Agata

sebastiaan · August 2016

Maybe it is a stupid question, but does it know that it should import from PIL?

That's correct. These direct imports are no longer valid; they were essentially a hack in older versions of PIL. It should be:

from PIL import Image
from PIL import ImageDraw

@Edwin Search-replace all these deprecated imports?

Edwin · August 2016

Fixed with this commit. Importing PIL is now the same across all PyGaze modules.

sebastiaan · August 2016

Good I'll push an update to PyPi and Anaconda later today.

Agata · August 2016

All the imports work fine now! Thanks for your help.

I seem to have some problem with the active window now, or the display. I added the piece of code that Edwin suggested:

Initialise a new Display instance

disp = Display(disptype='psychopy', dispsize=(1920,1080))

Get the handle to the active Window

win = pygaze.expdisplay

I have also changed in my code wherever it refered to the "window" to "win" but I only get a blank grey screen after putting in participant number and clicking ok (instead of getting the instructions that should follow). I maybe did something wrong with changing all "window" instances to "win"? Or is there something else that could be causing it?

Agata · September 2016

Hi again,

It took a while but I got my experiment to work and communicate with the Tobii eye tracker and the gaze data gets successfully recorded (wohoo!). But I have some new set of questions:

1) My output gives me just gazepoints (x, y), is it possible to get the eye movement type (fixation/saccade), the dwell time, pupil size etc? Should I define that somewhere in the code?

2) I was already asking about logging the events but I actually still don't know how to do that: are there pre-defined functions, for example for logging the start of the trial I should always use 'start_trial' and the screen flip will be always 'screen_flip' or 'win_flip' or something like that? Or should I use it as defined in my code? I would need the trial start, trial end and the keypress to be logged with the gaze data. It would be also nice to get the stimulus name logged together with that, but is it possible to get it logged as an event that will end up in the Events column in the output? Or in a whole separate column?

Thanks for any help!

Agata

Agata · September 2016

Hi,

I would really appreciate any help with the issues above! I was trying to deal with nr 1) and set the event detection to native, but is it supported for Tobii? It doesn't seem to change anything, but maybe I am not using the right command?

In nr 2) I got to log the events but they don't appear in the same rows as the gaze point coordinates but at the bottom of the log file, do I have to assign events like trial start etc to gaze coordinates post hoc based on timestamps when I will be analysing the data, or is there a way to make them appear in the same rows so that it's clearer which gaze coordinates belong to which trials?

Agata

Edwin · September 2016

Hi Agata,

Sorry, haven't been on the forum in a bit. Happy you got it running! As for your remaining questions:

1) It's not common practice to trust in your eye tracker. One would usually run an algorithm for saccade and fixation detection on the collected data. There is quite a bit of literature on this, but I would recommend you start with Engbert & Kliegl (2003), and Nystrom and Holmqvist (2010).

2) That depends on what analysis software you use. In general, no fixed keywords should be required, but some software does expect the logs to be formatted in a certain way. Choose what software you'd like to use, and then decide on how you'll format your log messages.

Cheers,
Edwin

Howdy, Stranger!

Categories

Running PsychoPy code in OpenSesame for eye-tracking

Comments

TobiiController

Initialise a new Display instance

Get the handle to the active Window

Howdy, Stranger!

Quick Links

Categories

Running PsychoPy code in OpenSesame for eye-tracking

Comments

TobiiController

Initialise a new Display instance

Get the handle to the active Window