Weird pygaze calibration reports (constant distance value & absurdly high acc values)

Comments

• hanzh

  April 2019

  Dear @Edwin

  I noticed the same problem as @ChrIm pointed out. For example, this is a calibration report on the Tobii T60:

  

  While I'm pretty sure that I was following the targets closely, the accuracy report seems to show otherwise. It is interesting that those values were so close to the center of the screen. I was wondering if the reports did not show the absolute deviation from the targets but some kind of average position of samples?

  
  

  Thanks,

  Han
• GEYUQING

  April 2019

  Hi everyone,

  I had the same problem using Tobii Spectrum, any advice would be helpful.

  Cheers,

  Ge
• Edwin

  June 2019 edited June 2019

  Hi all,

  Screen distance can only be picked up by systems that can. If yours can't, it will try to fall back on a user-defined value, or (in lieu of that) on the default of 57 cm. To define your screen distance, obviously measure it first , and then add this to your constants.py script:

  # Set the screen-eye distance to 60.5 cm
  SCREENDIST = 60.5
  

  The same holds true for the screen size, which should be user-defined. For example, if it's 40.3x30.0 cm, include this in your constants.py:

  SCREENSIZE = (40.3, 30.0)
  

  High values for "accuracy (in pixesl)" mean that your eye tracker thought you were looking at a different place on the screen than where the central marker was presented. One very common reason for this is that people don't set the correct display resolution. For example, if your display is 1024x768 pixels, but you set 800x600, the tracker will report in the 1024x768 frame whereas your experiment will display in the 800x600 frame. This will lead to mismatches. (The only exception to this when you use a different resolution to your display's, is when you set that different resolution in both the eye tracking software and your experimental software.) To set your resolution correctly, include the following in constants.py (obviously adjusted to your own resolution):

  # Set the resolution to 1024x768
  DISPSIZE = (1024, 768)
  

  Cheers,

  Edwin

  PS. VERY IMPORTANT: The values for the speed and acceleration thresholds are ONLY used when event detection is set to "pygaze". Hence, you will not notice any of these values being off if you don't use the "wait_for_*" event functions. Even if you do use them, you'll only be affected by these values being off if you're using the "pygaze" event detection as opposed to "native".
• hanzh

  June 2019

  Hi @Edwin ,

  Thank you for your reply! I have some follow-up questions. I'm now using the PyGaze plug-in in OpenSesame with EyeLink.

  1. I assume that DISPSIZE is automatically set as the experiment's resolution and I don't have to set it explicitly?
  2. Do any of the PyGaze settings (DISPSIZE, SCREENSIZE, SCREENDIST) override settings in the PHYSICAL.INI in the host PC? I have set them correctly in the .INI file but I'm not sure if they will be changed if I don't set them correctly in opensesame.
  3. Finally, if my screen is 1920*1280 but I run experiments in 1024*768, which one should be the correct DISPSIZE?

  Thanks a lot for your help!

  Han
• hanzh

  June 2019

  Perhaps @sebastiaan also knows answers to those questions above? :)
• Edwin

  June 2019

  1. Yes, the resolution you set in OpenSesame is passed as DISPSIZE to PyGaze.
  2. Nothing you set through OpenSesame or PyGaze changes the INI file, but they will be used as settings in PyGaze's computations. The INI file is not used for this.
  3. You should run your experiment in a 19020x1080 resolution if that is the resolution of your monitor. Alternatively, you could use a monitor with a 1024x768 resolution if you really mean to use 1024x768. (I don't really understand why you wouldn't use the native resolution?)
• Art

  July 2019

  Hi there,

  I'm using Tobii TX300, and apparently my Distance value in the log file is not getting updated by the calibration procedure. What should I do now?

  
  

  Art
• Art

  July 2019

  Just to clarify: calibration procedure for Tobii seems to detect the visual distance correctly, it's just that the calibration value in the eyetracker log file is not getting saved accordingly.
• Edwin

  July 2019

  Hi Art,

  If you're not using any of the implemented online event detection functions, you don't have to do anything.

  If you'd like for Tobii to implement the distance sensing, you could file a request on GitHub: https://github.com/esdalmaijer/PyGaze/issues (tag the Tobii devs on GitHub if you file an issue there: @grebdems and @pedrotari7)
• Art

  July 2019

  Hi Edwin,

  
  

  Thanks for the reply. I am confused: during calibration (the initial step, where Tobii positions subject's eyes in its bounding box), distance is estimated correctly. This value is simply not saved by pygaze in its log file. I wonder how to retrieve it and have it stored in the log file. I thought this is all done by pygaze?
• Edwin

  July 2019

  Hi Art,

  The question is whether it is saved and used later on. Examples:

  My SMI implementation in PyGaze estimates distance using the tracker, and later saves this to the log file, and uses it in event detection: https://github.com/esdalmaijer/PyGaze/blob/master/pygaze/_eyetracker/libsmi.py#L364

  The Tobii implementation takes the default value (https://github.com/esdalmaijer/PyGaze/blob/master/pygaze/_eyetracker/libtobii.py#L47), and it doesn't look like this is updated before being logged and used (https://github.com/esdalmaijer/PyGaze/blob/master/pygaze/_eyetracker/libtobii.py#L590). This is also true in my earlier Tobii implementation (tobii-legacy).

  I referred you to GitHub, because Tobii developers maintain the Tobii implementation in PyGaze. They would be much quicker to implement the functionality you're requesting. (Also, I don't have access to a Tobii, so wouldn't be able to test any changes I make.)

  Cheers,

  Edwin
• Art

  August 2019

  OK, I think we managed to dig into the appropriate portions of Pygaze code.

  
  

  Just one question: is there any way of making Pygaze detect saccades for Tobii output? I can't see how to enable it in Pygaze settings.
• Art

  August 2019

  Bump.

  I dug in libtobii.py code in the meantime, but still can not figure out how to make it log saccades as events in the Tobii log file. I mean, the saccade detection code in libtobii seems OK, I have eventdetection set to pygaze version, but it seems to not work. I am not a Python expert so can not really find the reason for this and check whether it detects saccades on-line and just not saves them or whether not saving results from saccades not being detected.

  
  

  I was also wondering, if there is any low-pass filtering implemented to the gaze data, as I see nothing like this in the code, apart from noise calibration parameters being applied to saccade detection/thresholds etc.
• Edwin

  September 2019

  As I said before, this stuff is really only for online event detection. I don't really see why you'd like to continuously try to detect events and log them to the data file. You can use much better event detection algorithms offline using the gaze samples.

  The same is true for filtering. Why would you want to do this during recording? While sensible for some analyses, ideally your raw data is as raw as possible, leaving you to do whatever you need offline.
• Art

  September 2019

  Right. Is there any code available for analysing saccade kinematics for Tobii output with Opensesame?

  Regarding filtering, I was basically wondering if what Tobii SDK outputs is already filtered but this seems to be not the case.

  But anyway, I also thought of using some on-line processing, for fixation control - do you know if there is anything available in these lines? I mean, code that could be implemented with Opensesame?
• Edwin

  September 2019 edited September 2019

  Not that I'm directly aware of, but there are more general toolboxes out there for eye movement analysis.

  Good question! There's likely some filtering going on, i.e. some smoothing, and there's also the head model they seem to use to compute samples. All proprietary implementation, though, so not sure whether we'll ever know exactly what filtering is going on. (Whether that's an issue highly depends on your actual research question, though.)

  You could use the built-in "wait_for_fixation_end" etc. functions, or do something like the following:

  # Example maximum deviation in pixels
  max_dev = 60
  # Example target coordinate (screen centre)
  target = (exp.get("width")//2, exp.get("height")//2)
  # Run until the deviation is higher than max_dev
  fixation = True
  while fixation:
      # Get a new sample
      gaze_pos = eyetracker.sample()
      # Ignore missing samples
      if gaze_pos == (None,None) or gaze_pos == (0,0) or gaze_pos == (-1,-1):
          continue
      # Compute distance to target
      d = ((gaze_pos[0]-target[0])**2 + (gaze_pos[1]-target[1])**2)**0.5
      # Break loop if the distance exceeds the maximum deviation
      if d > max_dev:
          fixation == False
  

  
  

  You might want to build in a timeout, and it might also be good to include a sample counter. (To break only if e.g. >5 samples were over max. deviation.)
• Art

  October 2019 edited October 2019

  Hi Edwin,

  
  

  Thanks! I think I will ask directly at Tobii whether they can provide this info about filtering. As soon as I have it I will post it here.

  
  

  Maybe it's a lame question, but can you recommend any toolboxes you know people have used for Tobii + Pygaze analysis? I tried to use GazeAlyze but it spits out a long list of errors, and it's probably no longer compatible with newer Matlab versions, but I haven't looked into it in detail.

  I was just wondering if it's more efficient to write my own tools or use something that's there already.
• Art

  October 2019

  Two more things:

  • where do I insert the code you suggested for on-line event detection? I am relatively new to Opensesame so I am not sure how to work with its modules.
  • when I run the Tobii tracker with pygaze_start_* and pygaze_stop_recording inserted at the beginning and end of each trial loop (to get reliable time markers for trial start and end), I noticed that IR lights of the tracker go off after each time stop_recording is called. I wonder if that's a bug or a feature and whether there was a way to ensure continuous illumination with Tobii.

  Thanks.
• eduard

  November 2019

  Hi,

  
  

  where do I insert the code you suggested for on-line event detection?

  Wherever you need it in your experiment. The code itself is put in an inline_script (if you use Opensesame), and should be put in the experimental sequence. Where exactly depends on your specific case.

  Does that help?

  Eduard

  
• Art

  November 2019

  Hi Eduard,

  Thanks, that sounds clear ?️
• Paula

  June 2021

  Hi! What would be "good" value for accuracy and precision? Is there any recommended threshold? Any reference would be much appreciated. Thanks in advance!
• eduard

  June 2021

  Not sure whether there are any clear rules. But I always went for an average of about 0.5, and ideally no single values higher than 1. Of course that depends a bit on participants, so if calibration accuracy couldn't be any better I also accepted averages of 1.

  If I am not mistaken, the Eyelink calls the calibration failed if there is at least one dot larger than 1.5 deviation.

  Probably best check out the manual of your eye tracker.

  Eduard