Using DataMatrix with Tobii_Pro eyetracker?

Comments

• sebastiaan

  July 2019

  Hi Alexandra,

  Here's a link to one of the datafiles from the semantic-pupil experiment, converted to .asc. This is done with a utility called edf2asc, which you can download for free from the SR Research forum. (But this does require you to register on their forum.)

  • http://files.cogsci.nl/tmp/sub3.asc

  It should be straightforward to build an adapter for the EyeLinkParser such that it can read the .tsv files above. (Similar to how this is done for the EyeTribeParser.) Could you upload one complete datafile so that I can see what it looks like exactly?

  Cheers!

  Sebastiaan

  
  
  Check out SigmundAI.eu for our OpenSesame AI assistant!
  
• alexkarousou

  July 2019

  Hi Sebastiaan,

  Thank you very much for your imediate reply and the asc output file!

  I will have study it, as well as the EyeTribeParser...

  Here you can find a Tobii_Pro output file https://www.dropbox.com/s/760i00vmabelsao/subject-101-3_TOBII_output.tsv?dl=0 (it wouldn't let me upload it here).

  Thanks again for everything.

  Best wishes,

  Alexandra
• melanni

  March 2020

  Hello Sebastiaan,

  Thanks for the useful and open-source toolbox.

  I'm also using data from Tobii, however I have the data en txt format.

  Since the name of the variables are different, my question is how can I rename the variables in order to match with my variables names?

  For example, the first error I get: No column named "set_size"

  I imagine that set_size refers to the pipul width of every trial?

  Then I guess I need to rename the variable set_size for PupilWidth as in my file

  Thank so much in advance!
• sebastiaan

  March 2020 edited March 2020

  Hi Melanni,

  It's not clear to me what you're doing exactly. It sounds like you're using some script to parse some data files and that something goes wrong. But you'll have to be a lot more specific for us to be able to help you! What are you doing exactly?

  Cheers!

  Sebastiaan

  
  
  Check out SigmundAI.eu for our OpenSesame AI assistant!
  
• melanni

  March 2020

  Hi Sebastiaan,

  Thanks for your response, I'm sorry if I was not clear.

  I have eye-tracker data from Tobii Pro Glasses (I attached the file of one subject).

  I just want to preprocesses the pupil-size signal (column PupilWidth) by getting rid of the blinks, smooth the signal, and all the necessary steps for later on use this signal as a regressor for fmri data.

  As I understood this is a toolbox able to perform such analysis with my data..?

  Cheers!

  Melanni

  suj200.txt

  
  
• sebastiaan

  April 2020

  Hi Melanni,

  The EyeLinkParser parser a variety of formats, but not this one. So no, you cannot really use it, not without extensive modifications to the code. You could load the data into a DataMatrix object and then use series.blinkreconstruct() and other functions to process the data, and then save it again.

  • https://datamatrix.cogsci.nl/0.10/series/

  However, this will require some knowledge of numerical computing with Python, and the nature of your questions suggests to me that you don't have this at the moment. Is there someone in your lab who can help you with this?

  Cheers!

  Sebastiaan

  
  
  Check out SigmundAI.eu for our OpenSesame AI assistant!
  
• wdhany

  August 2022

  Hi @sebastiaan,

  I'm also trying to parse .tsv data from Tobii (tobii fusion pro) using your eyelinkparser module. If I try to parse my data directly using the script you provided here, I get this error:

  Traceback (most recent call last):
  
  
    File "C:\Users\psychology.BINUS\AppData\Local\Temp\ipykernel_10976\1397237029.py", line 20, in <cell line: 20>
      dm = get_data()
  
  
    File "C:\Users\psychology.BINUS\Anaconda3\envs\opensesame-py3\lib\site-packages\datamatrix\_functional\_memoize.py", line 230, in _call_without_arguments
      self._fnc(*args, **kwargs)
  
  
    File "C:\Users\psychology.BINUS\AppData\Local\Temp\ipykernel_10976\1397237029.py", line 15, in get_data
      dm.subject_nr, dm.count_retention_trial_sequence, dm.count_test_trial_sequence, dm.phase, dm.size, dm.correct,dm.test_key_time, dm.ptrace_stim_cue, dm.ptrace_stim_show,
  
  
    File "C:\Users\psychology.BINUS\Anaconda3\envs\opensesame-py3\lib\site-packages\datamatrix\_datamatrix\_datamatrix.py", line 678, in __getattr__
      return self._getcolbyname(name)
  
  
    File "C:\Users\psychology.BINUS\Anaconda3\envs\opensesame-py3\lib\site-packages\datamatrix\_datamatrix\_datamatrix.py", line 458, in _getcolbyname
      raise AttributeError(u'No column named "%s"' % key)
  
  
  AttributeError: No column named "subject_nr"
  

  Any advice on what went wrong and how should I continue? I also attached an example of the data from one participant.

  subject-1_TOBII_output.tsv

  
  

  Cheers,

  Wisnu
• sebastiaan

  August 2022

  Hi @wdhany ,

  This data format, which is non-standard, is simply not supported by EyeLinkParser, or at least not directly.

  In order for this to work, you need to subclass the EyeLinkParser and (mainly) re-implement the split() function such that it takes a line from the data file in your custom format and returns it in the format expected by the EyeLinkParser for further processing. Once that's done, the rest of the parsing will happen automatically.

  This is not as complex as it sounds, and something like the code below should do the trick. Do carefully check whether the results make sense though, because this code has not really been tested!

  — Sebastiaan

  class TobiiParser(EyeLinkParser):
  
      def __init__(self, **kwargs):
          if u'ext' not in kwargs:  # accept tsv files
              kwargs[u'ext'] = [u'.tar.xz', u'.tsv']
          super().__init__(**kwargs)       
  
      def is_message(self, line):
          # Recognize message lines. This is inefficient, because we're splitting
          # each line twice
          return self.split(line)[0] == 'MSG'
  
      def split(self, line):
          l = super().split(line)
          if not l:
              return l
          if isinstance(l[0], str):
              return l
          # Convert to message
          # From: [7358.476, start_trial, 0]
          # To: [MSG, 7358, "start_trial", 0]
          if len(l) != 13:
              l = ['MSG', int(l[0])] + l[1:]
              return l
          # Convert to sample
          # From: See heading in data file for column meanings
          # To: [timestamp, x, y, pupilsize, '...']
          x = l[8]
          y = l[9]
          ps = .5 * (l[10] + l[12])
          return [l[0], x, y, ps, '...']
  
  
  dm = TobiiParser(folder='tobii-data').dm
  

  
  
  Check out SigmundAI.eu for our OpenSesame AI assistant!
  
• wdhany

  September 2022

  Hi @sebastiaan,

  I naively add the subclass you made above in between importing the eyelinkparser and defining the get_data() function, but it then returned an empty datamatrix. Here's the overall script, can you tell me what is missing? Thanks in advance!

  from datamatrix import (
    operations as ops,
    functional as fnc,
    series as srs
  )
  from eyelinkparser import parse, defaulttraceprocessor, EyeLinkParser
  
  from prettytable import PrettyTable
  
  class TobiiParser(EyeLinkParser):
  
  
      def __init__(self, **kwargs):
          if u'ext' not in kwargs:  # accept tsv files
              kwargs[u'ext'] = [u'.tar.xz', u'.tsv']
          super().__init__(**kwargs)       
  
  
      def is_message(self, line):
          # Recognize message lines. This is inefficient, because we're splitting
          # each line twice
          return self.split(line)[0] == 'MSG'
  
  
      def split(self, line):
          l = super().split(line)
          if not l:
              return l
          if isinstance(l[0], str):
              return l
          # Convert to message
          # From: [7358.476, start_trial, 0]
          # To: [MSG, 7358, "start_trial", 0]
          if len(l) != 13:
              l = ['MSG', int(l[0])] + l[1:]
              return l
          # Convert to sample
          # From: See heading in data file for column meanings
          # To: [timestamp, x, y, pupilsize, '...']
          x = l[8]
          y = l[9]
          ps = .5 * (l[10] + l[12])
          return [l[0], x, y, ps, '...']
  
  
  dm = TobiiParser(folder='data').dm
  
  
  @fnc.memoize(persistent=True)
  def get_data():
  
  
      # The heavy lifting is done by eyelinkparser.parse()
      dm = parse(
          folder='data', # Folder with .asc files
          traceprocessor=defaulttraceprocessor(
            blinkreconstruct=True, # Interpolate pupil size during blinks
            #downsample=10 # Reduce sampling rate to 100 Hz
            mode='advanced'
          )
      )
      # To save memory, we keep only a subset of relevant columns.
      #dm = ops.keep_only(
      #    dm,
      #    dm.subject_nr, dm.count_retention_trial_sequence, dm.count_test_trial_sequence, dm.phase, dm.size, dm.correct,dm.test_key_time, dm.ptrace_stim_cue, dm.ptrace_stim_show,
      #    dm.ptrace_test_cue, dm.ptrace_test_show
      #)
      return dm
  
  
  dm = get_data()
  print(dm.column_names)
  print(dm)
  
• sebastiaan

  September 2022

  Hi @wdhany ,

  One problem at least is that you're first reading the data with the custom TobiiParser class:

  dm = TobiiParser(folder='data').dm
  

  And then again with the default parse() function, thus replacing the datamatrix with an empty one:

  dm = get_data()
  

  You should do either the one or the other, but not both. The most elegant way is probable to pass TobiiParser as a custom parser class to the parse() function:

      dm = parse(
          folder='data', # Folder with .asc files
          parser=TobiiParser,
          traceprocessor=defaulttraceprocessor(
            blinkreconstruct=True, # Interpolate pupil size during blinks
            #downsample=10 # Reduce sampling rate to 100 Hz
            mode='advanced'
          )
      )
  

  — Sebastiaan

  
  
  Check out SigmundAI.eu for our OpenSesame AI assistant!