Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Quick Links

Categories

Supported by
Bestel Een wereld vol denkers door Sebastiaan Mathôt via bol.com

Using DataMatrix with Tobii_Pro eyetracker?

alexkarousoualexkarousou
in DataMatrix

Many thanks to @sebastiaan for contributing another excellent tool!

My question: Has anyone used Data Matrix for processing/analyzing data from a Tobii eyetracker?

In order to be able to compare the Tobii vs. Eyelink output files, I tried to read the Eyelink edf files of the semantic_pupil experiment https://github.com/smathot/semantic_pupil/tree/master/analysis, with no luck. I cannot convert the .edf files to .asc. Could someone, perhaps, upload one as .asc (or other readable format)?

I am also attaching two screenshots of a Tobii X3-120 output .tsv file.

Note that missing (-1) values in pupil size are due to Tobii X3-120 reporting pupil size only in bright pupil mode which is used only every third sample. Thus although this model records eye gaze at 120Hz, it records pupil size at only 40Hz -as this is only a pilot study, I guess this is not too important.


Tobii Pro X3-120 output 1.jpg
Tobii Pro X3-120 output 2.jpg


Any advice would be more than welcome!

Best,

Alexandra

Comments

  • sebastiaansebastiaan

    Hi Alexandra,

    Here's a link to one of the datafiles from the semantic-pupil experiment, converted to .asc. This is done with a utility called edf2asc, which you can download for free from the SR Research forum. (But this does require you to register on their forum.)

    It should be straightforward to build an adapter for the EyeLinkParser such that it can read the .tsv files above. (Similar to how this is done for the EyeTribeParser.) Could you upload one complete datafile so that I can see what it looks like exactly?

    Cheers!

    Sebastiaan


    SigmundAI
    Check out SigmundAI.eu for our OpenSesame AI assistant!

  • alexkarousoualexkarousou

    Hi Sebastiaan,

    Thank you very much for your imediate reply and the asc output file!

    I will have study it, as well as the EyeTribeParser...

    Here you can find a Tobii_Pro output file https://www.dropbox.com/s/760i00vmabelsao/subject-101-3_TOBII_output.tsv?dl=0 (it wouldn't let me upload it here).

    Thanks again for everything.

    Best wishes,

    Alexandra

  • melannimelanni

    Hello Sebastiaan,

    Thanks for the useful and open-source toolbox.

    I'm also using data from Tobii, however I have the data en txt format.

    Since the name of the variables are different, my question is how can I rename the variables in order to match with my variables names?

    For example, the first error I get: No column named "set_size"

    I imagine that set_size refers to the pipul width of every trial?

    Then I guess I need to rename the variable set_size for PupilWidth as in my file

    Thank so much in advance!

  • sebastiaansebastiaan
    edited March 2020

    Hi Melanni,

    It's not clear to me what you're doing exactly. It sounds like you're using some script to parse some data files and that something goes wrong. But you'll have to be a lot more specific for us to be able to help you! What are you doing exactly?

    Cheers!

    Sebastiaan


    SigmundAI
    Check out SigmundAI.eu for our OpenSesame AI assistant!

  • melannimelanni

    Hi Sebastiaan,

    Thanks for your response, I'm sorry if I was not clear.

    I have eye-tracker data from Tobii Pro Glasses (I attached the file of one subject).

    I just want to preprocesses the pupil-size signal (column PupilWidth) by getting rid of the blinks, smooth the signal, and all the necessary steps for later on use this signal as a regressor for fmri data.

    As I understood this is a toolbox able to perform such analysis with my data..?

    Cheers!

    Melanni


  • sebastiaansebastiaan

    Hi Melanni,

    The EyeLinkParser parser a variety of formats, but not this one. So no, you cannot really use it, not without extensive modifications to the code. You could load the data into a DataMatrix object and then use series.blinkreconstruct() and other functions to process the data, and then save it again.

    However, this will require some knowledge of numerical computing with Python, and the nature of your questions suggests to me that you don't have this at the moment. Is there someone in your lab who can help you with this?

    Cheers!

    Sebastiaan


    SigmundAI
    Check out SigmundAI.eu for our OpenSesame AI assistant!

  • wdhanywdhany

    Hi @sebastiaan,

    I'm also trying to parse .tsv data from Tobii (tobii fusion pro) using your eyelinkparser module. If I try to parse my data directly using the script you provided here, I get this error:

    Traceback (most recent call last):


  File "C:\Users\psychology.BINUS\AppData\Local\Temp\ipykernel_10976\1397237029.py", line 20, in <cell line: 20>
    dm = get_data()


  File "C:\Users\psychology.BINUS\Anaconda3\envs\opensesame-py3\lib\site-packages\datamatrix\_functional\_memoize.py", line 230, in _call_without_arguments
    self._fnc(*args, **kwargs)


  File "C:\Users\psychology.BINUS\AppData\Local\Temp\ipykernel_10976\1397237029.py", line 15, in get_data
    dm.subject_nr, dm.count_retention_trial_sequence, dm.count_test_trial_sequence, dm.phase, dm.size, dm.correct,dm.test_key_time, dm.ptrace_stim_cue, dm.ptrace_stim_show,


  File "C:\Users\psychology.BINUS\Anaconda3\envs\opensesame-py3\lib\site-packages\datamatrix\_datamatrix\_datamatrix.py", line 678, in __getattr__
    return self._getcolbyname(name)


  File "C:\Users\psychology.BINUS\Anaconda3\envs\opensesame-py3\lib\site-packages\datamatrix\_datamatrix\_datamatrix.py", line 458, in _getcolbyname
    raise AttributeError(u'No column named "%s"' % key)


AttributeError: No column named "subject_nr"

    Any advice on what went wrong and how should I continue? I also attached an example of the data from one participant.


    Cheers,

    Wisnu

  • sebastiaansebastiaan

    Hi @wdhany ,

    This data format, which is non-standard, is simply not supported by EyeLinkParser, or at least not directly.

    In order for this to work, you need to subclass the EyeLinkParser and (mainly) re-implement the split() function such that it takes a line from the data file in your custom format and returns it in the format expected by the EyeLinkParser for further processing. Once that's done, the rest of the parsing will happen automatically.

    This is not as complex as it sounds, and something like the code below should do the trick. Do carefully check whether the results make sense though, because this code has not really been tested!

    — Sebastiaan

    class TobiiParser(EyeLinkParser):

    def __init__(self, **kwargs):
        if u'ext' not in kwargs:  # accept tsv files
            kwargs[u'ext'] = [u'.tar.xz', u'.tsv']
        super().__init__(**kwargs)       

    def is_message(self, line):
        # Recognize message lines. This is inefficient, because we're splitting
        # each line twice
        return self.split(line)[0] == 'MSG'

    def split(self, line):
        l = super().split(line)
        if not l:
            return l
        if isinstance(l[0], str):
            return l
        # Convert to message
        # From: [7358.476, start_trial, 0]
        # To: [MSG, 7358, "start_trial", 0]
        if len(l) != 13:
            l = ['MSG', int(l[0])] + l[1:]
            return l
        # Convert to sample
        # From: See heading in data file for column meanings
        # To: [timestamp, x, y, pupilsize, '...']
        x = l[8]
        y = l[9]
        ps = .5 * (l[10] + l[12])
        return [l[0], x, y, ps, '...']


dm = TobiiParser(folder='tobii-data').dm


    SigmundAI
    Check out SigmundAI.eu for our OpenSesame AI assistant!

  • wdhanywdhany

    Hi @sebastiaan,

    I naively add the subclass you made above in between importing the eyelinkparser and defining the get_data() function, but it then returned an empty datamatrix. Here's the overall script, can you tell me what is missing? Thanks in advance!

    from datamatrix import (
  operations as ops,
  functional as fnc,
  series as srs
)
from eyelinkparser import parse, defaulttraceprocessor, EyeLinkParser

from prettytable import PrettyTable

class TobiiParser(EyeLinkParser):


    def __init__(self, **kwargs):
        if u'ext' not in kwargs:  # accept tsv files
            kwargs[u'ext'] = [u'.tar.xz', u'.tsv']
        super().__init__(**kwargs)       


    def is_message(self, line):
        # Recognize message lines. This is inefficient, because we're splitting
        # each line twice
        return self.split(line)[0] == 'MSG'


    def split(self, line):
        l = super().split(line)
        if not l:
            return l
        if isinstance(l[0], str):
            return l
        # Convert to message
        # From: [7358.476, start_trial, 0]
        # To: [MSG, 7358, "start_trial", 0]
        if len(l) != 13:
            l = ['MSG', int(l[0])] + l[1:]
            return l
        # Convert to sample
        # From: See heading in data file for column meanings
        # To: [timestamp, x, y, pupilsize, '...']
        x = l[8]
        y = l[9]
        ps = .5 * (l[10] + l[12])
        return [l[0], x, y, ps, '...']


dm = TobiiParser(folder='data').dm


@fnc.memoize(persistent=True)
def get_data():


    # The heavy lifting is done by eyelinkparser.parse()
    dm = parse(
        folder='data', # Folder with .asc files
        traceprocessor=defaulttraceprocessor(
          blinkreconstruct=True, # Interpolate pupil size during blinks
          #downsample=10 # Reduce sampling rate to 100 Hz
          mode='advanced'
        )
    )
    # To save memory, we keep only a subset of relevant columns.
    #dm = ops.keep_only(
    #    dm,
    #    dm.subject_nr, dm.count_retention_trial_sequence, dm.count_test_trial_sequence, dm.phase, dm.size, dm.correct,dm.test_key_time, dm.ptrace_stim_cue, dm.ptrace_stim_show,
    #    dm.ptrace_test_cue, dm.ptrace_test_show
    #)
    return dm


dm = get_data()
print(dm.column_names)
print(dm)
  • sebastiaansebastiaan

    Hi @wdhany ,

    One problem at least is that you're first reading the data with the custom TobiiParser class:

    dm = TobiiParser(folder='data').dm

    And then again with the default parse() function, thus replacing the datamatrix with an empty one:

    dm = get_data()

    You should do either the one or the other, but not both. The most elegant way is probable to pass TobiiParser as a custom parser class to the parse() function:

        dm = parse(
        folder='data', # Folder with .asc files
        parser=TobiiParser,
        traceprocessor=defaulttraceprocessor(
          blinkreconstruct=True, # Interpolate pupil size during blinks
          #downsample=10 # Reduce sampling rate to 100 Hz
          mode='advanced'
        )
    )

    — Sebastiaan


    SigmundAI
    Check out SigmundAI.eu for our OpenSesame AI assistant!

Sign In or Register to comment.
Sebastiaan Mathôt » v2.0.18 » Copyright 2011 — 2019 » Vanilla flavored » License