Type Error in Pupil Reconstruction

Weizhe_Li

Hello,
I'm processing the data collected by eyelink 1000+, and translated the data to .csv file. Here's the preprocessing procedure:
1. Read .csv data

2. Then I translate the dtype of pupil data from object to float.

3. Write to datamatrix and do pupil reconstruction but TypeEorror shown up

Error Log:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-99-1398dfa59801> in <module>
      1 dm = DataMatrix(length=1500)
      2 dm.pupil = df['LEFT_PUPIL_SIZE'][1359:2859]
----> 3 dm.recon = blinkreconstruct(dm.pupil, smooth_winlen=21)

~/miniconda3/lib/python3.7/site-packages/datamatrix/_datamatrix/_datamatrix.py in __setattr__(self, name, value)
    571                         self._set_default_col_type(value)
    572                         return
--> 573                 self._set_col(name, value)
    574 
    575         def __delitem__(self, value):

~/miniconda3/lib/python3.7/site-packages/datamatrix/_datamatrix/_datamatrix.py in _set_col(self, name, value)
    472                 if name not in self:
    473                         self._cols[name] = self._default_col_type(self)
--> 474                 self._cols[name][:] = value
    475                 self._mutate()
    476 

~/miniconda3/lib/python3.7/site-packages/datamatrix/_datamatrix/_basecolumn.py in __setitem__(self, key, value)
    820                         self._setintkey(key, value)
    821                 elif isinstance(key, slice):
--> 822                         self._setslicekey(key, value)
    823                 elif isinstance(key, collections.Sequence):
    824                         self._setsequencekey(key, value)

~/miniconda3/lib/python3.7/site-packages/datamatrix/_datamatrix/_basecolumn.py in _setslicekey(self, key, value)
    521                         return
    522                 length = len(self._seq[key])
--> 523                 self._seq[key] = self._tosequence(value, length)
    524 
    525         def _setsequencekey(self, key, val):

~/miniconda3/lib/python3.7/site-packages/datamatrix/_datamatrix/_basecolumn.py in _tosequence(self, value, length)
    388                 if len(value) != length:
    389                         raise Exception('Sequence has incorrect length: %s' % len(value))
--> 390                 return [self._checktype(cell) for cell in value]
    391 
    392         def _getintkey(self, key):

~/miniconda3/lib/python3.7/site-packages/datamatrix/_datamatrix/_basecolumn.py in <listcomp>(.0)
    388                 if len(value) != length:
    389                         raise Exception('Sequence has incorrect length: %s' % len(value))
--> 390                 return [self._checktype(cell) for cell in value]
    391 
    392         def _getintkey(self, key):

~/miniconda3/lib/python3.7/site-packages/datamatrix/_datamatrix/_basecolumn.py in _checktype(self, value)
    326                         return value
    327                 if not isinstance(value, BASESTRING_OR_NUMBER):
--> 328                         raise TypeError('Invalid type: %s' % value)
    329                 if fastnumbers:
    330                         return self._checktype_fastnumber(value)

TypeError: Invalid type: 1599

What can I do to fix this bug, please?

Sincerely
Weizhe Li

Weizhe_Li

Same Error shown on another trial:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-94-e932f80a10c7> in <module>
     62 for file in os.listdir("Report//Sample"):
     63     if file.split('.')[1] == 'csv':
---> 64         write(file)

<ipython-input-94-e932f80a10c7> in write(data)
     37                 print(df[eye_use + '_PUPIL_SIZE'][time[0] : time[1]])
     38                 dm.base = df[eye_use + '_PUPIL_SIZE'][time[0] : time[1]]
---> 39                 dm.recon = blinkreconstruct(dm.base, maxdur = 1000, smooth_winlen=21)
     40                 reconstruct = series.downsample(dm.recon, by=2)
     41                 all_trail[trail][:749] = reconstruct

~/miniconda3/lib/python3.7/site-packages/datamatrix/_datamatrix/_datamatrix.py in __setattr__(self, name, value)
    571                         self._set_default_col_type(value)
    572                         return
--> 573                 self._set_col(name, value)
    574 
    575         def __delitem__(self, value):

~/miniconda3/lib/python3.7/site-packages/datamatrix/_datamatrix/_datamatrix.py in _set_col(self, name, value)
    472                 if name not in self:
    473                         self._cols[name] = self._default_col_type(self)
--> 474                 self._cols[name][:] = value
    475                 self._mutate()
    476 

~/miniconda3/lib/python3.7/site-packages/datamatrix/_datamatrix/_basecolumn.py in __setitem__(self, key, value)
    820                         self._setintkey(key, value)
    821                 elif isinstance(key, slice):
--> 822                         self._setslicekey(key, value)
    823                 elif isinstance(key, collections.Sequence):
    824                         self._setsequencekey(key, value)

~/miniconda3/lib/python3.7/site-packages/datamatrix/_datamatrix/_basecolumn.py in _setslicekey(self, key, value)
    521                         return
    522                 length = len(self._seq[key])
--> 523                 self._seq[key] = self._tosequence(value, length)
    524 
    525         def _setsequencekey(self, key, val):

~/miniconda3/lib/python3.7/site-packages/datamatrix/_datamatrix/_basecolumn.py in _tosequence(self, value, length)
    388                 if len(value) != length:
    389                         raise Exception('Sequence has incorrect length: %s' % len(value))
--> 390                 return [self._checktype(cell) for cell in value]
    391 
    392         def _getintkey(self, key):

~/miniconda3/lib/python3.7/site-packages/datamatrix/_datamatrix/_basecolumn.py in <listcomp>(.0)
    388                 if len(value) != length:
    389                         raise Exception('Sequence has incorrect length: %s' % len(value))
--> 390                 return [self._checktype(cell) for cell in value]
    391 
    392         def _getintkey(self, key):

~/miniconda3/lib/python3.7/site-packages/datamatrix/_datamatrix/_basecolumn.py in _checktype(self, value)
    326                         return value
    327                 if not isinstance(value, BASESTRING_OR_NUMBER):
--> 328                         raise TypeError('Invalid type: %s' % value)
    329                 if fastnumbers:
    330                         return self._checktype_fastnumber(value)

TypeError: Invalid type: 930

Weizhe_Li

Problem solved after I translated pupil area to diameter(mm), I guess probably because the area data with int type is unsuitable for the blinkreconstruct function

sebastiaan

Hi,

This was kind of a bug in the sense that DataMatrix didn't (in this context) accept the numpy datatypes as numbers. I just pushed a bugfix release (0.9.10) which fixes this.

Other than that, your approach works but doesn't make use of one of most convenient aspects of DataMatrix, namely that you can use a SeriesColumn object to represent an entire pupil trace as one
'deep' cell. The advantage of doing this is that you can stick to a one-row-per-trial structure.

Here's how you could accomplish the same thing using a SeriesColumn:

from datamatrix import DataMatrix, SeriesColumn, series as srs

# This assumes that you have a pandas.DataFrame with a pupil column
dm = DataMatrix(length=1)
dm.pupil = SeriesColumn(depth=len(df))
dm.pupil[0] = df.pupil
dm.recon = srs.blinkreconstruct(dm.pupil)

Cheers,
Sebastiaan