Experiment data from file

ejs

Dear Developers,

i've got a strange issue of OpenSesame 3.1.4 running Windows x64 system and Pyhon 3.5:
The file is UTF-8 encoded, without BOM.

Stopped

The experiment did not finish normally for the following reason:

Failed to read text file (perhaps it has the wrong format or it is not utf-8 encoded): C:\Users\ejs\AppData\Local\Temp\tmp7i7ocxp9.opensesame_pool\Seghier-VU.csv
Details

item-stack: experiment[run].block_sequence[run].block_loop[run]
exception message: 'charmap' codec can't decode byte 0xa1 in position 248: character maps to
time: Tue Jan 24 19:56:23 2017
exception type: UnicodeDecodeError
Traceback (also in debug window)

File "X:\Install\OpenSesame\3.1.4-py3.5-win64\lib\site-packages\libopensesame\loop.py", line 209, in _create_live_datamatrix
src_dm = io.readtxt(src)
File "X:\Install\OpenSesame\3.1.4-py3.5-win64\lib\site-packages\python_datamatrix-0.3.1-py2.7.egg\datamatrix\io_text.py", line 49, in readtxt
for column in next(reader):
File "X:\Install\OpenSesame\3.1.4-py3.5-win64\lib\encodings\cp1257.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0xa1 in position 248: character maps to

Anything can be fixed by myself?
I've got similar error when tweking experiment file in "Programmers Notepd2" - Opensesame expect 8-byte file instead of Unicode and refuses to open the experiment if it is saved in UTF-8, with or without BOM.

sebastiaan

Hi,

Thanks for pointing this out. This appears to be a Python-3 specific issue, in which the Python csv.reader tries to decode the file using the default system encoding. In your case this is cp1257, a Windows encoding for Baltic script.

For now, I would convert the file to .xlsx format, if possible. You could also convert the encoding to cp1257, but I would recommend against it, because this workaround will break when the bug is fixed, and it won't work on Python 2.

Cheers,
Sebastiaan

sebastiaan

Hi,

Would you mind upgrading datamatrix (the Python package used by the loop item) to see if the issue is fixed now? That is, if you can now read utf-8 encoded csv files.

You can upgrade by entering the following in the debug window:

import pip
pip.main(['install', 'python-datamatrix', '--upgrade'])
pip.main(['install', 'fastnumbers']) # optional new dependency

If the upgrade was succesful, Tools → System Information should report that datamatrix 0.4.3 is installed.

Cheers,
Sebastiaan

ejs

Hi Sebastiaan,

seems the error is gone. Thank you for the solution.

Could you have a look at the attached file? 3.1.4 x64 Python 3.5.2
It's not Unicode, isn't it?

sebastiaan

Right, that's actually the same Python-3 specific bug, but then in OpenSesame itself. Would you mind trying if the problem is solved with the latest prerelease. On Windows, you cannot upgrade OpenSesame from within OpenSesame (because Windows locks the files), but you can navigate to the Scripts folder and run:

pip.exe install python-opensesame --upgrade

Or download the latest prerelease from here (now 3.1.5a5):

• http://files.cogsci.nl/software/opensesame/pre-releases/