[open] Error in __init__ interrupts experiment

Kiri

Hi,

I got a problem with OpenSesame Inlinescript. It reads a csv-file (showing no errors or bugs in any of the read lines) but it can't present them. At the first presentation of a part of the file-content, it interrupts the experiment. Only message is "'int' object has no attribute 'decode'". Within Debug-window, the content is presented as it should be (tested in Prepare-phase. But in Run-phase, as soon as the presentation should start, this error message shows up and interrupts the experiment.

The csv-files are coded in utf-8 without BOM (checked through Notepad++). The content of some fields got cyrillic characters, which don't cause any troubles in Prepare-phase. So I think it might be a canvas-problem, but how do I solve it, if it's been read in the right way?

Thank you very much in advance.

Josh

Hi Kiri,

Yes, the utf-8 can be a bit problematic. Basically, the program assumes by default that your text is ASCII encoded, even if the text itself was coded in utf-8. In order to prevent OpenSesame from getting confused, you want to add something like "stimulus = stimulus.decode('utf-8')", before you call the stimulus variable in a canvas.text command.

Hope this helps!

Cheers,

Josh

sebastiaan

Hi Kiri,

In general, Josh is right that it's best to use unicode strings if your text contains special characters. If you decode str to unicode as soon as possible (e.g. directly after reading from a file) you usually avoid any encoding/ decoding trouble (which is troublesome in Python 2).

However, this general guideline doesn't appear to be directly related to your problem. It seems that your text is actually numeric, that is, an int object. Possibly, this results from OpenSesame's 'smart typing', as described here:

• http://osdoc.cogsci.nl/usage/variables-and-conditional-statements/#smart-variable-typing-and-some-pitfalls

Could you post the full traceback from the debug window + a more detailed description of your experiment + your version of OpenSesame and operating system?

Cheers,
Sebastiaan

Kiri

Hi Josh and sebastiaan,

thank you for the quick answers. Tomorrow I will have the possibility to test your solutions and post the debug window as well as the versions of OpenSesame. The operating system is Windows 7.

In the experiment, some sentences will be presented in russian. To know which sentence is which one, they got integer codes in other cells of the csv file within the line. At first, the lines are read, then partet into the cells. During this progress, I printed the content into the debug window. Everything works fine there, and is presented as it should be.
But in Run-Phase, when I design the rows by the title-line, as soon as the first sentence should be presented the error message appears.

Kiri

The version of OpenSesame is 2.9.x (current).
I tested it so far.

The first message of the debug window is:

Unexpected error

line: 68
exception message: 'int' object has no attribute 'decode'
exception type: AttributeError

Traceback:
  File "dist\libqtopensesame\misc\process.py", line 139, in run
  File "dist\libopensesame\experiment.py", line 294, in run
  File "dist\libopensesame\sequence.py", line 57, in run
  File "dist\libopensesame\inline_script.py", line 158, in run
  File "dist\libopensesame\exceptions.py", line 68, in __init__
AttributeError: 'int' object has no attribute 'decode'

Decoding the fields of the csv-file right after reading it didn't change this error message.

sebastiaan

So what's in the inline script? Around line 68 you appear to be decoding the int.

Kiri

That's exactly the problem. Within the experiment, neither in Prepare nor in Run-Phase are integers declared in line 68. They are decoded in Prepare around line 60.

Here the actual decoding code (from line 52 on):

    for row in lines:
        for field in row:
            print row[field]
            #row[field].decode('utf-8')
            row[field] = row[field].replace('\xff','').replace('\xef','')
            row[field] = row[field].replace('\xbb','').replace('\xbf','')
#            row[field] = unicode(row[field],'utf-8').strip()

sebastiaan

The error message says that decode() is being called on an int object. Python is usually not wrong about these things, so the question is where this int comes from. I would start by adding proper debug print statements, for example, like so:

for i, row in enumerate(lines):
   for field in row:
       print(u'row %d, field %s has value %s of type %s' % (i, field, row[field], type(row[field])))

This way you can walk through all rows and fields to see if there's an int in there. Then, once you know that's the case (or not), you can go back to where row is created, and see what's going on.

Cheers,
Sebastiaan

Kiri

Thank you very much, Sebastiaan.

It seems that one of the condition-coding rows can't be decoded. Yet, another row with similar input (integers, too) doesn't provoke any errors.

Here's the new error-message:

row 0, field type has value 0 of type <type 'str'>

Error while executing inline script

phase: prepare
item: exp
line: 58
exception message: 
exception type: UnicodeDecodeError

Traceback:
  File "dist\libopensesame\inline_script.py", line 133, in prepare
  File "<string>", line 58, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128)

How do I change the type of the row into another variable-type? I already tried to change it into string by using str() (as str(row[field]) among others) but this didn't work. Codec of the csv-file still says "UTF-8 w/o BOM".

The file is read by the code as follows:

with open(file,'rb') as filein:
    stimuli_reader = csv.DictReader(filein)
    lines = [row for row in stimuli_reader]

sebastiaan

I think the easiest solution for you is to use the UnicodeReader, which is a recipe described on the Python docs:

• https://docs.python.org/2/library/csv.html#examples

You can add this snippet in a init script at the beginning of the experiment:

import csv, codecs, cStringIO

class UTF8Recoder:

    """
    Iterator that reads an encoded stream and reencodes the input to UTF-8
    """
    def __init__(self, f, encoding):
        self.reader = codecs.getreader(encoding)(f)

    def __iter__(self):
        return self

    def next(self):
        return self.reader.next().encode("utf-8")

class UnicodeReader:

    """
    A CSV reader which will iterate over lines in the CSV file "f",
    which is encoded in the given encoding.
    """

    def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):
        f = UTF8Recoder(f, encoding)
        self.reader = csv.reader(f, dialect=dialect, **kwds)

    def next(self):
        row = self.reader.next()
        return [unicode(s, "utf-8") for s in row]

    def __iter__(self):
        return self

Then you can read the csv file as before, except using UnicodeReader instead of csv.DictReader. Like so:

with open(file,'rb') as filein:
    stimuli_reader = UnicodeReader(filein)
    lines = [row for row in stimuli_reader]

Make sure you don't mix unicode and str! In other words, use u'unicode strings' instead of 'str strings' when typing string literals in your code.

Cheers,
Sebastiaan

Kiri

Thank you. I tested it but maybe I understood something wrong...

I filled in the code above (imports and class definitions) and afterwards let the script read the file as follows:

with open(file,'rb') as filein:
    stimuli_reader = UnicodeReader(filein)
    lines = [row for row in stimuli_reader]
    print (lines)
    for i,row in enumerate(lines):
        for field in row:
            print(u'row %d, field %s has value %s of type %s' % (i, field, row[field], type(row[field])))

But now the script seems to declare string to every field of the csv-file. Therefore, I get the following error message:

Error while executing inline script
phase: prepare
item: exp
line: 85
exception message: list indices must be integers, not unicode
exception type: TypeError

Traceback (also in debug window):
File "dist\libopensesame\inline_script.py", line 133, in prepare
File "", line 85, in
TypeError: list indices must be integers, not unicode

Neither in line 133 of prepare (where one of the conditions is assigned to a number for the trigger-function) nor in line 85 (which is the line right after printing the fields within the above code) seems to be a plausible reason for this error-message.

Within debug-window, the code is not printed out as letters, but as a row of character codes. (I think this is because of the print command using u', but deleting it or replacing it with str() doesn't work.)

Might there be a way to assign integer to specific fields of the csv-file?