[solved] Python inline code not saved to file

joost

Hi all,

I'm new to OpenSesame and liking it a lot! However, I noticed that when I added an inline script (to generate random ISIs), which works fine, is not save to the file. Next time I open the experiment, the 'prepare phase' box is empty.
I'm using OpenSesame 0.27.1 on Mac OS X.

Best,
Joost

sebastiaan

Hi Joost,

This sounds like a problem with the Mac OS package, which is still experimental. I forwarded this post to Daniel, who builds these packages (Edit: And who replied as I was typing this!). In the meanwhile, could you please give more details: i.e. which package have you downloaded (64/ 32 bit), and which version of Mac OS X are you using?

Cheers,
Sebastiaan

Daniel

Hi Joost,

that sounds like a nasty bug. I'll see if I can reproduce this error. Could you give me more info about your working environment, just in case? Like what version of OS X are you using, are you using the 32-bit or 64-bit version of OpenSesame and do you see any error messages appear in the debug window (or console) when you press 'save'. Also, have you saved your experiment in which OpenSesame has writing rights (thus you don't need to enter your password to save at that location).

I'll keep you posted!

Daniel

Alright, I managed to reproduce it in such a way that I can save the experiment, restart opensesame and when trying to reopen the file I get

Error: Failed to open '/Users/dschreij/Desktop/test.opensesame.tar.opensesame.tar.gz'
Description: Error: Script error
Description: Failed to parse line "set _run "exp.cnvs.show()". Is there a closing quotation missing?

Make sure that the file is in .opensesame or .opensesame.tar.gz format. If you should be able to open this file, but can't, please go to http://www.cogsci.nl/opensesame to find out how to recover your experiment and file a bug report

Is this also what you are seeing, Joost?

The problem appears to be in the "set _run "exp.cnvs.show()" line. In the OS script editor, the inline script section seems to be ok and there's no sign of mismatched quotes there:

define inline_script inline_script
    set _run "exp.cnvs.show()"
    set _prepare "exp.cnvs = self.offline_canvas()
exp.cnvs.fixdot()"
    set description "Executes Python code"

I'm at a loss, as this doesn't seem to be a newline problem that kept us busy before. Do you have any ideas what might be going on, Sebastiaan?

sebastiaan

I'm at a loss, as this doesn't seem to be a newline problem that kept us busy before. Do you have any ideas what might be going on, Sebastiaan?

Not directly, but it's most definitely again some sort of newline issue, although maybe a different one. This ...

set _prepare "exp.cnvs = self.offline_canvas()
exp.cnvs.fixdot()"

... is incorrect, and should be

___prepare__
exp.cnvs = self.offline_canvas()
exp.cnvs.fixdot()
__end__

... because it is a multiline variable (link). The fact that OpenSesame treats the multiline variable as consisting of only a single line (i.e., using set) suggests that newlines are causing trouble.

What happens if you create an experiment where all inline scripts consist of multiple lines? (For example by adding pass as dummy lines.)

Daniel

Adding extra statements like 'pass' doesn't help. What does help is to place a string somewhere in the code block.
When I added "This is the prepare block" (or simply "") at the top of the above mentioned script,

set _prepare "exp.cnvs = self.offline_canvas()
exp.cnvs.fixdot()"

changed into

___prepare__
"This is a prepare block"
exp.cnvs = self.offline_canvas()
exp.cnvs.fixdot()
__end__

Apparently the double quotes of the string forces OpenSesame to store the code as a multi-line variable. When there is no string at all, it tries to save it as a single-line variable, resulting in the set _prepare syntax...

All in all, this is very peculiar, especially because it only seems to occur in the OS X versions?

Joost, we need to work on a fix for this. For now, you could use the temporary workaround of placing a string at the top of your code blocks. That way, you should be able to continue working on your experiment...

sebastiaan

Ah, right, that's clever. The variable is stored as a multi-line string when it a) consists of multiple lines, or b) has a quote in it. Criterion b) still works.

Daniel

Hmmm that's good, so there is a workaround, but better still is to make sure that the eol characters are seen correctly. They are there, as in

set _prepare "exp.cnvs = self.offline_canvas()
exp.cnvs.fixdot()"

is printed on the next line.

Can't we correct for this by checking the string in which the script block is stored for invalid line separators? After some googling, I found in Python all line separators are designated simply as "\n", but that does not seem to work here. Maybe do a check against os.linesep instead?

sebastiaan

The strategy used by OpenSesame is to make sure that lines are always separated by \n characters, regardless of the operating system. This is implemented in the inline_editor widget, as you can see here:

• https://github.com/smathot/OpenSesame/blob/master/libqtopensesame/widgets/inline_editor.py#L157

However, presumably something goes wrong here on Mac OS.

Daniel

I've run some tests with that function, or more specifically

unicode(self.text()).replace(os.linesep, '\n')

and it seems to process the newlines without problems. The strange thing is that the inline_script runs fine after it is created (even though it shows in the incorrect manner in the OS script editor). The problem occurs when an experiment is saved to file, closed, and loaded back again from file. Could it be that something goes wrong in this last phase?

sebastiaan

it seems to process the newlines without problems.

Really, how have you tested this? What happens if you add the following debugging info to toPlainText() (apologies if you've already done something along these lines)?

Edit: There should also be a repr() clause around the fixed text bit. I corrected the script below.

    def toPlainText(self):

        """
        Convenience function for compatibility with QPlainTextEdit

        Returns:
        A unicode string with the editor contents
        """

        print 'original text: %s' % repr(unicode(self.text()))
        print 'linesep: %s' % repr(os.linesep)
        print 'fixed text: %s' % repr(unicode(self.text()).replace(os.linesep, '\n'))
        # Make sure that operating specific eol's are converted to unix style
        return  unicode(self.text()).replace(os.linesep, '\n')

You should now see first the script as represented internally (with the Mac-OS-style line separator), then the line separator according to Python, and finally the script with the line separators replaced by \n characters.

The strange thing is that the inline_script runs fine after it is created (even though it shows in the incorrect manner in the OS script editor). The problem occurs when an experiment is saved to file, closed, and loaded back again from file. Could it be that something goes wrong in this last phase?

No, I really think it's the inline_editor that causes the problem. The reason that it only breaks when you reload the file (or when you apply the script, right?) is that this causes the script to be parsed. Normally, the script is only generated by the GUI, which doesn't cause any immediate trouble, even when a corrupt script is generated.

Daniel

Applying works fine. It's only the reloading that makes it crash.

Alright, didn't know about the repr() function. The output is as follows:

original text: u'exp.cnvs = self.offline_canvas()\rexp.cnvs.fixdot()'
linesep: '\n'
fixed text: u'exp.cnvs = self.offline_canvas()\rexp.cnvs.fixdot()'
original text: u'exp.cnvs.show()\rself.sleep(1000)'
linesep: '\n'
fixed text: u'exp.cnvs.show()\rself.sleep(1000)'

The function doesn't seem to alter anything: The strange thing is that \r is used for newlines, but \n is reported as the newline character by os. Should we hardcode this from os.linesep to \r for OSX? Initial tests show this fixes the problem....

I couldn't find any other reports on this after some googling....

Daniel

Hmmm, I read it might as well be caused by a bug in scintilla... You could however explicitly set the editor to use Unix style line endings by default. See:

http://www.scintilla.org/ScintillaDoc.html#LineEndings

This would make the conversion to \n on any platform superfluous and should solve this bug on OS X

sebastiaan

Ah, great detective work. I made a special branch (0.27.2-mac), and added the fix that you suggest: I.e. specifying the line-endings in QScintilla and removing the hack that replaces the line-endings: https://github.com/smathot/OpenSesame/commit/9e6ddde7230ece0ad63b9a090b98b547b320fb1f

Of course I haven't been able to test it on Mac OS, but at least it doesn't appear to disrupt anything on Linux.

Daniel

Excellent! That fixes the problem. I have tested it in several ways and I have no trouble reopening files now. The prints with repr() now also show everything with \n. The script furthermore appears with the correct syntax in the OS script editor right away. I'm going to package OS 0.27.2 and put it online as soon as I can

joost

Hi guys,
Sorry for the delay, but I'm really glad you solved the problem! Package 0.27.2 solved the problem!
Thanks,
Joost

Nadia

Hi there

Here is the same situation as Joost's. Before running my experiment the descriptopn error 'failed to parse line' appears. These are the details: OpenSesame 0.27.4, probably 64-bits on a windows 7 computer. Where do I start to solve the problem?

Nadia

sebastiaan

Hi Nadia,

This message is presumably due to an error in the syntax of your OpenSesame script, and unrelated to the end-of-line bug discussed here. Would you mind opening a new discussion for this? Please include details about the error message, what you did to receive the error message, and what you want to accomplish.

Cheers and happy holidays,
Sebastiaan

Nadia

Hi Sebastian,

Thank you for your reply. Yesterday the problem happened to be solved. I just saved the experiment under the wrong name.

Cheers NAdia