libeyelink closing the experiment during drift correction (using Esc?)

Wouter · February 2013

Hi there,

When running an experiment with the eyelink, it is currently not possible to exit the experiment using the Esc-key during drift correction (as one would during a trial). I was trying to look for a way to implement this, because when testing an experiment, this is usually the point where one would like to abort testing (i.e. in between trials ), but I ran into some questions:

Is there a specific reason why this is as of yet impossible?
what would be the desired behaviour? One would probably want to be able to close the experiment but also exit the calibration setup menu. My suggestion would be: do one with ESC and the other with Q, but another sensible solution would be that Esc only closes the experiment when in the setup menu.
in libeyelink, fix_triggered_drift_correction() and (manual) drift_correction() seem to have a rather different approach (aside from the fact that it is fix triggered off course), using prepare_drift_correct; getCalibrationResult and applyDriftCorrect for the first, and doDriftCorrect for the other. What is the main difference?
In the case of manual drift correction, there's a bunch of while loops. the drift_correct item itself implements a while not libeyelink.drift_correct() - loop in run(), and then in there, there's a while-true construction, that will always end in a return True/False if I'm correct. Why's the while True there?
On a related note the while not libeyelink.drift_correct() loop seems to enter calibration (i.e. tracker setup) upon drift corrcetion failure. But pylink doDriftCorrection() itself also is called with allow-setup = 1 . Shouldn't these than do eachother's jobs cancel eachother out?
In general: where do you suggest checking for an esc-press should/could go?

Cheers!
Wouter

sebastiaan · February 2013

Hi Wouter,

Is there a specific reason why this is as of yet impossible?

Well, yes: people tended to accidentally abort their experiment. The possibility of pressing Escape was present in the earlier version of the plug-ins, but because many people pressed Escape instead of 'q' to exit the set-up screen, I took it out. That's not to say that adding it back in is a bad idea (it would be really convenient, actually), but there would have to be a confirmation screen.

what would be the desired behaviour? One would probably want to be able to close the experiment but also exit the calibration setup menu. My suggestion would be: do one with ESC and the other with Q, but another sensible solution would be that Esc only closes the experiment when in the setup menu.

I think that pressing Escape once should bring up a confirmation screen, and that pressing 'y' at that point will raise an Exception to abort the experiment. Perhaps it would be best to exit the EyeLink set-up screen as well when the experiment is aborted.

in libeyelink, fix_triggered_drift_correction() and (manual) drift_correction() seem to have a rather different approach (aside from the fact that it is fix triggered off course), using prepare_drift_correct; getCalibrationResult and applyDriftCorrect for the first, and doDriftCorrect for the other. What is the main difference?

Right, the reason is that fixation-triggered drift correction is a case of DIY — This functionality is not present in the EyeLink. I don't remember exactly, but I based it on my previous C code, which was in turn based on something from the SR Research forum. Essentially it monitors eye position until it's sufficiently stable. Then it sends a spacebar press to the eyelink.

In the case of manual drift correction, there's a bunch of while loops. the drift_correct item itself implements a while not libeyelink.drift_correct() - loop in run(), and then in there, there's a while-true construction, that will always end in a return True/False if I'm correct. Why's the while True there?

Right, that's a silly loop, it should be removed.

On a related note the while not libeyelink.drift_correct() loop seems to enter calibration (i.e. tracker setup) upon drift corrcetion failure. But pylink doDriftCorrection() itself also is called with allow-setup = 1 .

I don't remember exactly how this works. It might be that the allow-setup parameter indicates that pressing the 'q' key should break the drift correction. I believe it's up to libeyelink to make the set-up screen actually appear.

In general: where do you suggest checking for an esc-press should/could go?

Right. eyelink_graphics.get_input_key() is continuously called to poll for keypresses. What it basically does is take a keypress from the experimental PC, and sends it to the EyeLink PC. This is why you can operate the EyeLink from both computers, if that makes sense.

You could quite easily have it respond to an 'escape' key as well, but then not pass it on to the EyeLink, but call, say, an eyelink_graphics.confirm_abort_experiment() function.

Cheers!

Wouter · February 2013

Hey Sebastiaan,

Your last lines suggested it would be a quick fix, but unfortunately I'm running into some issues.

I wrote a little function (http://pastebin.com/gb0dmy9M), and added to get_input_key():



try:

    _key, time = self.my_keyboard.get_key()

except: # is it better to add the type of exception? exceptions.response_error ?

    self.confirm_abort_experiment() # the function in the pastebin link

    return None:

to eyelink_grahpics.get_input_key()... just to be sure: this is what you suggested, right?

The biggest issue is that my exception doesn't 'nicely' terminate the experiment. I get a 'unknown software exception' notification w/ some memory addresses, and then opensesame just crashes.
Would you have any idea why this crashes (so hard) and doesn't first transfer data, etc etc. ? Is it because its pylink.EyelinkCustomDisplay parent class attempts to handle this exception (and fails)? How could I enforce a 'nice' crash?

Also, the current implementation of libeyelink.fix_triggered_drift_correction actually does seem to do a 'nice' (immediate) crash when Esc is pressed: this seems due to line 398 and onwards in libeyelink.py, which uses its own keyboard instance instead of the eyelink_graphics one:



    # Pressing escape enters the calibration screen

    # WK: no, it doesn't! it aborts because get_key raises an uncaught exception

    if my_keyboard.get_key()[0] != None:

        self.recording = False

        print "libeyelink.fix_triggered_drift_correction(): 'q' pressed"

        return False

...where get_key raises an exception if that key is escape.
According to your description, this shouldn't happen, yet still, the experiment seems to nicely close the datafile and then exit with 'the esc key was pressed'...so who calls libeyelink.close() in that case?

If I want to catch esc-presses both for fix-triggered and manual dc (as well as from the setup menu?) it seems I need to call my 'confirm'- function both from libeyelink as well as from eyelink_graphics . Right?

As a final issue, I think it would be desirable that any non-confirmed esc-press (i.e. my function returns instaid of raising an exception) takes one to the tracker setup. Do you see how I could enforce this? Drawing the setup canvas is no problem, but I got a bit lost actually doing the setup.

Sorry, maybe I'm making this too complicated, but I hope you see the issue(s) here..

Cheers!
Wouter

sebastiaan · February 2013

The eyelink_graphics class is pretty difficult to understand, also for me, because most of the work is actually hidden in PyLink, and there is little documentation to go by. I found out how it works based on the examples that come with PyLink and by trial-and-error.

But let me try to explain my understanding of how this works.

eyelink_graphics vis-à-vis pylink

PyLink essentially does everything, except the final input/ output bit. This is were eyelink_graphics comes in, by providing hooks to collect user responses (and pass these on PyLink) and present stimuli on the screen (but PyLink decides what will be shown and when).

On the output level, for example, draw_cal_target() is a hook that draws a calibration target. But, aside from the drawing, the calibration is performed by PyLink. You will find that draw_cal_target() is not actually called anywhere in libeyelink.py.

On the input level, get_input_key() collects input from the keyboard and passes these on to PyLink. But PyLink decides what to do based on this user input.

So eyelink_graphics is just the tip of the iceberg.

About your fix and the hard crash

just to be sure: this is what you suggested, right?

Indeed, this is what I had in mind. That this triggers a hard crash (what they would call a segmentation fault in Linux) means that there is a bug somewhere in the EyeLink C code. You can never trigger such an error in Python per se, unless there is a memory error downstream.

But, at any rate, we'll have to deal with it. An alternative would be to have 'Escape' work in the same way as 'q' (i.e. have get_input_key() return a pylink.ESC_KEY), and in addition set some flag so that the eyelink_drift_correct plug-in knows about it, for example by doing someting like:

self.experiment.eyelink_escape_pressed = True

You then handle the confirmation screen and potential exception in the eyelink_drift_correct plug-in. Does that make sense? So when Escape is pressed you set a marker (in one form or another) in eyelink_graphics.get_input_key(), and handle the rest in eyelink_drift_correct.run().

I don't fully understand how the interaction with PyLink works, so I cannot give you more than a rough outline. For example, I'm not sure whether returning pylink.ESC_KEY in eyelink_graphics.get_input_key() necessarily ends the interaction with PyLink in all cases. Imagine it does, and, if so, the fix described above should work.

About fix_triggered_drift_correct

The fixation triggered drift correction is hacked on. I'm not sure how this function relates to eyelink_graphics.get_key(). It might be that this event loop is still running in the background, in which case you can probably remove all keyboard stuff out of fix_triggered_drift_correct(). It might also be that eyelink_graphics.get_key() doesn't operate in parallel at that moment, in which case you will have to change the user interaction in fix_triggered_drift_correct() so that it matches the way it works elsewhere.

Right now, it's clearly suboptimal, because there is no confirmation screen when Escape is pressed. This is certainly going to cause trouble if there is a confirmation screen when the user presses Escape at a slightly different moment.

As a final issue, I think it would be desirable that any non-confirmed esc-press (i.e. my function returns instaid of raising an exception) takes one to the tracker setup. Do you see how I could enforce this? Drawing the setup canvas is no problem, but I got a bit lost actually doing the setup.

Once you have moved the confirmation screen to the plug-in level (eyelink_drift_correct.run()), this should be fairly straight forward. See:

https://github.com/smathot/opensesame_eyelink_plugins/blob/master/eyelink_drift_correct/eyelink_drift_correct.py#L73

Good luck!

sebastiaan · February 2013

Just to follow up on my previous comment ...

I just realized that I didn't take into account that there are threads involved. I'm not sure what kind of threading system PyLink uses, but generally speaking, exceptions in one thread do not affect other threads. Since eyelink_graphics.get_input_key() is called in a thread, throwing an exception there should not cause the experiment to abort at all, it should only stop that particular thread. Clearly, it shouldn't cause a segmentation fault either, but it does mean that my initial suggestion couldn't work. The 'flagging' approach should work though, as far I can see.

:-B

Wouter · February 2013

Just a quick response until I get to some more testing today (hopefully I will) :
W.R.T. threading and your comments:

When I enabled fix_triggered_drift_correction and threw an exception within eyelink_graphics.get_input_key() I got both: I got a segfault notification, but in the background I saw my experiment closed off with the usual 'escape key was pressed' behavior.

Wouldn't this suggest that both run in different threads and the keyboard event will be caught by both, and handled 'in their own ways' separately? And that a solution could simply be to deal with confirmation/throwing an exception within the libeyelink.drift_correct implementation, and leaving eyelink_graphics as it is? (i.e. it catches the keyboard.get_key() exception and simply returns None )

I hope to test this today if I find the time. More on this later!

sebastiaan · February 2013

Wouldn't this suggest that both run in different threads and the keyboard event will be caught by both, and handled 'in their own ways' separately?

Indeed it does.

And that a solution could simply be to deal with confirmation/throwing an exception within the libeyelink.drift_correct implementation, and leaving eyelink_graphics as it is? (i.e. it catches the keyboard.get_key() exception and simply returns None )

No, not in the comprehensive way that you envision it, I think. This would only work for the fixation triggered drift correction, which is, as I said above, a special case. To handle 'Escape' presses during the setup screen, regular drift-correction, calibration, etc., you will have to dive into eyelink_graphics.

Wouter · February 2013

Hey Sebastiaan,

I'm pretty sure I fixed it!

libeyelink.py

diff for libeyelink.py

eyelink_drift_correction.py

diff for eyelink_drift_correction.py

Quick summary of the changes I made, primarily in libeyelink.py

in the libeyelink init:
experiment.eyelink_esc_pressed = False

in eyelink_graphics.get_input key()

        try:
            key, time = self.my_keyboard.get_key()
        except openexp.exceptions.response_error:
            key = 'escape'
        except:
            # any other exception still returns None
            return None
    (...)
        elif key == "escape": # escape does the same as 'q', but also marks esc_pressed
            self.experiment.eyelink_esc_pressed = True
            keycode = pylink.ESC_KEY
            self.state = None

- I also removed the key_to_char() construct (deprecated, right?)
- it also sets self.experiment.eyelink_esc_pressed = False in its init, just to be sure (should be unnecessary)

in libeyelink.drift_correction()
- set self.experiment.eyelink_esc_pressed = False
- I changed the 'silly loop' construct to:

        # 'silly loop, it should be left out:'
        # while True:
        if not self.connected():
            raise exceptions.runtime_error("The eyelink is not connected")
        try:
            # Params: x, y, draw fix, allow_setup
            print 'attempting drift correct...'
            error = pylink.getEYELINK().doDriftCorrect(pos[0], pos[1], 0, 0)
            if error != 27:
                print "libeyelink.drift_correction(): success"
                return True
            else:
                # TODO should that be "...escape or q pressed" ?
                print "libeyelink.drift_correction(): escape pressed"
                return False
        except Exception, e:
            print "libeyelink.drift_correction(): try again"
            return False

in libeyelink.fix_triggered_drift_correction :
- I changed your keyboard check for q into:


            try:
                key,time = my_keyboard.get_key()
            except:
                # if escape key pressed:
                self.experiment.eyelink_esc_pressed = True
                self.recording = False
                return False
            else:
                if key != None: # i.e. 'q' was pressed
                    self.recording = False
                    print "libeyelink.fix_triggered_drift_correction(): 'q' pressed"
                    return False

confirm_abort_experiment(self) was written (in libeyelink), very similar to what I previously suggested. (Do you like the timeout idea?)
In eyelink_drift_correct.py : run()
when eyelink.drift_correction() returns False, check whether this is due to an escape press. if so, run eyelink.confirm_abort_experiment(self) if it is not confirmed, set eyelink_escape_pressed back to False

This all seems to work fine, except that it became too much work for now to also have it respond to 'Esc' during the setup screen, so now these are handled as q-presses (but now that I think about it, that might actually be easy to add). So I only have some minor questions/points:

putting eyelink_esc_pressed so open in 'experiment' seems a bit weird to me.. should we put it away somewhere? or give it a name with an underscore?
in eyelink_graphics.get_input_key() _ I imported _openexp.exceptions to dissociate the response_error from other possible errors this function could rise (other exceptions do occur...sometimes the thread still seems to be polling while the experiment is already finished, which raises another exception. However no one sees it because it's not in the 'main' opensesame thread...does that make sense? ). But my actual question was: can I also use the libopensesame.exceptions (which is already imported) instead of openexp exceptions? Why are these different?
Like pressing 'q' during manual drift correction, pressing 'Esc' triggers pylink.ESC_KEY, which seems to automatically imply the low frequency BEEP. This beep is then combined with the message 'calibration unsuccesful!'. Do you see a way around this beep?
I'm still having trouble understanding what happens when the 'main' loop of opensesame throws an exception? Why does this seem to not bother pylink and its separate thread(s?) at all? From the output it seems that even libeyelink.close() is still called by someone/something. Who shuts this whole connection down so perfectly?
When collecting keyboard responses, I noticed that one might run into a problem when using a keyboard item that already has a timeout set (e.g. timeout=0), but wants to call get_key() without a timeout (i.e. get_key(timeout = None). In that case, the lines 184-185 in openexp/_keyboard/legacy.py seem to make that impossible. Maybe that should be changed, using something like -1 instead of None?

Are you satisfied? Or do you immediately spot things you'd like to have changed before I commit stuff through Daniel again?

Cheers,
Wouter

sebastiaan · February 2013

putting eyelink_esc_pressed so open in 'experiment' seems a bit weird to me.. should we put it away somewhere? or give it a name with an underscore?

It's a bit strange maybe, although the experiment object is used this way on other occasions as well (which doesn't necessarily make it a good thing, of course). Perhaps we can move it to the eyelink object then:

self.experiment.eyelink.escape_pressed

But my actual question was: can I also use the libopensesame.exceptions (which is already imported) instead of openexp exceptions? Why are these different?

openexp is the back-end layer that lies below libopensesame. If an error occurs at this level, an openexp.exception is thrown, otherwise a libopensesame.exception is thrown. The distinction can be a bit fuzzy, and it makes no practical difference. But in this case I think a libopensesame exception makes more sense.

Like pressing 'q' during manual drift correction, pressing 'Esc' triggers pylink.ESC_KEY, which seems to automatically imply the low frequency BEEP. This beep is then combined with the message 'calibration unsuccesful!'. Do you see a way around this beep?

Only a hackish way. If PyLink wants to play a beep, then there's not much we can do about that. But you can specify in eyelink_graphics.play_beep() that a beep shouldn't be played if the Escape flag is set.

I'm still having trouble understanding what happens when the 'main' loop of opensesame throws an exception? Why does this seem to not bother pylink and its separate thread(s?) at all? From the output it seems that even libeyelink.close() is still called by someone/something. Who shuts this whole connection down so perfectly?

To make sure that things don't become too nasty when an exception occurs, plug-ins can register a clean-up function, like so:

self.experiment.cleanup_functions.append(self.my_cleanup_function)

As long as there is no segmentation fault, in which case all bets are off, clean-up functions are always called when the experiment is finished. In this case, the clean-up function is eyelink_calibrate.close().

When collecting keyboard responses, I noticed that one might run into a problem when using a keyboard item that already has a timeout set (e.g. timeout=0), but wants to call get_key() without a timeout (i.e. get_key(timeout = None). In that case, the lines 184-185 in openexp/_keyboard/legacy.py seem to make that impossible. Maybe that should be changed, using something like -1 instead of None?

Hmm, I see what you mean. Changing the API in this way is probably not a good idea, because it might break current experiments. What you could do though is explicitly call:

keyboard.set_timeout(None)

Are you satisfied? Or do you immediately spot things you'd like to have changed before I commit stuff through Daniel again?

I think it looks great, so yes please send me a pull request, so I can try it out as well. Why don't you make your own account, by the way? You don't need special permission or anything, anybody can send a pull request through GitHub.

Once I have merged these changes and the dummy-mode changes by Edwin, I think I'll restructure the whole thing a bit, by putting each class in its own file. libeyelink.py is becoming a bit of a monster.

Great work!

Sebastiaan

Wouter · February 2013

Perhaps we can move it to the eyelink object then

Tried that already .. then I ran into the realization that eyelink_graphics is constructed before experiment.eyelink is fully constructed (i.e. exists). We could probably make it work, but we have to be sure that the eyelink initialization will always be finished before eyelink_graphics ever tries to set the esc_pressed flag.

Come to think of it; should we worry about thread safety with this flag at all?

openexp is the back-end layer that lies below libopensesame. If an error occurs at this level, an openexp.exception is thrown, otherwise a libopensesame.exception is thrown. The distinction can be a bit fuzzy, and it makes no practical difference. But in this case I think a libopensesame exception makes more sense.

Initially I imported it for catching the error thrown by get_key in eyelink_graphics.get_input(), and then simply raised that exception anyway when abort was confirmed... I can throw a different one of course, but I still need to import openexp.exceptions to catch it, right? (a generic exceptGDN won't work, the try-except block in eyelink_graphics has to dissociate between an esc-press-exception and other exceptions, which seem to occur when the experiment closes but the thread still tries to capture kb input).

you can specify in eyelink_graphics.play_beep() that a beep shouldn't be played if the Escape flag is set.

hmm yes... but it would still beep when 'q' is pressed . I was thinking that maybe something could be done with the following dissociation beepid == pylink.CAL_ERR_BEEP vs beepid == pylink.DC_ERR_BEEP, that you might know off the top of your head. It would still beep, but at least it won't show the 'calibration unsuccessful' canvas during DC. I could give this a try...later .

...plug-ins can register a clean-up function...

Briliant!

(...)Changing the API in this way is probably not a good idea, because it might break current experiments (...)

Perhaps you could check whether timeout was explicitly set in the call using something like inspect . But probably that's overcomplicating things

(...) please send me a pull request (...)

I'll set up a Git account and snapshot, and then get used to the terminology and commands I'll probably send one tomorrow.

Howdy, Stranger!

Categories

libeyelink closing the experiment during drift correction (using Esc?)

Comments

eyelink_graphics vis-à-vis pylink

About your fix and the hard crash

About fix_triggered_drift_correct

Howdy, Stranger!

Quick Links

Categories

libeyelink closing the experiment during drift correction (using Esc?)

Comments

eyelink_graphics vis-à-vis pylink

About your fix and the hard crash

About fix_triggered_drift_correct