[solved] Gamepad or Joystick as input device

Erwin

Hi

For several experiments, I have previously used gamepads as input devices. These are a lot faster than keyboards and way cheaper than serial response boxes.

Since Open Sesame uses pygame, is there a way in which I can integrate the joystick function so that I can set up a gamepad as the input device in an experiment in Open Sesame?

Regards,

Erwin

sebastiaan

Hi Erwin,

Yes, if you use the legacy (pygame) back-end you should be able to use all PyGame functions. For example, as demonstrated here: http://osdoc.cogsci.nl/back-ends/legacy

I have never used a joystick/ gamepad myself, but they are supported by PyGame. The documentation page is here: http://www.pygame.org/docs/ref/joystick.html

You would have to write some Python inline code to use a joystick, though. But probably nothing overwhelmingly difficult.

Please let me know how it goes! It would be useful to add some documentation about using joysticks/ gamepads.

Regards,
Sebastiaan

Erwin

Hi again

I have being trying to the inline script to recognize a gamepad. I have this code now:

self.experiment.set("allowed_responses", "m;f")
self.experiment.set ("description", "Collects joystick responses")
self.experiment.set ("timeout", "infinite")
self.experiment.set ("flush", "yes")

import pygame
from pygame.locals import *

pygame.init()
gamepad = pygame.joystick.Joystick(0)
gamepad.init()
joystick_count=pygame.joystick.get_count()

if joystick_count > 0:
                                print "Joysticks found = %i" % (joystick_count) #Shows in debugging

else:
                print "no Joystick found"

Start_time = pygame.time.get_ticks()
trial_duration = 3000
RT = None 
button = None
response = None
m = 7
f = 8
while self.time() - Start_time < trial_duration and RT == None:
                for event in pygame.event.get():
                                if event.type == JOYBUTTONDOWN:
                                                RT = pygame.time.get_ticks () - ST
                                                button = event.button+1
                                                print  RT, button #Shows in debugging window

if button:
                self.experiment.set("button", button)
else:
                self.experiment.set("button", "NA")
if RT:
                self.experiment.set("RT", RT)
else:
                self.experiment.set("RT", "NA")
if button == m:
                response = "m"
else:
                response = "f"
if self.experiment.correct_response == response:
                self.experiment.set("correct", 1)
else:
                self.experiment.set("correct", 0)

In my experiment, I have audio feedback after every trial (a bell for correct answer, or a buzzer for a wrong answer). However, the script is not waiting for a response to play the feedback.

Any suggestions?

sebastiaan

Hi Erwin,

I don't see anything obviously wrong with the code. What exactly happens?

• Does it continue right away, as though you immediately pressed a button on the joystick? If so, what button is logged?
• Or do you get a time out after the specified 3000ms? As though it doesn't receive any button presses at all?

What I would probably do is create a dummy experiment, where you continuously poll the joystick (for say 10 seconds) and output the value of event.button, either to a canvas or to the debug window. That way you can see what type of output you get depending on what buttons you press.

Quite possibly, the joystick always returns some default value, which changes when you press a button. If that's the case you would have to wait until event.button does not have the default value, rather than just waiting for the first JOYBUTTONDOWN event. But I'm just guessing...

Hope this helps and good luck!

Regards,
Sebastiaan

Erwin

Oh, no. The indentation got lost in the previous comment.

Erwin

Hi

I'm trying to build a plug-in for the gamepad. How can I make generic_reponse read the the input from the gamepad?

Thanks

Erwin

sebastiaan

The most important function of generic_response, or at any rate the most useful for you, is that it does some bookkeeping: It calculates the response time based on the response interval, checks whether the response was correct if 'correct_response' has been set, and maintains the response averages for feedback purposes.

This conveys the general idea:

def run(self):

    # Mark the onset of the item (as 'time_[my_item]').
    self.set_item_onset()

    # Set the start of the response interval to the onset of the item, but only
    # if 'start_response_interval' hasn't already been set elsewhere.
    self.set_sri()

    # Write custom code to set (at least) the following two variables
    # * self.experiment.end_response_interval
    # * self.experiment.response

    # Have generic response do the rest
    generic_response.generic_response.response_bookkeeping(self)

    # Report success
    return True

You may also want to take a look at generic_response.py and srbox.py:

• https://github.com/smathot/OpenSesame/blob/master/libopensesame/generic_response.py#L118
• https://github.com/smathot/OpenSesame/blob/master/plugins/srbox/srbox.py

PS. You can use the <PRE> </PRE> tags to preserve indentation etc. on the forum.

sebastiaan

Hi Erwin,

I don't know if you're still working on this, but I saw in a post on the PsychoPy developer mailing list that support for joysticks will be introduces as of 1.73.00. Perhaps that's worth checking out?

• http://www.psychopy.org/changelog.html

Regards,
Sebastiaan

Erwin

Hi Sebastian

Thanks for the tip on Psychopy. I tried several times before last year to install Psychopy on my laptop, but not even the demos would'nt run. I guess it's a compatibility problem with the video controller.

But, yes, I'm still working on the gamepad. After reading the Psychopy documentation, I can see that I was pretty much doing the same on Open Sesame. I did manage to have Open Sesame recognizing the gamepad input, but I have still some problems with the timing. In particular, the way I wrote the experiment script makes Open Sesame wait till the end of the audio file to collect the button press. I tried setting the sampler to Duration =0, but it didn't work.

I'll start from scratch with a new experiment script using the keyboard as input devise to check that my gamepad plugin is not creating the problem.

I'll let you know how it goes.

martinc

Hi Erwin/Sebastiaan,

did you get any further with a gamepad plugin? This sounds like exactly what I'm after (as part of an eyetracking setup).

Best, MC

sebastiaan

Hi Martin,

I don't know what the status is on this (Erwin?), but you might want to check out this discussion: http://forum.cogsci.nl/index.php?p=/discussion/comment/484

Cheers,
Sebastiaan

Erwin

Hi Sebastiaan/Martinc

Yes, I wrote the joystick/gamepad plugin for my experiment. I'm now analysing the reponse times I collected using this plugin and a Genius USB gamepad, and all seems pretty good!

The plugin only collects data from button presses. However, I think it wouldn't be too hard to add the hats or direction buttons to the pluggin, if your experiment calls for that.

Sebastiaan, I have sent the plugin to your email account s.mathot@cogsci.nl. Let me know
how it goes.

Erwin

sebastiaan

Great! I don't have a joystick myself to really test it, but as far I can tell it should work fine (at least it shows up in OpenSesame for me, and you used it in an actual experiment, right?).
I uploaded the plug-in here, I'm sure it will be useful to some people: http://files.cogsci.nl/misc/joystick_sesame.zip

Are you planning to develop the plug-in any further? No pressure, but if so, it might be an idea to host it on GitHub. Or I could give you an ftp account for cogsci.nl, so you can upload updates yourself. In your email, you mentioned that the icons are a bit out of style with respect to the rest of the GUI. These are the icons that I use: http://tiheum.deviantart.com/art/Faenza-Icons-173323228

Thanks for sharing your efforts!

Erwin

Hi

Yes, I did use the plugin in an actual experiment. It was a spoken lexical decision task.

I'm not working on the pluging right now, but I've thought of integrating the hats to the plugin. However, I'm not sure about how to get the logger to register the contious flow of data during a trial.

Edwin

Hi all!

I tried the plugin and it didn't quite work with my Logitech Attack 3 (that is: the response isn't logged). So I did some coding myself (allowing for input from axes, hats and trackballs as well), but than a bit different. I think it would be a nice addition if we were to add the joystick-functionality to OpenSesame itself (like the mouse and keyboard options). In that way, one would be able to use simpler (compared to PyGame) functions, very similar to these: http://osdoc.cogsci.nl/python-inline-code/keyboard-functions . This would solve the problem of hat/ball/axis-input, since OpenSesame-users would be able to use functionality for these within while-loops in inline-scripts (to move stimuli about or whatever one would want to do with a joystick).

Regrettably, I ran into some trouble (mostly with the GUI-stuff) and the code is with Sebastiaan now :P But how do you like the idea?

sebastiaan

Hi Edwin and Erwin,

Good to see that work is being done on the Joystick!

It would be very nice to have simple Joystick functionality in OpenSesame by default. I'm not sure it should be implemented as a back-end though, in the same way as the keyboard and mouse. The reason is that I'd like to keep the OpenSesame core as simple as possible, and I see the joystick more as an non-standard external device like e.g. the srbox and the eyelink.

From a development perspective this only makes things easier, because the joystick doesn't need to be added as a back-end to make it available in inline_scripts (because I agree with Edwin that this would be very convenient). The way I've approached this type of situation with the Eyelink and SR Box is that I first created a class that interfaces with the Joystick (in this case using PyGame I suppose).

So I would create a file called libjoystick.py that contains a libjoystick class, more or less like this https://github.com/smathot/OpenSesame/blob/master/plugins/srbox/libsrbox.py.

Next I would create a joystick plug-in that checks whether an instance of libjoystick already exists, and if not, adds it as a property to the experiment. The code to do this is a bit messy, because it needs to dynamically import the libjoystick module from a file in the plug-in directory, but it goes like this:

# Dynamically load a libjoystick instance
if not hasattr(self.experiment, "joystick"):
    path = os.path.join(os.path.dirname(__file__), "libjoystick.py")
    _joystick = imp.load_source("libjoystick", path)
    self.experiment.joystick = _joystick.libjoystick(self.experiment, [ ... other parameters ... ] )
    #self.experiment.cleanup_functions.append(self.close) # If a clean-up function is necessary

So the joystick plug-in doesn't interface with the joystick directly, but uses the libjoystick module.

(Also see https://github.com/smathot/OpenSesame/blob/master/plugins/srbox/srbox.py#L106.)
After the joystick plug-in has initialised a joystick object, the user will be able to interface with the joystick in an inline_script as well, like so:

self.experiment.joystick.my_joystick_function_that_is_defined_in_libjoystick()

Does this make sense? So this way a plug-in can offer functionality in the GUI as well as in inline_scripts. I'm not saying that there are no other good ways to add Joystick functionality to OpenSesame, but this would at least be one of them.

Perhaps I should create a decent example plug-in, that would make things easier for other developers (because I really want people to start making plug-ins!).

Cheers,
Sebastiaan

sebastiaan

Btw, Edwin, so openexp._joystick.legacy.py is perfect (or at least it appears to be, I can't test it because I have no joystick), but I would add it as a libjoystick module to the plug-in in the way described above. Functionality-wise that doesn't affect the module itself.

Edwin

Thanks for the feedback! I do agree with you on the point of a joystick being a non-standard input device and I can relate to your point of trying to keep OpenSesame's core as simple as possible.

So, what basically needs to be done is:

• rename 'openexp._joystick.legacy.py' to 'libjoystick.py'
• add 'libjoystick.py' to the pluging
• add the code from your post to the plugin

And then possibly add some axis/hats/balls-functionality to the plugin.

That's all?

P.S. Yes please! to the example plugin.

sebastiaan

Yep, that's about it. Until the bug reports start coming in, of course

Edwin

And, of course, they do:

It all goes to bits on this line:

self.experiment.joystick = _joystick.libjoystick(self.experiment, self.__joystick)

Reporting the following:

AttributeError: 'module' object has no attribute 'libjoystick'

I'm kind of lost on what exactly your code does and where to put it (it's now in def prepare, I figured that would make the most sense), so tips are appreciated

sebastiaan

Right, the script now assumes that the class inside libjoystick.py is called libjoystick. It's probably still called legacy, right? So just make sure that the class name (_joystick.[class name]) matches the actual class.

And yep, I would add it to the prepare function.

Edwin

Oh, silly me, I renamed it 'joystick' instead of 'libjoystick'... Now it works like a charm! I'll set up a new githubthingy and try to get it to you.

Edwin

Don't have the time to set up github again now, so it's in your mailbox! I redid the whole joystick-plugin (the joystick.py part, that is), to improve it's similarity to the rest of OpenSesame (it resembles the srbox-plugin to a great extent).

Thanks for the help!

EDIT: fixed: sent you a pull request

sebastiaan

Thanks!, I'll merge it into the playground branch, which contains the experimental stuff, and then we can go from there (but see my comments on the pull request).

Erwin, do you also have a GitHub account by any chance?

Edwin

For future reference: the plugin can be found here: https://github.com/smathot/OpenSesame/tree/playground/plugins/joystick

An example experiment here: https://github.com/smathot/OpenSesame/blob/playground/examples/joystick_demonstration.opensesame