[solved] Multi-character keyboard response *or* simultaneous sketchpad and text_input

jjholliday

Hello again,

I need my participants to be able to view a picture on sketchpad and then enter a multi-character keyboard response. Right now I have them viewing the sketchpad, doing a keypress to advance, and then using text_input to enter their response. I think the intermediate keypress-to-advance-to-text_input is distracting and I would like to eliminate it if possible.

The only way I see sketchpad can accept responses is by setting the duration to "keypress" (which will advance after a single keypress), or by using the keyboard response module (which also does not (seem to) allow multi-character responses). Ideally I would like the participant to be able to continue viewing the sketchpad while doing text_input, because it lets them see what they've typed and then press enter before advancing to the next step. I tried messing with the foreground of text_input but it seems to only accept colors.. I also tried telling the text_input to draw an image from a file but it doesn't like that either. Is what I'm describing possible in OpenSesame? Thanks in advance!

sebastiaan

No, I'm afraid that isn't possible without some inline coding. But fortunately nothing terribly complicated. The following code snippet accepts keyboard input until 'return' is pressed and overlays the resulting string over the canvas of a sketchpad called 'my_sketchpad' (which should be the last item prior to this piece of inline code).

I hope this will help you to get started! You can find more information about the keyboard en canvas classes here, so you can tweak this code a bit to fit your needs (text position, font sizes, etc.):
http://osdoc.cogsci.nl/python-inline-code/canvas-functions
http://osdoc.cogsci.nl/python-inline-code/keyboard-functions

Update: As of OpenSesame 0.27, using to_chr() is not necessary anymore.

from openexp.canvas import canvas
from openexp.keyboard import keyboard
my_canvas = canvas(self.experiment)
my_keyboard = keyboard(self.experiment)
resp = "" # Here we store the response
while True:
    # Get a keyboard response
    key, time = my_keyboard.get_key()
    # keyboard.to_chr() converts the keycode to a description,
    # like 'space' or 'return'. If return is pressed the loop
    # should be exited.
    if my_keyboard.to_chr(key) == "return":
        break           
    # Handle backspace by removing the last character
    if my_keyboard.to_chr(key) == "backspace":
        resp = resp[:-1]    
    else:
        # The built-in chr() converts the keycode to a character,
        # like ' ' and '/'.
        resp += my_keyboard.to_chr(key)
    # Copy the canvas from a sketchpad called 'my_sketchpad' and
    # overlay the response
    my_canvas.copy(self.experiment.items["my_sketchpad"].canvas)
    my_canvas.text(resp)
    my_canvas.show()    
# Save the response
self.experiment.set("response", resp)

jjholliday

This works great, thank you! For future reference, what if I wanted to change the location of the text input? Right now it's in the middle of the screen.

jjholliday

Nevermind, I figured it out. Putting in x- and y-coordinates is easy: my_canvas.text(resp,x=100,y=100) etc. Thanks!

taniaC

Hi,
I partecipated yesterday to the workshop you made in Rovereto.
If you remember I asked you a question about my experiment, but I have some problems.
I found this code in this post very useful but not exacty what I need.

I have to do an experiment where my participants ear a word and simply write the word on the keyboard (during the sound). Using the code you suggested I can do it quite easy.
The problem is that I have to register every key press reaction time. So, if the word is “dog” I need the timing of “d”, “o” and “g”.
In the log file I see that I have a “correct column” with 0 and 1 if the word is correct. And this is perfect. But I want to record also what they write as incorrect response and mostly the reaction time for each key press.

Then, I need exactly the same experiment on a mobile phone.
Do you think it is possible? Can you kindly suggest me how to do?

Thanks!
Tania

sebastiaan

Hi Tania,

Sure, I remember!

The openexp.keyboard module will, as you have noticed, show and hide the Android virtual keyboard for every keypress. If you want to bypass this limitation, you have to use pygame directly, and show/ hide the virtual keyboard yourself. This makes the script a little more complicated, but nothing too difficult. Below you see basically the same script as above, except that it uses pygame to collect key presses and it also logs information about every individual key press. I don;t know what kind of format you prefer for your logfile, but this shows the basic idea. See the code comments for more explanation.

See also:

• http://pygame.renpy.org/api.html#module-android

Cheers!
Sebastiaan

Update: Fix script to work around event.unicode bug.

# Import relevant modules
try:
    import android
except:
    android = None
import pygame
from openexp.canvas import canvas

my_canvas = canvas(self.experiment) # Create empty canvas
accept_resp = False # Set to True on return
resp = u'' # Here we store the response
t0 = self.time() # The start of the response interval

# Pop up android keyboard if available
if android != None:
    android.show_keyboard()

# Response loop
i = 0
while not accept_resp:
    t1 = self.time()
    # Collect key presses
    for event in pygame.event.get():
        # Only process key presses
        if event.type != pygame.KEYDOWN:
            continue
        # Escape aborts the experiment
        if event.key == pygame.K_ESCAPE:
            raise Exception(u'The escape key was pressed.')
        # Backspace removes last character
        if event.key == pygame.K_BACKSPACE:
            resp = resp[:-1]
        # Return accepts the response
        elif event.key == pygame.K_RETURN:
            accept_resp = True
        # Spacebar
        elif pygame.key.name(event.key) == 'space':
            resp += u' '
        # All other keys are appended to the response. Note that the
        # event.unicode attribute doesn't function correctly on Android.
        else:
            resp += pygame.key.name(event.key)
        # Set the RT and unicode value of each keypress
        exp.set('keypress_unicode_%d' % i, event.unicode)
        exp.set('keypress_rt_%d' % i, t1-t0)
        i += 1

    # Copy the canvas from a sketchpad called 'my_sketchpad' and
    # overlay the response
    my_canvas.copy(self.experiment.items["my_sketchpad"].canvas)
    my_canvas.text(resp)
    my_canvas.show()

# Determine RT, correctness, and set the response so that it can be logged
rt = t1 - t0
if resp == self.get_check('correct_response', default=''):
    correct = 1
else:
    correct = 0
exp.set_response(resp, t1-t0, correct)

# Hide android keyboard if available
if android != None:
    android.show_keyboard()

taniaC

Thanks a lot. This code works perfectlly on my pc.
I tested it on a tablet S3 tab 10 – android 4.2.2.
the keyboards responses appear as punctuations. For example, I write "Home" and it appears ");):"...punctuations.
Furthermore, no logfile is saved.
Do you think that this is a problem of the device/android version?
Sorry to bother, I’m testing on available devices. Before bay a new one I want to be sure that it works.

Thanks again
Tania

sebastiaan

Ow, you're right, that's due to a problem/ bug in the library that is used for Android (called pgs4a), which slipped my mind for a second. I updated the script above. (It's nothing to do with your device or Android version.)

Cheers!

taniaC

Ok. Now I can use also my Nexus 5, Android version 4.4. Great! Thank you so much

The last strange problem that I have: I'm trying to run a trial of the experiment. I notice that the logfiles change occasionally...I really don't know way:

The program is able to record keypresses: the Unicode works well on the pc, not on Android. I can accept this. But sometimes every keypress is recorded and sometimes only 5 keyresses. So, if I have a word of 6 letters I cannot see the rt of the last keypress. This is very annoying. I risk to loose data. The strange thing is that it happens not all the time.

Do you have any idea?

Thank for you availability!

taniaC

Sorry if I write again, any possible explanation?

sebastiaan

(...) sometimes every keypress is recorded and sometimes only 5 keyresses. So, if I have a word of 6 letters I cannot see the rt of the last keypress. This is very annoying. I risk to loose data. The strange thing is that it happens not all the time.

I suspect that the logger detects only the variables that exist when the item is first called. So if you enter five characters on the first trial, you log the first five keypresses, etc. Is that it? If so, you can tell the logger explicitly which variables should be logged, by disabling the 'Automatically detect and log all variables' option and using 'Add custom variable' (or select variables from the list).

Alternatively, you can open the general script (by clicking on 'Edit script' in the general tab) and add all variables that you want to log like so (add to the top of the script):

set keypress_unicode_0 dummy
set keypress_rt_0 dummy
set keypress_unicode_1 dummy
set keypress_rt_1 dummy
# etc.

This is a simple trick to make OpenSesame aware that these variables exist.

Does this help at all?

Cheers!
Sebastiaan

taniaC

Thanks Sebastian for your help. if I use the code for dummy variables, the log file give me columns without rt, but the word "dummy". however I manage to do it in another way.
By the way, I want to ask you one curiosity that ca be helpful for me: Opensesame automatically reverse the screen on my smartphone. Is it possible to mantain it straight?

Thanks a lot for halping me. Now I can start my experiment using this helful software!

Cheers
Tania

sebastiaan

(...) I manage to do it in another way.

Good!

Opensesame automatically reverse the screen on my smartphone. Is it possible to mantain it straight?

No, not at the moment. The screen orientation for the Android runtime for OpenSesame is fixed to landscape, due to a limitation in the underlying software. I might release two different apps (one for landscape, one for portrait) in the future to work around this.

Cheers!
Sebastiaan

elTumbillon

I tried the first code Sebastian posted: I copy-pasted it in an Inline Object following the sketchpad item. I have version 2.9. Trying to run the exp, it crashes after the sketchpad item shows and accepts response, and before getting to the In-Line code item. The error message is w.r.t the line that reads: "if my_keyboard.key.to_chr=="RETURN" "; the error message reads: " 'legacy' object has no attribute 'key'' and Exception Type is Attribute Error. I tried removing to_chr, and replacing RETURN with ENTER. No luck; same message.

I want to collect multi-key response (allowable numerical, comma, backspace, and Enter to collect and terminate"; I want the response collection to happen while the sketchpad remains displayed.

I am a complete coding novice and would appreciate any help.

sandrazeng1987

Hi Sebastiaan or anyone who had the similar experience,

I have the same problem with the first poster jjholiday here. Thanks to Sebastiaan's codes at the second post above and it works well. However, my question is how can I change the input language to other language (in my case, Mandarin Chinese). Even though I changed the PC language input through the right bottom button on my PC screen to Chinese (or using alt+ctr on windows), in the experiment, it still enters in English letters. I'm quite green in Python programming, can you show me how to do it?

Great thanks in advance!

sebastiaan

@sandrazeng1987 What exactly do you want to do? In most cases, nowadays, using a form would be preferable, does not (necessarily) require Python programming, and should support Chinese input.

See here:

• http://osdoc.cogsci.nl/3.1/manual/forms/about/

Do forms suit your needs?

sandrazeng1987

@sebastiaan
Hi, thank you for your reply. So my experiment is to show English word on the screen and ask the participants to type a Chinese translation (with Chinese input ). I tried the form item, but there are two problems here:

1) No matter I change my language input before or after I star the running of the experiment, the typing are always in English

2) Put aside my experiment, which font do I choose in the list of "font family" under the form item, in order to support the Chinese presentation? I tried many of the listed fonts which look like they are designed for Chinese characters. But the thing is in the designing interface, I can see the Chinese works perfect, but when I run the experiment, what I put in the form question turned into unrecognizable squares on the screen.

Can I ask why and whether there are ways to solve the two problems?

Thank you!

sebastiaan

Hi Sandra,

Entering non-Latin characters is possible but not very convenient, and the details depend on your operating system. Here are a few pointers:

• Select the chinese-japanese-korean font in the General Properties tab of the experiment.
• Select the legacy backend.
• In your operating system, select a keyboard layout that allows you to directly enter Chinese characters. Many non-Latin keyboard layouts work such that you enter a few Latin letters, which are then recombined into a (say) Chinese character. This won't work, because OpenSesame expects one character input per keystroke. On Mac OS, the Wubi Xing keyboard layout works for me. Whether this is a satisfying solution is hard for me to tell, because I can imagine that it results in funny Chinese text. (?)

Cheers!
Sebastiaan

Elie_F

Hello,

I'm trying to run the same experiment as Tania, it did work with latin keyboard, but I would like to do the same with an arabic keyboard.
The problem is that nothing happen when I press arabic letters on the keyboard, it's like OpenSesame doesn't get that a key is pressed (but it see the special keys in the script like space or return).
Do you have any ideas why the software does not detect arabic keypress?
(I am working on a soft keyboard on samsung tab with the droid backend.)

Thank you,

Elie

sebastiaan

Hi Elie,

Input of non-Latin text is, as far as I know, not possible with the droid backend. In fact, I might only work correctly with QWERTY (soft) keyboards, although of that I'm not sure. This is a serious limitation, but since the droid backend is no longer developed, this will not be fixed.

A few other solutions that you could consider:

• Using a Windows tablet with a regular of OpenSesame
• Using OSWeb and then run the experiment in a browser. Whether this is feasible depends on whether OSWeb supports all the functionality that you need.

Cheers!
Sebastiaan

Jib

Hi Sebastiaan,

I can't find a way to change the fonttype and fontsize of the text that the subject is typing.

I tried inserting "my_canvas.set_font('sans', 40)" in the inline but it doesn't work. Do you have any ideas ?

Thanks a lot !

Jb.

Jib

Ok I finally find out how to do it, I just had to set it in the general experiment settings. Right know I just have a problem writting accents...

Karolmou

Hello,

I used Sebastiaan's script (second comment) to create a study with a multi-character keyboard response, but I need to modify it in two ways :

1. I want to manipulate what is printed on the screen when a key is pressed (e.g. when the participant presses C I want the letter V to be printed on the screen),
2. I want the characters to be printed in upper case instead of lower case.

Could anyone help me with this ?

Thanks!

Karolina

eduard

Hi Karolina,

Without going into a lot of detail in Sebastiaan's code, you can captalize a string with the upper function. So if the variable resp is the variable that you want to capitalize you can call resp = resp.upper() to change it to upper case. For your second point, you need either some if statements, or create a mapping:

1) if statement:

if resp == 'c':
    resp = 'v'
elif resp == 

2) mapping

letter_mapping = {'c': 'v'}
resp = letter_mapping[resp]

How this integrates with Sebastiaan's code I can't tell you right away. Can you figure this out yourself?

Eduard