[open] Arduino+tablet

maximecauchoix

Hey Everyone,

What do you think about controlling an Arduino with Opensesame from a tablet? The task would be to activate a motor through the arduino each time the subject give a good answer (reward system). Could I do that with the function used to send EEG triggers?

Thanks

Maxime

sebastiaan

Hi Maxime,

The Arduino is generally controlled via a serial port emulated through USB. This means that you can simply use the serial module and don't need to bother with the complicated stuff required to get the parallel port working. Of course, I don't know what the communication protocol with your specific Arduino device is, but the serial docs should give you a general feeling for how this works:

• http://pyserial.sourceforge.net/shortintro.html

Update: Assuming, of course, that you can plug the Arduino USB cable into the tablet?

Cheers,
Sebastiaan

maximecauchoix

Hi Sebastiaan,
thanks a lot for your quick answer. Could you give me an example of how to integrate this serial command in my opensesame experiment?
Thanks
Maxime

sebastiaan

Hi Maxime,

I just noticed something pretty crucial! The Python implementation for Android (pgs4a) does not include the pyserial module. Therefore, on Android it's not possible to communicate with the Arduino via the USB emulated serial port.

There are many other ways to set up communication between the Arduino and the tablet, tough. The easiest way would probably be to use WiFi, because it's easy to connect a tablet to a wireless network. You could add a WiFi shield to the Arduino or connect the Arduino to a small computer (a netbook or something) and have this computer listen to the tablet via WiFi and control the Arduino via the (USB emulated) serial port. This last option, using a small computer as a bridge, may be most convenient, and is not as difficult as it sounds.

But what the best solution would be in your case depends on what exactly you're trying to set up, and what kind of equipment you have available, of course.

Cheers,
Sebastiaan

Edwin

Hi guys,

If timing isn't crucial, couldn't you just use the socket module? It's in Python for Android, so that shouldn't be an issue. Is there a LAN port on your Arduino? I've done similar stuff with Raspberry Pi's. Sample code for the client side:

import socket

ip = '192.168.100.2' # put in your own IP here
port = 40007

print 'Opening socket'
socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

socket.settimeout(0.1)
socket.bind((ip, port))

print('Listening')
while True:
    try:
        msg, addr = socket.recvfrom(5)
        print '%s says: %s' % (addr,msg)
    except:
        msg = None
    if msg == 'start':
        socket.sendto('start', addr)
    elif msg == 'close':
        break
    elif msg == 'moto1':
        # implement code to make motor 1 run
        print("starting motor 1!")
print 'Connection closed'
socket.close()

On the server side you can send string codes of 5 characters to make the client do stuff. Example:

import socket

ip = '192.168.100.2' # The IP address of the other PC
port = 40007 # An arbitrary port that matches the client

print 'Opening socket'
socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# Reducing the timeout will improve synchronization
socket.settimeout(0.1)
print 'Listening'
while True:
    socket.sendto('start', (ip, port))
    print 'send start'
    try:
        s, addr = socket.recvfrom(5)
        print 'recv %s' % s
    except: # A timeout occurred
        s = None
    if s == 'start':
        print 'Client says hi! Say something back.'
        msg = ''
        while msg != 'close':
            msg = input('What do you want to say? ')
            socket.sendto(msg, (ip,port))
        break
print 'Connection closed!'
socket.close()

This particular example lets you send stuff you put in via the command line, but you can see how this could be implemented in an actual experiment, using inline scripting.

Good luck!

maximecauchoix

Hi Guys,

Thanks for answers. Unfortunately, I do care about timing. I want to control a tiny engine that will give a reward to the animal, if perform correctly the task. This conditioning procedure only works if the timing is good. With the wifi option, what would be the opensesame function (/ python code) to send the information by wifi to the computer that control the engine?

Thanks

Maxime

Edwin

If you care about the timing, wifi won't be much better (actually, it might well be worse than using a wired connection). But I'm talking about delays in the order of (several) dozens of milliseconds. The timing for reinforcement isn't THAT crucial, right?

sebastiaan

As Edwin says, your temporal precision will still be measurable in tens of milliseconds, which I imagine is fine for food rewards.

More generally though, you don't provide enough detail for us to make any specific suggestions!

• So you have a custom Arduino-based device for giving food rewards. But how exactly does this work? My guess is that the device provides food rewards in response to certain commands that it receives via the serial port. Is that correct? And what is the exact communication protocol, what 'language' does the Arduino speak? You need to know this in order to implement any communication with OpenSesame or other software.
• What are the exact requirements? Is, say, a 50 ms delay acceptable? Do you work with a set-up that's already there, or can you add an additional 'bridge' computer, as I suggested?
• What do kind of hardware are you working with? What does your set-up look like? Does the tablet have an ethernet port? Does the Arduino have an ethernet or a WiFi shield, or is it just a basic Arduino with just a USB connection? The connections that you determine how you set things up. I just guessed that your tablet is WiFi only and the Arduino is USB-based serial port only, but I might be wrong.

Without specifics we can only provide a general outline of what a solution might look like!

maximecauchoix

Hi Edwin and Sebastiaan,

Sorry for not giving enough details. First, you both are right, a 50ms delay is reasonable.
Basically I'm building my own set-up from scratch.

The only requirements are:

• a resistive screen or infrared panel (that can be activated by a bird beck)
• a screen with a high refresh rate (bird have a high flicker fusion threshold about 100Hz)
• a software that can present visual displays (for discrimination, associative learning, visual search experiment), collect responses and that control the angle of rotation of a small rotating motor (rewards are placed on a wheel controlled by the engine, a correct answer activate the motor with a certain angle of rotation, which makes one of the reward accessible for the bird through a hole in the cage)
• a hardware solution to control the engine from the presentation software.

For now all options are possibles! The whole setup should be a reasonable price (<1000$) because we want to have 12 of them.

That's why I chose the android tablet with a resistive screen (Flytouch 7, http://chinaphonehouse.org/2012/08/flytouch-7-allwinner-a10-test-android-4/) + opensesame option. Everything is working perfectly except the activation of the motor.
To specifically answer your questions Sebastiaan the tablet has:

• 2 USB
• 1 HDMI
• 1 ethernet (RJ45).

I bought a basic arduino because I read on internet that it was a good option to control an engine through a USB port (I think the language is Arduino, a language that I don't speak at all), but I can buy any (not too much expensive) electronic stuffs that would bring me closer to a solution that works!

Do not hesitate to propose any system you think would be more appropriate.

Thank you very much for your help.

Maxime

sebastiaan

Hi Maxime,

Interesting project! So basically you have freedom, as long as it's not too expensive.

Tablet - computer via WiFi or ethernet

I think I would link the tablet to a computer (maybe a cheap netbook). This can be done via WiFi or via an ethernet cable. That doesn't matter for the software side of things, so you can see later on which is most convenient.

As Edwin, says you can have your OpenSesame experiment on the tablet communicate with a script on the computer using the Python socket module. Basically, you will need something like the scripts that Edwin posted above: One for in your OpenSesame experiment on the tablet (the server in Edwin's example) and one for one the computer (the client). But I wouldn't initially focus on this, see below.

Computer - Arduino via USB-emulated serial port

You will also need to connect the computer to the Arduino, using the USB port and the python serial module. But -- just to make sure that you are aware of this -- you will need to write custom software yourself that runs on the Arduino. You cannot simply solder some servo connections onto the Arduino board and control these through your PC. Instead, you will need to write a so-called 'sketch' for your Arduino that listens to incoming messages from the PC and uses these to control the servos that handle the food rewards. Writing the Arduino sketch and deciding how you are going to build a system that can dispense food rewards will be the most challenging part of the whole setup.

So I would not connect the tablet directly to the Arduino, even though this might be possible, but use a computer that sits in between the tablet and the Arduino. This will make it easier to test and debug each part of the setup separately. It also gives flexibility, in case you want to do more complicated experiments later on.

A first step

A good first step would be to build the Arduino device that dispenses food rewards in response to information received via the serial port. I can give you some hints, although this has nothing to do with OpenSesame, but the best place to get information on this is probably the Arduino site itself. You should be able to connect your Arduino to a computer and trigger a food reward from the computer using a simple Python script like this:

import serial
# Connect to the Arduino via the serial port. On Linux these are usually
# called `ttyACM[X]`. On Windows these are called `COM[X]`.
s = serial.Serial('/dev/ttyACM0')
# Send the byte 1. The Arduino should listen for incoming information and
# respond to this!
s.write(chr(1))
# Close the serial connection
s.close()

Once you can do this, it will be relatively easy to control the food rewards via the tablet.

Cheers!
Sebastiaan