sending triggers to NIC2 via labstreaming layer (LSL) or TCP

Uros

Hello everyone, 

 

We are setting up an EEG experiment using OpenSesame for purpose of stimulus presentation. EEG device we will use is Neuroelectrics Startstim 32 with NIC2 software to design protocols and monitor EEG signals. It is recommended to use labstreaming layer (LSL) to connect OpenSesame with NIC2. To completely set up our experiment we need to send triggers/markers during the experiment from OpenSesame to NIC2.  

I suppose that we need to do some coding by inline script items. Does anyone have experience with sending triggers from OpenSesame to any EEG device via LSL? If so, I would appreciate so much if someone could send a code that had been used for this purpose. If there isn’t a finished code/solution to share, does anybody have an idea how this could be resolved? I've searched throughout the web about LSL and OpenSesame and had not found adequate help for OpenSesame particularly, but I did find some codes for LSL and PsychoPy. Do you think that PsychoPy code would work in OpenSesame also (since both are written in Python)? 

Thank you very much in advance for any help! :) 

Uros 

eduard

Hi Uros,

Do you think that PsychoPy code would work in OpenSesame also (since both are written in Python)?

Yes, most likely they will be very useful. From within inline scripts you can execute pretty much any python code. If this code happens to succesfully make contact with the LSL, then you are set.

Were your referring to this github page?

I haven't checked it in detail, but relevant commands seem to be following:

# Set up LabStreamingLayer stream.
info = StreamInfo(name='example_stream', type='Markers', channel_count=1,  channel_format='int32', source_id='example_stream_001')
outlet=StreamOutlet(info) # Broadcast the stream.

# send a trigger
outlet.push_sample(triggercode)

When using this python code synced to stimulus presentation/ response collection, you probably want to implement these things also in inline_scripts (even though it might not be necessary).

Eduard

Uros

Hey Eduard,

thank you so much for the answer!

1) I've added an inline script with the code you've copied in your response. When I try to run the experiment I get the following error:

 Inline script, line 2, in <module> NameError: name 'StreamInfo' is not defined.

I'm not sure whether some parts of the code should be in the "prepare" or "run" set of the inline script.

I've attached a simple oddball paradigm task (oddball_paradigm_1) in which I tried to implement the code. Do you have an idea what could possibly miss in my code or whether I've put the inline scripts in the correct place in the overview?

2) In the meantime, I've found some instructions here. Following these instructions, I made the other file in the attachment (oddball_paradigm_2). When I run that code, I get a similar sort of error:

Inline script, line 2, in <module> NameError: name 'i' is not defined.

I have the same questions for this version of the task/inline scripts: What could be possibly missing in the code and did I put the right items in the right place in the overview?

Thank you so much in advance! I'm kinda using the try and error method for solving this problem :)

Uros

oddball_paradigm_1.osexp

oddball_paradigm_2.osexp

eduard

Hi,

1) I've added an inline script with the code you've copied in your response. When I try to run the experiment I get the following error:

I just meant to point out the parts that are relevant. I don't think you can just copy them and expect them to work. You'll definitely need to import the libraries:

from psychopy import core, visual, event  
from pylsl import StreamInfo, StreamOutlet

Not sure, but I guess you will have to at least also install pylsl . On https://rapunzel.cogsci.nl/manual/environment/ it is explained how you can install packages in Opensesame.

Once you have correctly set up the packages (which you will know if you can import them in an inline_script or the debug window and won't get any errors), you can start thinking about where to put the actual lines to make them work with your experiment, but that is secondary for now.

I have the same questions for this version of the task/inline scripts: What could be possibly missing in the code and did I put the right items in the right place in the overview?

I can't tell for sure, because I don't have anything to socket to, but given that you don't define nor use i anywhere in your experiment, it could be somewhere in the underlying package. Could you post the entire stacktrace of the error message?

Eduard

Uros

Hi Eduard,

Thank you so much for helping us with this.

So focusing on what seems to be a simpler solution:

import socket
 s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
 #if task is running on same computer as data capture then use 'localhost' else use IP address, 
 #which is shown bottom right on capture software. The port number is 1234.
 s.connect(("localhost",1234))

It works through this part with no errors (when we turn on and connect the device to NIC ofc).

This is the part that seems to be the issue:

 #where i is the marker number
 s.send("<trigger>" + str(i) + "</trigger>")

it stops here with the following error: NameError: name 'i' is not defined

We are not sure where to define it (we tried adding it to the loop sequence but that did not work)

When we replace it with the number (s.send("<trigger>" + str(1) + "</trigger>")) it gives the following error: TypeError: a bytes-like object is required, not 'str'

Do you have an idea what should we change in this line? :)

Thank you so much in advance!

Uros

eduard

I think it is pretty arbitrary what i is. Is there information from the Labstream side what valid trigger values are? For example, for parallel ports the values have to be between 0 and 255. Just give it some value beforehand and see whether it appears in the data.

(we tried adding it to the loop sequence but that did not work)

variables that are added to the loop table are stored as part of the var object. So you would access it by calling var.i

Hope this helps,

Eduard

Uros

Hi Eduard,

thank you very much for the fast and valuable responses :)

Actually, we managed to run our line without error by modifying it to:

s.send(bytes(var.trigger))

The variable "trigger" is defined in the loop in this case.

We have to try out the whole procedure to test if triggers are being sent in the correct manner. We will post the whole code/experiment file if we manage to do it. Otherwise, we will report if potential troubleshooting comes up. :)

Uros

lvanderlinden

Dear @Uros , thank you for getting back to us! :)

Uros

Hey everyone,

I'm not sure whether I should make a new discussion regarding that the issue is not congruent to the name of the discussion. But, actually, our problem is still about sending triggers from OpenSesame to NIC2. In the meantime, we established that actually we are using TCP connection to send triggers from OpenSesame to NIC2, but not LSL.

Like I've mentioned in the upper part of the thread, we managed to run our experiment without the errors by modifying one line to:

s.send(bytes(var.trigger))

Also, we moved the inline script items to another place at the overall view, comparing to previously uploaded tasks.

But unfortunately, although the experiment runs fine, triggers are not being sent/or not being recieved to NIC2. We found on google that the item which is supposed to be sent is a string. Literally, we found this:

Once a client is connected it needs to send the following string in order to send a marker:

<TRIGGER>XXXX</TRIGGER>

Where XXXX can represent any integer number different from zero (from -2147483647 to +2147483647). This marker will be co-registered in the output files generated by NIC to the corresponding EEG sample.

We need it to be loaded from the loop so we would have recorded adequately matched trials and triggers. Would you know how we could modify the line so it would actually send it as a string? Or do you know some extra coding to recommend that could be helpful?

The send function requires the first argument defined by bytes, that's why we may have a problem sending it as a string since it's the only way we managed to run it without errors.

We tried many variations to solve this, and one of the maybe best shots that didn't work out was:

s.send("<trigger>" +bytes.(var.trigger) + "</trigger>")

"can only concatenate str (not "bytes") to str.

Do you have an idea what should we try? Thanks in advance for any help. :) Please let me know If I should make a new discussion with a more suitable name. :)

Uros

Hello everyone,

We managed to solve sending triggers from OpenSesame to NIC2! :) We did it via TCP connection protocol, not via LSL (unfortunately it is still in the title of the discussion). I'm sending you the inline script codes that we added and a screenshot of the OpenSesame overview so you could see where we had put which inline script in the experiment. The actual experiment is an example of the Oddball paradigm, and its structure is very simple.

We used OpenSesame on one computer and recorded signals with NIC2 on the other computer. In the first inline script (TCPsetup), at the beginning of the experiment we put this code in the prepare section:

import socket
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
HOST = '192.168.1.40' # The server's hostname or IP address
PORT = 1234 # The port used by the server
#if task is running on same computer as data capture then use 'localhost' else use IP address,
#which is shown bottom right on capture software. The port number is 1234.
s.connect((HOST, PORT))

In the other inline script (presentation_trigger) we put this code in the run section:

s.send(('<trigger>'+str(var.trigger_p)+'</trigger>').encode())

var_trigger_p is defined in the loop but you can use optional value/variable instead for your own purpose. This trigger appears when stimuli/sketchpad in the sequence appear on the screen.

The next inline script (response_trigger) appears after the keyboard response. We used the if function to send separated triggers if the participant responds correctly/incorrectly. Our code in the run section was:

if var.correct_response ==1 : # var.correct_response or however your experiment logs correct responses
s.send(('<trigger>'+str(3)+'</trigger>').encode())
else:
s.send(('<trigger>'+str(4)+'</trigger>').encode())

At the very end of the experiment, we added an inline script (closeTCP) which is needed to close the socket. The code we added in the run section was:

s.close()

That's it, we hope that this will help someone who would possibly have the same issue. :)

eduard

Hi Uros,

Thanks for sharing! I adapted the title of your discussion somewhat to reflect the actual implementation but also the origin of the problem.

Eduard

merelhoskens

Hi,

Thank you so much @Uros for providing your solution to this. We are actually trying to run this with Opensesame and Neuroelectrics as well. It does connect the two softwares, but it does not show the markers yet (is it meaning to show the markers in the live EEG capture?).

We're thinking it might not recognise the trigger input. Could you maybe have a look at our script and see where it might go wrong? We simply running a reaction time task, showing either a green or a blue screen and the participant has to respond as quick as they can.

Thanks!

Kind regards,

Merel Hoskens

reaction_time_task.osexp

Uros

Hey Merel,

I'm sorry for the late response, I didn't log in to forum for a while and now I see your post.

It looks that code for response triggers are good but for the presentation (trigger_script item) you should change str(var.correct_response) to str(var.var_correct_response) since you defined your triggers in a loop as a variable named var_correct_response.

I also suggest that you should use 1, 2 and 3 as a triggers numbers so maybe to change 3 to 1 in a loop variable var_correct_response. Furthermore, to change str(4) and str(5) to str(2) and str(3) in response_script for sending triggers.

Hope that you've already managed to solve this, but if you didn't maybe this will help :)

All the best,

Uroš