OpenSesame online edition components diagram

shershen08

Hello, OpenSesame developers

Discussing the architectural design of web-version we came to the following scheme:

Among questions : should server-side be written on Python or it could be PHP, or perhaps both?

If you have questions or comments, please post them in this thread, or send them directly to experimentlab2013@giphouse.nl.

sebastiaan

Hi Michael,

I think the most practical implementation would be entirely client side. As I understand your schematic, you envision a Python-based runtime environment that runs on a server, prepares canvases etc., and maintains a connection with the browser. This will be very difficult to implement.

A much easier approach would be (as I also wrote also in the email, see below) to design the runtime entirely as a client-side JavaScript/ HTML5 script. This script would essentially take the experiment as a long string, and parse this string line by line, in a way that is analogous to what the Python runtime currently does. This will not be easy, which is why I suggested initially focusing on only a subset of the functionality.

So no part of the runtime environment would use Python, and there would be no real server-side part. Also, your schematic suggests that OpenSesame experiments are Python scripts, which they are not: They use a special, very simple syntax, called OpenSesame script. The fact that OpenSesame experiments are not Python makes it possible (but not easy!) to implement an alternative runtime environment in any language, such as JavaScript. (Of course, the inline_script items would be lost, because they require a Python interpreter, but the other items can be implemented in any language.)

Since there is quite a bit of overlap with the questions that you asked earlier per e-mail, here's again (part of) my response. See especially the part about implementing only a subset of the functionality using JavaScript/ HTML5.

Does this clear things up?

Cheers,
Sebastiaan

--

Is it possible to run OS in server environment with Python installed ? (Like OS portable?)

Do you mean whether you can run OpenSesame on a server and give users remote access through a browser? In principle, no. Of course, you could run OpenSesame on a server and use some kind of remote-desktop tool to remotely run experiments. But that's not really what we're aiming for here: There should be a new implementation of the OpenSesame runtime that lives in a browser.

Subject's profiles - is there any internal data-storage or how are those data kept?

Do you mean the log files? The data is simply written to a .csv file. For a web-based version this can be done in many ways, for example by calling a .php script that stores information in a MySQL database. But I wouldn't focus on this initially.

More generally, I would start by familiarizing yourself with OpenSesame as it works on the desktop, by walking through the tutorials and reading the documentation. Once you have a good idea of how OpenSesame works as a desktop program, you can start thinking about how this can be ported to a web-friendly system, presumably using JavaScript / HTML5.

Regarding the structure of OpenSesame. Basically, each experiment consists of a script that is written in a very simple domain-specific language, described here:

• http://osdoc.cogsci.nl/usage/opensesame-script/

You can also take a look at some simple experiments, to see what an OpenSesame script looks like for an actual experiment:

• https://github.com/smathot/OpenSesame/blob/master/examples/tilt_adaptation.opensesame

Each item (each part of the experiment) has its own script. And each item support a slightly different syntax, as described above. Basically, you would need to write a JavaScipt that reads this syntax and transforms it to an experiment running in a browser. For example:

define sketchpad
    my_sketchpad set duration 100 draw fixdot 0 0 color=white show_if="always"

This should result in a display with a fixation dot being shown for 100 ms.

There are 10 core items, which can be implemented separately from each other, assuming that you use a modular design. So as a first step, you could aim for only the items that are really necessary for running a simple experiment. These are the loop, the sequence, the sketchpad and the keyboard_response. If you manage to run an experiment consisting of only these items in a browser, you have already made an enormous accomplishment.

So a few concrete tips. (This is just how I would approach the project, but there may be other ways as well, of course.)

• You shouldn't modify the current Python code, but rather write a JavaScript/ HTML5-based interpreter from scratch. This intepreter would live in a browser and parse and execute OpenSesame scripts.
• I would implement the whole thing client-side.
• Initially focus on a subset of functionality, presumably the loop, sequence, sketchpad, and keyboard_response items.
• Create a very simple experiment, using only the items above, and focus on getting this reference script to run in a browser in the same way that it runs on the desktop.

Maarten

Hi all,

I wonder whether it would be a better idea to write an offline 'opensesame script -> html5/javascript' converter. The conversion would only have to happen once, serverside, and it would generate (almost) static webpages. This way it would relieve the clients of having to do the conversion everytime themselves.

Another advantage this has, which could be useful during development, is that you can manually change the generated html5/javascript a little bit, if it contains a (small) bug. This would be a quicker workaround, than waiting for the converter to have this bug fix incorporated, after which the clients can do the experiment.

On a different note, are there advantages in using python for writing this converter? For example, can it share part of the code base of the opensesame script interpreter? This way it might also be easier to be integrated in the desktop version of opensesame, and possibly use a simple webserver for local testing.

Cheers,
Maarten

shershen08

Sebastiaan, thanks for corrections about OpenSesame script
I got your point about client-side experiment generating, but still we'll need a utility server-side anyway i suppose.

Here's a new version of a scheme

and simple process diagram for discussion:

EoinTravers

Hi all,

Over the the last few weeks, independent of what I've been doing with OpenSesame, I've been designing a JavaScript platform for running experiments online, tentatively called PsychScript.

This has grown from some JavaScript I had written for my own research, which I wanted to abstract out into reusable functions.
To my mind, the two projects aren't radically different, although not all of PsychScript makes use of the HTML5 canvas (for the functions that do, take a look in the mouse-tracking directory, or the demo here). PsychScript also isn't as object-oriented as OpenSesame, although I plan to work on that.
Basically, what I'm suggesting is that it should be possible to compile much of the functional aspects of OpenSesame to JavaScript (using, for instance, PyJs, and use PsychScript as the basic of a HTML5 backend, handling the canvas, keyboard, and mouse events.

Cheers,

Eoin

sebastiaan

Good to see some more input!

I wonder whether it would be a better idea to write an offline 'opensesame script -> html5/javascript' converter. The conversion would only have to happen once, serverside, and it would generate (almost) static webpages.

@Maarten Yes, this would also be an option, I think. And it might be easier than writing an interpreter, I'm not sure. Also see Eoin's suggestion, for another possibility (maybe?).

On a different note, are there advantages in using python for writing this converter? For example, can it share part of the code base of the opensesame script interpreter?

I think the overlap with the current Python-based runtime would be fairly small (although non-zero), so that in itself would not be a decisive reason to implement such a compiler in Python. For consistency with the rest of the OpenSesame codebase, it would be sensible to use Python though. Do you have another language in mind?

I got your point about client-side experiment generating, but still we'll need a utility server-side anyway i suppose.

@shershen08 That's true, of course, because the experiment would need to be served to the browser and the data would need to be collected on a server. The reason that I de-emphasized the server part is that all of this will be very simple in comparison to the complexity of writing a JavaScript-based interpreter or, as Maarten suggested, a OpenSesame-script-to-JavaScript compiler. And therefore I wouldn't initially not focus on the server-side of things.

Over the the last few weeks, independent of what I've been doing with OpenSesame, I've been designing a JavaScript platform for running experiments online, tentatively called PsychScript.

@EoinTravers That looks pretty good.

Basically, what I'm suggesting is that it should be possible to compile much of the functional aspects of OpenSesame to JavaScript (using, for instance, PyJs, and use PsychScript as the basic of a HTML5 backend, handling the canvas, keyboard, and mouse events.

I don't understand well enough how PyJs works to really envision how all of this would come together. Do you think it would be possible to use the Python runtime as is, and only implement a new HTML5 back-end? Can you elaborate a bit on how this would work?

EoinTravers

Honestly, I don't fully understand what I mean either.

I can see two possibilities that, theoretically, should be possible.

The first would be to basically try to pass all the Python (or possibly the .opensesame script, along with the code needed to translate it) from OS straight into PyJs, see what's compatible, and then work on what isn't. Doing this, you probably wouldn't use PsychScript at all, as PyJs has it's own methods for working with the HTML5 canvas.

The second option would be to approach the problem from the JavaScript end: build up PsychScript structurally to the point where it could take a .opensesame file as input, parse, and then run it. This parsing could potentially be done server side (Python or PHP), meaning all that gets sent to the participant is pure JavaScript.

However, as I said, all this is quite speculative. For my part, I'm going to continue to work on PsychScript, because it's something I need as part of my own PhD (and OpenSesame mobile when I get the chance, I promise), and hopefully dove-tail with OpenSesame in the future.

Eoin

Edit: I've just realized that my 'second option' threads heavily in the footsteps of what @Maarten has said. Obviously, this is what I see as the easier path too.

shershen08

I think that splitting the "convertor" and the "player" (or a presenter) of a test by design is a good idea, so we write a "convertor" in a way that it generates a code, ready to use by the "player".

Probably it would be a good idea try to use as a "player" engine one of open source libraries working with HTML5 canvas, like http://kineticjs.com/ or http://canvasengine.net/ etc

@EoinTravers, @Maarten - concerning pro and cons where to make a conversion - it seems to me that doing it on client-side may be more useful.
In that way we'll be storing and uploaded OSS-file on server and it will be loaded into client-side application in a moment subject runs the test. It's a clean set of native instructions, that could be edited and forked via web-interface.

Otherwise there will keep on a server a mixture of JS-code intended only to be run by client-side application: 1) it'll be bigger by size; 2) impossible to make any noticeable corrections by hand for a non-progreammer

sebastiaan

it seems to me that doing it on client-side may be more useful. In that way we'll be storing and uploaded OSS-file on server and it will be loaded into client-side application in a moment subject runs the test. (...) Otherwise there will keep on a server a mixture of JS-code intended only to be run by client-side application.

If I understood correctly, that's not really what Maarten was suggesting. Or at least it was not what I had in mind when talking about a 'compiler'. I meant that a compiler writes a full HTML page, including all necessary JavaScript, directly from an OpenSesame script. This would bypass the need for a runtime environment altogether. To make things a bit more concrete, the compiler could be a Python script that allows you to do something like this:

python opensesame_to_html.py --in=my_experiment.opensesame --out=my_experiment.html

So there are two sensible options:

1. Write a compiler that builds an .html page from an OpenSesame script. In this scenario there would not be an online runtime environment at all. All the meat would be in the compiler, which could be written in any language (probably Python would be best).
2. Write an HTML5/ JavaScript runtime environment that interprets and executes an OpenSesame script in the browser. Here there would not be a compiler, and all the meat is in the runtime environment, which would need to be written in HTML5/ JavaScript.

Note that in neither scenario there is a real server-side to the implementation. Although I initially suggested option 2, I'm actually gravitating towards option 1 now. I think it might end up being easier to implement.

Any thoughts?

shershen08

I actually don’t get what you guys mean by “full HTML page” ?
If using Javascript to generate client experience in experiment - we basically need only head and body container tags, to initiate runtime (we actually can create them also, but that´s not the point).
This html-page will have links to other libraries ex. jQuery or and svg/canvas stuff, css-files maybe anyway?

Or you meant that compiler must create a “low-level” native Javascript code that can be run without any dependencies , like:

<script>
document.writeln("Hello World!");
document.createElement("div");
..

sebastiaan

Or you meant that compiler must create a “low-level” native Javascript code that can be run without any dependencies:

Right, exactly! The output of the compiler would then be a single .html file that you could upload somewhere, and voilà you have a running online experiment.

shershen08

But if we still need a server-side for: session management, database collecting results.

Maarten and me we've been thinking for the details of client-side implementation with something like this:

sebastiaan

Right, that looks like a possible implementation scheme, yes. The inner workings of the client-side part are still a bit unclear, though. What exactly do you mean with the terms in each of the boxes, e.g. what is the 'data exchange controller' and it what sense is it different from the 'runtime-libraries manager'?

shershen08

@sebastiaan by following i meant:

1) runtime-libraries manager - for a basic visualization we'll need a pack of several libraries (jQuery, some Canvas-js etc.) and (further) we'll need to switch on additional js-libraries like for adding web-camera support or smth. this manager will control asynchronous loading those libraries before the experiment starts. That's a common practice today for web-dev.


2) data exchange controller - we need to save each step on a sever, first when user has received the visual stimuli and then he/she gave some input. Of course not by each of them but in some batches maybe. Because in web-version we must be aware about network connectivity and cases like user has closed the browser tab or smth like this.

I'll draw a more detailed schema of experiment process itself later - so we could see key points in detail.

That's great that we're getting somewhere with more detailed concept!
because for me this "single .html" sounded strange, I am glad that we understand each other

sebastiaan

I just came across a tool that looks kind of interesting: ScriptingRT

• http://reactiontimes.wordpress.com/scriptingrt/

If I understand correctly (I'm not familiar with this technology), it appears to be a Flash-based set of routines that interprets .xml files and let's you run experiments in a browser.

I'm not sure whether this is something to base an OpenSesame runtime on, because I suspect that the functionality is too limited. But it's interesting to check out different approaches anyway.