[solved] Simultaneous image, video, and sound presentation

allision

Hello there,

I am creating a discrimination task for children where I need to present two images, two .avi files, and a .wav file simultaneously. There seems to be no straightforward way of doing this with the gui - has anyone else perhaps written code making this possible?

Thank you!
Allison

sebastiaan

Hi Allison,

You will need to do a bit scripting. Nothing too overwhelming, but maybe a bit tricky if you don't have much programming experience.

Your best bet is probably to use the PsychoPy video routines. In order to do this, you must use the psycho back-end. Then you can play a movie and simultaneously show a picture with something like the code below. Please see the PsychoPy documentation for details on all the functions: http://www.psychopy.org/api/api.html. If OpenSesame complains about missing modules (I'm not sure whether that will happen, but it might), please try the Python portable distribution.

from psychopy import visual
from openexp.keyboard import keyboard
myKeyboard = keyboard(exp, timeout=2)
movPath = '/path/to/movie.avi'
mov = visual.MovieStim(win, movPath)
picPath = '/path/to/picture.png'
pic = visual.PatchStim(win, tex=picPath)
while True:
    resp, time = myKeyboard.get_key()
    if resp != None:
        break
    mov.draw()
    pic.draw()
    win.flip()

Playing sound simultaneously is quite easy. Simply insert a sampler item and set the duration to 0. This will make the experiment advance right away, rather than waiting for the sound file to be finished.

Good luck!

Cheers,
Sebastiaan

allision

Hi Sebastiaan,

I've run into a couple more problems trying to build this experiment that hopefully you can help me with.

First off, after changing the back-end to PsychoPy, I am receiving the following error:

Traceback (most recent call last): <br>
  File "libqtopensesame\qtopensesame.pyo", line 1647, in run_experiment <br>
  File "libopensesame\experiment.pyo", line 258, in run <br>
  File "libopensesame\sequence.pyo", line 59, in run <br>
  File "libopensesame\loop.pyo", line 152, in run <br>
  File "libopensesame\sequence.pyo", line 119, in prepare <br>
  File "libopensesame\sketchpad.pyo", line 184, in prepare <br>
  File "openexp\_canvas\psycho.pyo", line 478, in image <br>
  File "PIL\Image.pyo", line 1980, in open <br>
IOError: cannot identify image file

After investigating, it looks as though the issue is that PsychoPy requires the exact path to the image file being used, whereas OpenSesame can only provide the filename.ext provided in the file pool. Also, I should mention I was attempting to run the experiment without yet implementing the code you provided above and only using the sketchpad.

Coincidentally, I am having a problem compressing/decompressing the files in my OpenSesame file pool because it contains around 240 .avi files. This causes OpenSesame not to respond whenever I am trying to save the entire package and makes it very slow to work on. It looks as though there is a place in the source code where one might be able to specify a folder (where pool_folder is by default set to 'none'), and I'm wondering whether there would be a fix that could address these problems simultaneously (i.e., allow me to call files from an outside source using an entire path name so my file pool is not so cluttered).

I realize this is a non-trivial problem that is difficult to express in a couple of paragraphs, so please ask if you need further clarification.

Thank you!
Allison

sebastiaan

After investigating, it looks as though the issue is that PsychoPy requires the exact path to the image file being used, whereas OpenSesame can only provide the filename.ext provided in the file pool. Also, I should mention I was attempting to run the experiment without yet implementing the code you provided above and only using the sketchpad.

OpenSesame will automatically translate to the full path, which refers to some temporary folder where the file pool has been extracted. So that's probably not the issue. The error sounds like PsychoPy cannot handle the file format, or doesn't recognize the file as an image. Could that be it? Is it some kind of unusual file format?

I'm wondering whether there would be a fix that could address these problems simultaneously (i.e., allow me to call files from an outside source using an entire path name so my file pool is not so cluttered).

Yep. If you simply specify the full path, OpenSesame will not attempt to look in the file pool. So you can bypass the file pool in that way, which is what I do as well when working with a large set of images/ videos/ sounds (for the same reason, because saving is too slow otherwise).

The algorithm for interpreting file names is as follows (if you're tech savvy you can check the source here: https://github.com/smathot/OpenSesame/blob/master/libopensesame/experiment.py#L330).

• First, check if the file is in the file pool
• If not, check if the file is in the same folder as the experiment
• Else, interpret the path as is, which can be a full path or a relative path from the current working directory.

Hope this clears things up!

froukehe

I've tried the code Sebastian is suggesting, but unfortunately, I get the following error message: object has no attribute 'ManagedSoundPlayer'. Would there be a way to solve this?

Thanks!