[open] Playing Separate Video and Audio Files Simultaneously

hshatzer

I'm currently working on a crossmodal experiment that uses .avi files of a face saying a word (without sound) and separate .wav files of that word being spoken. So far, I have been able to play the video and the sound files in succession, but I have been unable to figure out how to play the audio file at the same time as the video (so that the spoken word matches up in time with the movement of the mouth). The procedure will ultimately be adaptive and move the sound file more out of sync with the video. First, however, I need to be able to play the video file in sync with the sound file. Is there a way to do this with Open Sesame?

Daniel

It is not really possible from the GUI and you will need to use an inline script to achieve this. I haven't tried out myself what I'm proposing below, so I can't guarantee that it is a working solution:

Add the following three items in a sequence:

• a sampler item with your audio file loaded (/its file name indicated in the Sound file field). Rename this sampler item to something appropriate (in this example I will just call it "video_dub").

• an inline script item with the following code in the run phase:

  exp.items["video_dub"].run()

• a media_player item with the audio muted and your video file loaded.

In the configuration pane of the sequence item in which you have placed these three items, change the run if field of video_dub from always to never. This will cause the audio file to be loaded, but not played once the sampler item is encountered in the run phase.

You start playback of your audio file with the inline_scipt, and because this item doesn't pause the experiment, playback of the movie should start immediately after. It might still be that your audio and video stream are out of sync, but I think it's easier to correct for this by incorporating a silence in the audio file itself. There's no easy way to correct for audio delays in OpenSesame.

That being said, why don't you incorporate the audio streams you would like to use in the movie file itself, by using VirtualDub or such? I think that is a much easier and more durable solution than the OpenSesame one that I just described.