Simultaneously show image and play audio.

amoupsou · December 2023

Hi everyone,

I am trying to write a script that will play audio and show an image simultaneously. Every image shown is matched to a specific sound, they never change (for example, an image of a dog and a sound of a dog).

The code I have written thus far doesn't work. Can someone take a look? thank you in advance!

#loading images
bird = exp.get_file("bird.png")
cat = exp.get_file("cat.png")
cow = exp.get_file("cow.png")

#loading audios
bird_aud = exp.get_file("bird.wav")
cat_aud = exp.get_file("cat.wav")
cow_aud = exp.get_file("cow.wav")

#load list of possible stimuli, and shuffle it (i want to show one item randomly)
my_list_stimuli=[bird, cat, cow]
shuffle(my_list_stimuli)
#show one of the shuffled items.
var.stimulus1=my_list_stimuli[0]

#match images with audios
if var.stimulusImage == bird:
    var.stimulusAudio=bird_aud

elif var.stimulusImage == cat:
    var.stimulusAudio=cat_aud

elif var.stimulusImage == cow:
    var.stimulusAudio=cow_aud

#create sampler
src = pool[var.stimulusAudio]
my_sampler = Sampler(src)
my_sampler.play()
sleep(1000)

#create canvas
my_canvas1=Canvas(exp)
my_canvas1.image(var.stimulusImage)
my_canvas1.prepare()
my_canvas1.show()
clock.sleep(1000)

eduard · December 2023

Hi,

separate preparations and executions steps
get rid of the sleep statement for the sound

# emy_list_stimuli=[bird, cat, cow]
shuffle(my_list_stimuli)
#show one of the shuffled items.
var.stimulus1=my_list_stimuli[0]

#match images with audios
if var.stimulusImage == bird:
    var.stimulusAudio=bird_aud

elif var.stimulusImage == cat:
    var.stimulusAudio=cat_aud

elif var.stimulusImage == cow:
    var.stimulusAudio=cow_aud

#create sampler
src = pool[var.stimulusAudio]
my_sampler = Sampler(src)

#create canvas
my_canvas1=Canvas(exp)
my_canvas1.image(var.stimulusImage)
my_canvas1.prepare()

# execute everything
my_sampler.play()
my_canvas1.show()
clock.sleep(1000)

Eduard

amoupsou · December 2023

Hi Eduard,

I keep getting "The variable 'var.stimulusAudio' does not exist". Any clues?

eduard · December 2023

Probably your if /elif structure is not airtight. Add an else statement to find out:

if var.stimulusImage == bird:
    var.stimulusAudio=bird_aud

elif var.stimulusImage == cat:
    var.stimulusAudio=cat_aud

elif var.stimulusImage == cow:
    var.stimulusAudio=cow_aud
else:
    print("the Variable var.stimulusImage is neither bird, nor cat or cow")

Eduard

amoupsou · December 2023

I tried it but nothing. Is there a reason why it wouldn't be able to open a file if it is also synced in my drive?

Unable to open file 'C:\\Users\\annam\\OneDrive\\\u03a5\u03c0\u03bf\u03bb\u03bf\u03b3\u03b9\u03c3\u03c4\u03ae\u03c2\\Current OpenSesame Struggles\\__pool__\\elephant.wav'

eduard · December 2023

Hi,

This one has nothing to do with your earlier problem. It is a bit hard to follow you. Can you please provide more detail on what you want to accomplish, how you try it, and what the output is? Including error messages and code and preferably a simplified example of your experiment? Otherwise, I can't help you more.

Eduard

amoupsou · December 2023

Hi Eduard,

The experiment goes as such: I have a pool of 7 possible stimuli (pictures of animals). In each trial, I want to pick randomly three of those animals, present each one with the corresponing sound (for e.g. a dog and a bark), and in the end, pick randomly one of the 7 possible stimuli , and ask the participant whether it was presented previously, or not.

I am currently trying to build the presentation phase. However, I keep getting errors like this one

pygame.error: Unable to open file 'C:\\Users\\annam\\OneDrive\\\u03a5\u03c0\u03bf\u03bb\u03bf\u03b3\u03b9\u03c3\u03c4\u03ae\u03c2\\Current OpenSesame Struggles\\__pool__\\cat.wav'

However, all the files are uploaded in the file pool. The images in png format, and the audios in wav format.

I also attach the code below.

Thank you for your help!!!

from random import shuffle
from openexp.canvas import canvas
from openexp.sampler import sampler

bird = exp.get_file("bird.png")
cat = exp.get_file("cat.png")
cow = exp.get_file("cow.png")
dog = exp.get_file("dog.png")
duck = exp.get_file("duck.png")
elephant = exp.get_file("elephant.png")
bee = exp.get_file("bee.png")
goat = exp.get_file("goat.png")

bird_aud = exp.get_file("bird.wav")
cat_aud = exp.get_file("cat.wav")
cow_aud = exp.get_file("cow.wav")
dog_aud = exp.get_file("dog.wav")
duck_aud = exp.get_file("duck.wav")
elephant_aud = exp.get_file("elephant.wav")
bee_aud = exp.get_file("bee.wav")
goat_aud = exp.get_file("goat.wav")


#load list of stimuli and shuffle it
my_list_stimuli=[bird, cat, cow, dog, duck, elephant, bee, goat]
shuffle(my_list_stimuli)


########################
###first stimulus######
########################
#pick the first item to show from the shuffled items
var.stimulusImage=my_list_stimuli[0]
#create rule to match images with audios
if var.stimulusImage == bird:
   var.stimulusAudio=bird_aud

elif var.stimulusImage == cat:
   var.stimulusAudio=cat_aud

elif var.stimulus1 == cow:
   var.stimulusA=cow_aud

elif var.stimulusImage == dog:
   var.stimulusAudio=dog_aud

elif var.stimulusImage == goat:
   var.stimulusAudio=goat_aud

elif var.stimulusImage == elephant:
   var.stimulusAudio=elephant_aud

elif var.stimulusImage == bee:
   var.stimulusAudio=bee_aud

else: print("Error")

#create sampler for the first stimulus
src = pool[var.stimulusAudio]
my_sampler1 = Sampler(src)
#create canvas for the first stimulus
my_canvas1=Canvas(exp)
my_canvas1.image(var.stimulusImage)
my_canvas1.prepare()
#execute everything
my_sampler1.play
my_canvas1.show()
clock.sleep(1000)


########################
###second stimulus######
########################
#pick the second item to show from the shuffled items
var.stimulusImage=my_list_stimuli[1]


#create rule to match images with audios
if var.stimulusImage == bird:
   var.stimulusAudio=bird_aud

elif var.stimulusImage == cat:
   var.stimulusAudio=cat_aud

elif var.stimulus1 == cow:
   var.stimulusA=cow_aud

elif var.stimulusImage == dog:
   var.stimulusAudio=dog_aud

elif var.stimulusImage == goat:
   var.stimulusAudio=goat_aud

elif var.stimulusImage == elephant:
   var.stimulusAudio=elephant_aud

elif var.stimulusImage == bee:
   var.stimulusAudio=bee_aud

else: print("Error")

#create sampler for the second stimulus
src = pool[var.stimulusAudio]
my_sampler2 = Sampler(src)
#create canvas for the second stimulus
my_canvas2=Canvas(exp)
my_canvas2.image(var.stimulusImage)
my_canvas2.prepare()
#execute everything
my_sampler2.play
my_canvas2.show()
clock.sleep(1000)


########################
###third stimulus######
########################

#pick the third item to show from the shuffled items
var.stimulusImage=my_list_stimuli[2]

#create rule to match images with audios
if var.stimulusImage == bird:
   var.stimulusAudio=bird_aud

elif var.stimulusImage == cat:
   var.stimulusAudio=cat_aud

elif var.stimulus1 == cow:
   var.stimulusA=cow_aud

elif var.stimulusImage == dog:
   var.stimulusAudio=dog_aud

elif var.stimulusImage == goat:
   var.stimulusAudio=goat_aud

elif var.stimulusImage == elephant:
   var.stimulusAudio=elephant_aud

elif var.stimulusImage == bee:
   var.stimulusAudio=bee_aud

else: print("Error")

#create sampler for third stimulus
src = pool[var.stimulusAudio]
my_sampler3 = Sampler(src)
#create canvas for the third stimulus
my_canvas3=Canvas(exp)
my_canvas3.image(var.stimulusImage)
my_canvas3.prepare()
#execute everything
my_sampler3.play
my_canvas3.show()
clock.sleep(1000)

#delay
my_canvas_delay=Canvas(exp)
my_canvas_delay.clear()
my_canvas_delay.prepare()
my_canvas_delay.show()
clock.sleep(1000)

#####################
######question#######
#####################
#shuffle again the list
shuffle(my_list_stimuli)

#pick one item randomly for the question part.
var.question=my_list_stimuli[3]

#create rule to match images with audios
if var.question == bird:
   var.stimulusAudio=bird_aud

elif var.question == cat:
   var.stimulusAudio=cat_aud

elif var.question == cow:
   var.stimulusA=cow_aud

elif var.question == dog:
   var.stimulusAudio=dog_aud

elif var.question == goat:
   var.stimulusAudio=goat_aud

elif var.question == elephant:
   var.stimulusAudio=elephant_aud

elif var.question == bee:
   var.stimulusAudio=bee_aud

else: print("Error")

#create sampler
src = pool[var.stimulusAudio]
my_sampler5 = Sampler(src)
#create canvas
my_canvas_question=Canvas(exp)
my_canvas_question.text("Did you see it before?", center=True, x=500, y=200)
my_canvas_question.image(var.question)
my_canvas_question.prepare()
#execute
my_sampler5.play
my_canvas_question.show()

eduard · December 2023

Hi,

Can you follow this syntax here? Not sure it matters, but the exp.get_file is quite outdated and not necessary anymore. So you can just leave that out. Normally, if a file is in the filepool its original location is irrelevant.

Eduard

amoupsou · January 3

Hi Eduard.

Thank you very much. I tried the syntax but I keep getting other errors, so I will look for help for those first.

Thanks :)

Howdy, Stranger!

Categories

Simultaneously show image and play audio.

Comments

Howdy, Stranger!

Quick Links

Categories

Simultaneously show image and play audio.

Comments