Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

Building experiment with videos

Hey everyone,

Following a recent mail exchange with Florian, I tranfer my issues to the forum to get some insight from the community.

For my first experiment I need to design an experiment using moving stimuli and I have to implement an expending or concentric movement on a circle (increase and decreasing the diameter of the circle in a way) and on annular gratings (Gabor Patch). So we conclude that it would be easier to just create videos of the stimuli and to present them.

So i went over the begginer's tutorial and tried to follow this example to create a very simple, 1 block, 4 stimuli experiment. I also follow the instruction about playing videos but I'm unsure on how to build and on how to code this efficiently. I coded the following experiment this morning but only return an error during the preloading of the video.

Anyway, I'll join what I coded so we can share thoughts and ways to improve things so that I can learn to efficiently code experiments in the future.

`import expyriment

exp = expyriment.design.Experiment(name="First Experiment")
expyriment.control.initialize(exp)

block_one = expyriment.design.Block(name="Block_1")

trial_one = expyriment.design.Trial()
trial_two = expyriment.design.Trial()
trial_three = expyriment.design.Trial()
trial_four = expyriment.design.Trial()

circle_s = expyriment.stimuli.Video('Circle_S.mp4')
circle_s.preload()
circle_s.play()
circle_s.present()
video_presentation_time = exp.clock.time
circle_s.wait_end()
circle_s.stop()

circle_e = expyriment.stimuli.Video('Circle_E.mp4')
circle_e.preload()
circle_e.play()
circle_e.present()
video_presentation_time = exp.clock.time
circle_e.wait_end()
circle_e.stop()

gabor_s = expyriment.stimuli.Video('Gabor_S.mp4')
gabor_s.preload()
gabor_s.play()
gabor_s.present()
video_presentation_time = exp.clock.time
gabor_s.wait_end()
gabor_s.stop()

gabor_e = expyriment.stimuli.Video('Gabor_E.mp4')
gabor_e.preloas()
gabor_e.play()
gabor_e.present()
video_presentation_time = exp.clock.time
gabor_e.wait_end()
gabor_e.stop()

trial_one.add_stimulus(circle_s)
trial_two.add_stimulus(circle_e)
trial_three.add_stimulus(gabor_s)
trial_four.add_stimulus(gabor_e)

block_one.add_trial(trial_one)
block_one.add_trial(trial_two)
block_one.add_trial(trial_three)
block_one.add_trial(trial_four)

exp.add_block(block_one)

expyriment.control.start()
for block in exp.blocks:
for trial in block.trials:
trial.stimuli[0].present()
exp.clock.wait(1000)

expyriment.control.end()`

Also; the error I get looks like :

Traceback (most recent call last): File "test.py", line 20, in <module> circle_s.preload() File "/home/jrdnalvs/anaconda3/envs/expyriment/lib/python3.4/site-packages/expyriment/stimuli/_video.py", line 303, in preload videorenderfunc=self._update_surface) File "/home/jrdnalvs/anaconda3/envs/expyriment/lib/python3.4/site-packages/mediadecoder/decoder.py", line 73, in __init__ if not self.load_media(mediafile, play_audio): File "/home/jrdnalvs/anaconda3/envs/expyriment/lib/python3.4/site-packages/mediadecoder/decoder.py", line 162, in load_media self.clip = VideoFileClip(mediafile, audio=play_audio) File "/home/jrdnalvs/anaconda3/envs/expyriment/lib/python3.4/site-packages/moviepy/video/io/VideoFileClip.py", line 91, in __init__ fps_source=fps_source) File "/home/jrdnalvs/anaconda3/envs/expyriment/lib/python3.4/site-packages/moviepy/video/io/ffmpeg_reader.py", line 33, in __init__ fps_source) File "/home/jrdnalvs/anaconda3/envs/expyriment/lib/python3.4/site-packages/moviepy/video/io/ffmpeg_reader.py", line 243, in ffmpeg_parse_infos is_GIF = filename.endswith('.gif') TypeError: endswith first arg must be bytes or a tuple of bytes, not str

thanks again for your time,

Comments

  • The error is related to a bug. This has been fixed, but no official new version of Expyriment has been released yet.

    You can either use the development snapshot directly from the repository (github.com/expyriment/expyriment/) or you can install a beta version of Expyriment (https://github.com/expyriment/expyriment/releases/tag/v0.9.1b2).

    The new official release of Expyriment (0.9.1) will include this fix.

    Florian Krause (Developer)
    http://www.expyriment.org

  • Hi !

    Thanks for that, switching to the beta version fixed the problem.
    Now during the preparation of the experiment I get this error message :

    Traceback (most recent call last):
      File "Pilot_test.py", line 24, in <module>
        circle_s.present()
      File "/home/jrdnalvs/anaconda3/envs/expyriment/lib/python3.4/site-packages/expyriment/stimuli/_video.py", line 496, in present
        self.update()
      File "/home/jrdnalvs/anaconda3/envs/expyriment/lib/python3.4/site-packages/expyriment/stimuli/_video.py", line 514, in update
        self._surface, quadDims=(1,1,1,1), position=self._position)
      File "/home/jrdnalvs/anaconda3/envs/expyriment/lib/python3.4/site-packages/expyriment/stimuli/_visual.py", line 62, in __init__
        surface_size = surface.get_size()
    AttributeError: 'numpy.ndarray' object has no attribute 'get_size'
    

    The video file is in .mp4, so the experiment video module should be able to launch it.
    Is it because of a mistake in the experiment coding ? Should I specify the size of the screen/canvas to presents the video on ?

    Thanks again,

  • Ah, now you ironically ran into another bug. This bug, however, is also fixed already, but not in the beta yet. We will release a new beta with the fix soon, but for now, you can install Expyriment directly from the repository. That should work.

    Florian Krause (Developer)
    http://www.expyriment.org

  • Hi !
    Is installing from the repository = git clone from the git hub page in my environment ? Because if yes it still really doesn't work (now I return the error that design isn't a module in expyriment). If not there's no real documentation on how to do that in the installation docs pages. Can you help/guide through it ?

  • Hi,

    easiest is to just download the zip (expyriment-master.zip) file. You install it then in the standard way to install Python packages: pip install expyriment-master.zip (assuming you are in the directory wherethat file is located).

    Florian Krause (Developer)
    http://www.expyriment.org

  • Hi ! Thanks for that !

    It's now working, the videos are loading and are presented so I can go back to my experiment design.
    One question now would be : how do I get a smooth rendering of the videos ? Because it keeps on dropping frames resulting in a non smooth video and poor quality.

  • What resolution are the videos in? On the machines I tested it on (Windows, Mac, Linux) I could play full HD videos without dropping any frames.

    How exactly are you showing the videos? Can you post your code?

    Florian Krause (Developer)
    http://www.expyriment.org

  • The videos were created in 4K res.

    For now I just tested out the basic way of preload, play, present, stop & unload. Now I'm trying to create trials that will present 100+ the videos to present in one a block.

  • Since the videos are played without hardware acceleration, 4K can indeed lead to dropped frames, I am not too surprised by this. It's a very high resolution. You might consider recoding to full HD (or try with a more powerful machine; i.e. with a more powerful CPU). Maybe changing the video format can also help (to something uncrompressed), but I am not sure about this.

    Florian Krause (Developer)
    http://www.expyriment.org

  • Hi Florian !

    So I changed the stimuli and now everything run smoothly. I also found on a discussion here something some how similar to what I want to do so I tried to use it as a base to code something for me.
    I don't get any error, the experiment seems to run as I get to the goodbye message but it won't present the videos.

    Here's what I tried to code to loop and present the videos randomly multiple time :

    import expyriment as xpy
    
    exp = xpy.design.Experiment("First experiment")
    xpy.control.initialize(exp)
    
    videos = {"circle_s": xpy.stimuli.Video('/home/jrdnalvs/Documents/Pilot_exp/Stim/Circle_S.mp4'),
              "circle_e": xpy.stimuli.Video('/home/jrdnalvs/Documents/Pilot_exp/Stim/Circle_E.mp4'),
              "gabor_s": xpy.stimuli.Video('/home/jrdnalvs/Documents/Pilot_exp/Stim/Gabor_S.mp4'),
              "gabor_e": xpy.stimuli.Video('/home/jrdnalvs/Documents/Pilot_exp/Stim/Gabor_E.mp4')}
    for video in videos:
        videos[video].preload()
    
    canvas = xpy.stimuli.BlankScreen(colour=(255,255,255))
    canvas.preload()
    
    for motion_type, patterns in {"shrinking": ["circle_s", "gabor_s"],
                                  "expanding": ["circle_e", "gabor_e"]}.items(): 
        block = xpy.design.Block()
        block.set_factor("MotionType", motion_type)
        for trial_repetition in range(100):
            trial = xpy.design.Trial()
            pattern = xpy.design.randomize.rand_element(patterns)
            trial.set_factor("Pattern", pattern)
            trial.add_stimulus(videos[pattern])
        exp.add_block(block)
    
    ########## START ##############
    xpy.control.start()
    
    canvas.present()
    
    videos.present()
    videos.play()
    videos.wait_end()
    videos.stop()
    
    ######## END EXPERIMENT ########
    xpy.control.end(goodbye_text="Thank you for participating",
                           goodbye_delay=3000)
    

    I also tried some variations of that such as :

    xpy.control.start()
    
    canvas.present()
    for block in exp.blocks:
        for trials in block.trials:
            videos.present()
            videos.play()
            videos.wait_end()
            videos.stop()
    

    Or also things along those lines :

    xpy.control.start()
    
    canvas.present()
    for block in exp.blocks:
        for trials in block.trials:
            block.trials.stimuli.present()
            block.trials.stimuli.play()
            block.trials.stimuli.wait_end()
            block.trials.stimuli.stop()
    

    But nothing seems to do the trick. Any advice on how to get this to play the videos ?
    Thanks for the support !

  • edited September 24

    Hi,

    Two things:

    1. You forgot to add your trial to the block:
    for trial_repetition in range(100):
        trial = xpy.design.Trial()
        pattern = xpy.design.randomize.rand_element(patterns)
        trial.set_factor("Pattern", pattern)
        trial.add_stimulus(videos[pattern])
        block.add_trial(trial)
    exp.add_block(block)
    
    1. You need to address a specific stimulus, not the entire stimulus list. Since you only add a single stimulus, it is the first element:
    for block in exp.blocks:
        for trial in block.trials:
            trial.stimuli[0].play()
            trial.stimuli[0].present()  # only needed if you need the exact timing of this presentation
            trial.stimuli[0].wait_end()
            trial.stimuli[0].stop()
    

    Florian Krause (Developer)
    http://www.expyriment.org

  • Hi Florian !

    Thanks for that ! I tried it and it runs all the trials. There's also some frame that are still droped even after I lowered the quality of the video but I guess this is just my personal settings that are not as powerful as I wished.
    There's also this things, sometimes, the BlankScreen doesn't load and leave the back screen black and only the video playing in the middle. Besides my graphic settings, do you think of any possible other causes for that ?

    Thanks again for everything,
    Jordan.

  • Well, in your code above you only present the blank screen once at the beginning of the experiment.Not sure how your current code looks like.

    Florian Krause (Developer)
    http://www.expyriment.org

  • It's pretty much the same as it is above, I just added your comments.
    Should I implement a blankscreen before each stimuli presentation ?
    The thing is, it's there sometimes but the pattern seems random. It just appears for 10+ stim and then it's not presented again etc..

  • Given your code, the presentation of the blank screen cannot be random. In fact, it is only presented a single time, in the beginning. Afterwards, it is not presented anymore. If you are still seeing it afterwards, it might be related to a potential double buffer.
    If you want to see it before each stimulus, then you have to present it explicitly before each stimulus.

    Florian Krause (Developer)
    http://www.expyriment.org

  • Okay !
    I added it on the loop of stimuli presenting, ending up to something like :

    xpy.control.start()
    
    canvas.present()
    for block in exp.blocks:
        for trial in block.trials:
            canvas.present()
            trial.stimuli[0].play()
            trial.stimuli[0].present()
            trial.stimuli[0].wait_end()
            trial.stimuli[0].stop()
    

    If I want to link a different marker to each stimuli, should I embed the expyriment.io.MarkerOutput function in the stimulus or should I create another object and include it in the loop ?

  • You only need one marker output object. On every call to its send method you can tell it what marker to send. Please also see: http://docs.expyriment.org/expyriment.io.MarkerOutput.html

    Florian Krause (Developer)
    http://www.expyriment.org

  • Hi !

    So I tried things but it does seem to do what I want it to do. Also, I can't really seem to get the logic behind that because in my attempt here :

    import expyriment as xpy
    xpy.control.defaults.open_gl = False
    samplerate=5000
    
    exp = xpy.design.Experiment("First experiment")
    xpy.control.initialize(exp)
    
    videos = {"circle_s": xpy.stimuli.Video('/home/jrdnalvs/Documents/Pilot_exp/Stim/Circle_S.mp4'),
              "circle_e": xpy.stimuli.Video('/home/jrdnalvs/Documents/Pilot_exp/Stim/Circle_E.mp4'),
              "gabor_s": xpy.stimuli.Video('/home/jrdnalvs/Documents/Pilot_exp/Stim/Gabor_S.mp4'),
              "gabor_e": xpy.stimuli.Video('/home/jrdnalvs/Documents/Pilot_exp/Stim/Gabor_E.mp4')}
    
    for video in videos:
        videos[video].preload()
    
    markers = {'1':xpy.io.MarkerOutput(interface=xpy.io.SerialPort(port='name'),
                                       default_code=1, 
                                       default_duration=1000/samplerate),
               '2':xpy.io.MarkerOutput(interface=xpy.io.SerialPort(port='name'),
                                       default_code=2, 
                                       default_duration=1000/samplerate),
               '3':xpy.io.MarkerOutput(interface=xpy.io.SerialPort(port='name'),
                                       default_code=3, 
                                       default_duration=1000/samplerate),
               '4':xpy.io.MarkerOutput(interface=xpy.io.SerialPort(port='name'),
                                       default_code=4, 
                                       default_duration=1000/samplerate)}
    
    canvas = xpy.stimuli.BlankScreen(colour=(255,255,255))
    canvas.preload()
    
    for motion_type, patterns in {"shrinking": ["circle_s", "gabor_s"],
                                  "expanding": ["circle_e", "gabor_e"]}.items():   # use iteritems() in Python 2
        block = xpy.design.Block()
        block.set_factor("MotionType", motion_type)
        for trial_repetition in range(100):
            trial = xpy.design.Trial()
            pattern = xpy.design.randomize.rand_element(patterns)
            trial.set_factor("Pattern", pattern)
            trial.add_stimulus(videos[pattern])
            block.add_trial(trial)
        exp.add_block(block)
    
    ########## START ##############
    xpy.control.start()
    
    canvas.present()
    for block in exp.blocks:
        for trial in block.trials:
            canvas.present()
            trial.stimuli[0].play()
            trial.stimuli[0].present()
            trial.stimuli[0].wait_end()
            trial.stimuli[0].stop()
    
    ######## END EXPERIMENT ########
    xpy.control.end(goodbye_text="Thank you for participating",
                           goodbye_delay=3000)
    

    what I present is randomized. So I'm afraid it will call any random marker when I want it to send a specific marker for a specific video, not to choose randomly in the markers.
    So I'm kind of lost on how to actually code this. I went to the doc but I don't really get it either.

    Thanks for helping
    Jordan.

  • In your code, you never send any marker. Also, you do not need to create a marker object for each marker.
    Assuming, that you want to send a marker everytime a video is being presented, here is some code:

    import expyriment as xpy
    xpy.control.defaults.open_gl = False
    samplerate=5000
    
    exp = xpy.design.Experiment("First experiment")
    xpy.control.initialize(exp)
    
    videos = {"circle_s": xpy.stimuli.Video('/home/jrdnalvs/Documents/Pilot_exp/Stim/Circle_S.mp4'),
              "circle_e": xpy.stimuli.Video('/home/jrdnalvs/Documents/Pilot_exp/Stim/Circle_E.mp4'),
              "gabor_s": xpy.stimuli.Video('/home/jrdnalvs/Documents/Pilot_exp/Stim/Gabor_S.mp4'),
              "gabor_e": xpy.stimuli.Video('/home/jrdnalvs/Documents/Pilot_exp/Stim/Gabor_E.mp4')}
    
    for video in videos:
        videos[video].preload()
    
    marker = xpy.io.MarkerOutput(interface=xpy.io.SerialPort(port='name'),  # don't forget to replace 'name' by the actual port name!
                                 default_duration=1000/samplerate) # not sure why you need this, but I left it in
    markers = {'circle_e': 1,
               'circle_s': 2,
               'gabor_s': 3,
               'gabor_e': 4}
    
    canvas = xpy.stimuli.BlankScreen(colour=(255,255,255))
    canvas.preload()
    
    for motion_type, patterns in {"shrinking": ["circle_s", "gabor_s"],
                                  "expanding": ["circle_e", "gabor_e"]}.items():   # use iteritems() in Python 2
        block = xpy.design.Block()
        block.set_factor("MotionType", motion_type)
        for trial_repetition in range(100):
            trial = xpy.design.Trial()
            pattern = xpy.design.randomize.rand_element(patterns)
            trial.set_factor("Pattern", pattern)
            trial.add_stimulus(videos[pattern])
            block.add_trial(trial)
        exp.add_block(block)
    
    ########## START ##############
    xpy.control.start()
    
    canvas.present()
    for block in exp.blocks:
        for trial in block.trials:
            canvas.present()
            trial.stimuli[0].play()
            trial.stimuli[0].present()
            marker.send(markers[trial.get_factor("Pattern")])
            trial.stimuli[0].wait_end()
            trial.stimuli[0].stop()
    
    ######## END EXPERIMENT ########
    xpy.control.end(goodbye_text="Thank you for participating",
                           goodbye_delay=3000)
    

    Florian Krause (Developer)
    http://www.expyriment.org

  • Okay ! Thanks a lot for the correction & the support :)
    I'll get the name of the port name & test it out !

Sign In or Register to comment.