Another "[Errno 24] Too many open files" error

ChattoRaptor22

Hi, everyone

The general structure of the experiment is as follows:
1. a short video (about 3000ms) is shown (no audio)-video file is the same for all images
2. then a image sketchpad is shown (2000ms)
3. and finally a "response" sketchpad is shown, here the subject have to choose between three response options
4. repeats 196 times
The experiment works well with a small pool of files (12 images) but when i tried to make quickrun with the real image pool (196 images), experiment crash in the last trials.

This is the complete error detail:
nexpected error
Details

item-stack: experiment[run].learning_loop[run].learning_sequence[prepare].pre_stimulation[prepare]
exception type: OSError
exception message: [Errno 24] Too many open files
time: Sun Apr 08 19:43:36 2018
Traceback (also in debug window)

  File "C:\Program Files (x86)\OpenSesame\lib\site-packages\libqtopensesame\misc\process.py", line 154, in run
  File "C:\Program Files (x86)\OpenSesame\lib\site-packages\libopensesame\experiment.py", line 439, in run
  File "C:\Program Files (x86)\OpenSesame\lib\site-packages\libopensesame\item_store.py", line 103, in execute
  File "C:\Program Files (x86)\OpenSesame\lib\site-packages\libopensesame\item_store.py", line 122, in run
  File "C:\Program Files (x86)\OpenSesame\lib\site-packages\libopensesame\sequence.py", line 51, in run
  File "C:\Program Files (x86)\OpenSesame\lib\site-packages\libopensesame\item_store.py", line 122, in run
  File "C:\Program Files (x86)\OpenSesame\lib\site-packages\libopensesame\loop.py", line 325, in run
  File "C:\Program Files (x86)\OpenSesame\lib\site-packages\libopensesame\item_store.py", line 102, in execute
  File "C:\Program Files (x86)\OpenSesame\lib\site-packages\libopensesame\item_store.py", line 142, in prepare
  File "C:\Program Files (x86)\OpenSesame\lib\site-packages\libopensesame\sequence.py", line 124, in prepare
  File "C:\Program Files (x86)\OpenSesame\lib\site-packages\libopensesame\item_store.py", line 142, in prepare
  File "C:\Program Files (x86)\OpenSesame\share\opensesame_plugins\media_player_mpy\media_player_mpy.py", line 119, in prepare
  File "C:\Program Files (x86)\OpenSesame\lib\site-packages\mediadecoder\decoder.py", line 162, in load_media
  File "C:\Program Files (x86)\OpenSesame\lib\site-packages\moviepy\video\io\VideoFileClip.py", line 111, in __init__
  File "C:\Program Files (x86)\OpenSesame\lib\site-packages\moviepy\audio\io\AudioFileClip.py", line 63, in __init__
  File "C:\Program Files (x86)\OpenSesame\lib\site-packages\moviepy\audio\io\readers.py", line 50, in __init__
  File "C:\Program Files (x86)\OpenSesame\lib\site-packages\moviepy\video\io\ffmpeg_reader.py", line 256, in ffmpeg_parse_infos
  File "C:\Program Files (x86)\OpenSesame\lib\subprocess.py", line 425, in __init__
OSError: [Errno 24] Too many open files   

eduard

Hi,
The error message seems to be quite straightforward, but the tricky part is to fix it. I have never worked with videos, so I don't know the syntax or what is happening during video playback, but if there is some coding interface somewhere, try to close the videos after you played them.

Hope this helps,

Eduard

ChattoRaptor22

Thanks for your reply Eduard i'll try to check the code!!!

Brendan

I was having this problem too. The only solution I found was to avoid using the media_player_mpy plugin. It seems that the plugin doesn't close the video adequately so there is a build up of open files, which becomes critical for memory. For a sign language lexical decision task with 400 trials (each with a 3 sec video) I was getting Too many open files errors or Out of memory errors after about 250 trials.

Based on snippets I found on the forum, I scripted the video presentation (with a keypress response) using OpenCV in an inline script as follows.

Prepare tab

import cv2
import numpy
import pygame

#this requires legacy backend

#create the pygame clock object
FPSCLOCK=pygame.time.Clock()

# Full path to the video file in file pool
path = pool[var.vid_file]

# Open the video
video = cv2.VideoCapture(path)

responseKey='None'

Run tab

keyboard().flush()

# set start time
start_time=self.time()
end_time=self.time()

# Play the video file until a specified key is pressed
while responseKey=='None':
    #set the frame rate to 50fps and wait if too fast
    FPSCLOCK.tick_busy_loop(50)
    # Get a frame
    success, frame = video.read()
    if not success:
        break

    # Rotate it, because for some reason it otherwise appears flipped.
    frame = numpy.fliplr(frame)
    frame = numpy.rot90(frame)
    # The video uses BGR colors and PyGame needs RGB
    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

    # check for keypress, which terminates video
    for event in pygame.event.get():
        if pygame.key.name(event.key) in ['m', 'z']:
            #get pygame key name
            responseKey=pygame.key.name(event.key)
            end_time=self.time()
            break

    # Create a PyGame surface
    surf = pygame.surfarray.make_surface(frame)
    # Show the PyGame surface at coordinates vidx and vidy
    exp.surface.blit(surf, (vidx, vidy))
    pygame.display.flip()

#calculate rt. Will be 0 if there is no response during the video.
rt = end_time-start_time
exp.set('response_time', rt)
exp.set('response', responseKey)
video.release()
cv2.destroyAllWindows()

The critical thing here seems to be video.release(). I also included cv2.destroyAllWindows() but I'm not sure if it's necessary.

The media_player_mpy plugin seems to be great for playing a few videos in an experiment, but if the experiment is video heavy, you need to handle the videos via scripting. Would be great if the plugin could be improved to avoid this problem, but maybe that's a big ask.

Daniel

Hi guys, I haven't worked on the media_player_mpy plugin for a while, but I obviously need to do so. I will check this whenever I work on it again. There are some other openstanding issues, and this one has been added by @humanboring so it is being tracked.

I am quite amazed this happens though, because I always try to be as meticulous as possible in letting my code clean up after itself. Another hunch I have is that this is caused by the prepare/run structure of OpenSesame; files are opened during the prepare phase, and played and closed after in the run phase. So if many videos are placed in a single sequence, they are all opened at once (because all prepare functions precede the first run function). Needless to say, this needs to be looked at...

camilo_r_r

Hi everyone,

I am having a very similar error running an experiment with videos, close to the end of the experiment (after about 80 iterations of the loop that plays videos) the experiment crashes and it shows an out of memory error. I've tried a bunch of different things (changing the backend, changing the backend settings, converting to videos to different formats) and nothing seems to work. The weird thing is that it works on my mac but not in the lab's PC! Anyone have any ideas?