As you know, I've started trying to write a screen recorder plugin. This plugin would allow the user to write (either all or a selection of) image files to the disk, to capture what's on the canvas at certain times. At the end of the experiment (-- or actually, at any point), I want the plugin to offer the user the possibility of collecting all these images and generating a video file. (to me, this just screams openCV). I have a bunch of issues that I'd like to talk through before I fully start hacking:
To write a selection of 'screencaps' away to a file doesn't seem too difficult, as the openexp.canvas.surface object would be callable from anywhere, right? (as long as a canvas has been created)
Pygame itself aready contains functions to save images using surface objects (pygame.image.save). However, I'm thinking to maybe import openCV to do this, as this will offer more flexibility to the user in terms of output format or output geometry (e.g. shrink the surface; one will not always want to generate large fullscreen images). Would there be any objection to this?
One interesting detail would be that, as this operation would be uncoupled from 'openexp.canvas.show()', this would mean that there is a possibility that the user generates images that are not necessarily those that are on the screen at a given point. Do you feel this is a problem? Or does simply reporting this in the documentation seem sufficient?
Making a plugin that writes images everytime .show() is called, seems a bit more tricky, and maybe even unfeasible at this point. You suggested writing an extension to the canvas class which extends .show(), but that seems to be difficult because of precisely this stament which determines the way it morphs:
openexp/canvas.py, l. 46:
exec("openexp._canvas.%s.%s.init(self, experiment, bgcolor, fgcolor)" % (experiment.canvas_backend, experiment.canvas_backend))
=> which seems to imply that whatever extension is used, the way it is now, it needs to be in the openexp._canvas module;
I feel there are two solutions here:
1) I could write a canvas class which would need to be placed in _canvas . The object would then use similar morphing, and take on the form of whatever canvas class is used, and would implement it's own show()-function which also writes the image file.
The downside of this is, that then the plugin wouldn't be much of a 'plugin' the way they were intended: you can't simply download it, store it in plugins/ and expect it to work anymore; However, a screen_recorder item clearly seems to be one of those things that not all users want or need, and clearly a thing that one DOES want to work in a plugin fashion.
2)the canvas constructor could be extended to make it possible to have plugins define the desired canvas to be used, thus including a constractor statement that looks whether experiment.canvas_backend (?) is one of the 'normal' types, meaning it would look for its definition in openexp._canvas, or look for it elsewhere.
Maybe an even more elegant option would be that experiment would get an extra variable, e.g. 'canvas_definition_path' ; usually this would be set to openexp_canvas, and a plugin could set this varibale to plugins. before any canvas is created. In that case, if the plugin wants to modify the canvas definition, it could, in that folder, implement the new class definitions for every backend type (or whichever way, morph into an extended version of the appropriate backend canvas).
Extending the plugin to eventually form a movie file would then not be too difficult; python's openCV implementation has functions that are more than capable of doing this, so a simple item could be implemented that collects all images with a predetermined filename, sorts them using their timestamp and creates a movie file out of them.
One remaining issue: would you think it's necessary that the files generated by this plugin are automatically stored in the file_pool ? Or does that seem somewhat irrelevant?
I'm curious to hear your thoughts about this!