[open] xpyriment does not force vsync per default: should we warn users?
Hi Sebastiaan,
A colleague just informed me that he thought he had seen some irregularities in displaying stimuli using the xpyriment back-end. After a quick test, I noticed that he was right. The script below is a simple test that should rapidly alternate the monitor colour (either black or white). It works perfectly with the legacy and psycho back-ends. With xpyriment, however, black and white lines appear on screen together! This is a clear sign of not waiting for the vertical refresh.
This behaviour only occurs when the vsync is not explicitly enabled in the graphics card settings, which will not come as a surprise to you (when it is forced, xpyriment behaves correctly). However, I think it would be a good idea to more explicitly warn users about this. Granted: the enabling of the v-sync is mentioned in the documentation of expyriment (and probably on osdoc too). But this is quite hard to miss and regular Intel graphics cards do not seem to have the option of enabling it (tested on a student's new windows 8 laptop: could not find ANY option for v-syncing anywhere!).
Of course, we could assume that the users that think timing is critical, all use computers with decent enough specifications and are aware of the fact that the syncing should be enabled. But I still think we should warn people about this and consider using either the psycho or the legacy as default back-end (since these DO enable the sync, regardless of the graphics card settings).
Cheers!
from openexp.canvas import canvas
from openexp.keyboard import keyboard
# create objects
canvas1 = canvas(exp)
canvas1.clear(color='white')
canvas2 = canvas(exp)
canvas2.clear(color='black')
kb = keyboard(exp)
# run until keypress
while True:
canvas1.show()
canvas2.show()
if kb.get_key(timeout=0)[0]:
break
Comments
Hmm, it's worrying that the xpyriment back-end does not enable v-sync when the legacy and psycho back-ends do. But it's difficult to give users concrete advice (e.g. "with the xpyriment back-end, you can always expect [...]"), because there is quite a bit of variation in how the different back-ends work across different systems. Still, I agree that it would be good to make users more aware of this, for example by adding a section on this to the tutorial.
As an aside, xpyriment came out poorly in a poll that I ran recently, being rated least used, least stable, and least preferable as default, so I'm considering switching away from it as a default.
Check out SigmundAI.eu for our OpenSesame AI assistant!
Hi, I just wanted to add my two cents.
From my experience, I must say that the xpyriment back-end seems the most stable of all. I did not experience any crashes or strange errors with it so far (I have used it since... , well, roughly from the middle of the pre-releases of OS 0.27).
However, I always had big trouble with the PsychoPy back-end: Sometimes it does not recognize relative paths any more and I have to use absolute ones.
Some experiments we have built here with the xpyriment back-end are running fine. When switching to PsychoPy they will just not run and one has to track down errors again in experiments that in effect were already prepared. Every now and then I also experienced crashes with PsychoPy. The main problem is that most of these errors were not reproduceable. On one computer they occur but not on the other (although running Windows on both of them) and sometimes problems come up only after some days. Generally, I think the source of all these problems is that I often copy OpenSesame-scripts back and forth between different computers, usb-drives etc.
The strange thing is that we never had any problems of this kind with xpyriment.
In my own timing tests expyriment was just fine for sending triggers:
(These 8 ms delay are probably produced by the sound card.)
I think that's not that bad/better than a comparable experiment programmed with presentation.
Cheers,
Johannes
As a side node, the v-sync of OpenGL-based apps is often controlled by specific settings of the video-card driver. If I look in my graphic card settings, I have to explicitly indicate that I want to use v-sync for 3D accelerated applications. If you switch on v-sync on driver level, expyriment might become synced to vertical refresh as well, but that's just a hunch...
I guess this illustrates how variable these things are. My personal experience is also more like this: Expyriment is faster and more stable than PsychoPy. Still, the poll suggests that most people feel otherwise. In case anybody is interested, here's the outcome of the poll (Daniel asked about it):
Which back-end do you use most?
In your experience, which back-end is most stable?
In your opinion, which back-end should be the default?
As you can see, the number of responses (in brackets) is fairly small, much smaller than the other questions about how important stability etc. are. I think this might reflect that many people are not sure what back-ends actually are, and are therefore unlikely to make an informed choice, simply going with the default (which is why the default is important). Unfortunately, I framed the other questions very poorly. asking participants to rate the importance of stability, temporal precision, and speed: Pretty much everybody marked everything as very important.
Check out SigmundAI.eu for our OpenSesame AI assistant!
I think that Xpyriment, a "new entry" in the OpenSesame back-ends, needs more time to be evaluated by OpenSesame users. In my experience, I use Psycho only when precision is important, otherwise I use Xpyriment which appears more stable. This choice is dictaded by a habit (and a prejudice) of mine to consider Psycho as the most accurate back end, but I could switch to Xpyriment in future, if I will check that it is fully reliable.
@ Daniel: adjusting the settings to enable v-sync is exactly what I did (see OP). The problem is that both legacy and psycho v-sync automatically, regardless of my graphic card settings. Xpyriment only v-syncs when this is explicitly enabled in the graphic card settings (my original point was that it's often quite difficult to adjust these settings on PC's with a standard Intel graphics card and that users might not be aware of the fact that they should adjust their graphic card settings in order to make sure OpenSesame behaves as promised).