Timing of expyriment: it is pretty good!

Comments

• fladd

  April 2022

  Hi Christophe,

  
  

  yeah, unfortunately, the "Mega-Study" is quite problematic for multiple reasons (which we all outlined in our commentary: https://www.expyriment.org/timing-accuracy/).

  Thanks a lot for doing those tests and making them available! I really appreciate that. They are much more in line with the results we have observed in all our previous tests. Your data shows that Expyriment sometimes skips or misses one exact refresh rate, which is not too unusual (at least the missing), but then there also seem to be a few outliers that fall between refresh rates. Those are quite interesting, and I wonder whether they could be measuring errors? It would also be interesting to see the Expyriment test suite results for the visual timing test on that machine. You might be able to improve on your results still by taming the video card's power saving mode and turning off compositing in Ubuntu.

  We actually also started an effort to replicate the exact test scenario from the "Mega-Study" at some point in the past, but then Covid hit and that effort was put on hold. If you intend to run more tests, maybe you can have a look at our Expyriment scripts as well here: https://github.com/fladd/collaborative_timing_tests

  
  

  All the best,

  Florian

  Warning There was an error rendering this rich post.
• chrplr

  April 2022

  Thx for your answer.

  Indeed, I saw several flaws in the experiment script used for the mega study.

  Otherwise, I was also surprised by the durations that were not a multiple of refresh rate. It could be my settings of the opto threshold on the bbtkv2. I need to check with an oscilloscope recording the timecourse of the photodiode.

  I did not know the existence https://github.com/fladd/collaborative_timing_tests I actually sent a pull request on experiment-stash for a similar script two days ago... It is probably useless with this repo.

  I will run your test with the bbtkv2. One suggestion: maybe we could add to your repo `collaborative_timing_tests` a Markdown file containing a table listing hardware/OS and the results. THis may guide people who want to buy hardware.

  Finally I reran my timing test on my current laptop, a Dell Latitute Precission 5530 and, when the Nvidia GPU is selected with `prime-select`, not a single frame is missed on 1000 trials although I can see ~6% misses when the integrated intel driver is selected). All this under Ubuntu 20.04.