Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

[open] Timing Accuracy and Stimuli Onset Times

riprip
edited December 2014 in OpenSesame

Hello,

I'm evaluating OpenSesame for a study that requires accurate timing and I'm trying to get an idea of how accurate the timing is. I've built an experiment that just displays a single image for 20 cycles at durations of 34 and 84ms (which I would think should yield durations of 50 and 100ms respectively taking the added refresh into account). My sequence includes a single sketchpad item and a logger. I am logging the time_sketchpad variable.

The durations that are being logged are frankly bizarre. Setting the sketchpad duration to 84ms yields the following
results (this is a smaller sample):

time_sketchpad - - onset to onset difference
1837 250
2087 267
2354 283
2637 284

I calculated the onset to onset difference in the second column.

Questions:

1) Why are the durations all greater the 200ms?

2) Why isn't there a constant duration error?

I would expect the onset to onset difference to be ~100.67ms (84ms duration + 16.67ms for missed refresh).

I'm using the Expyriment back-end

Thank you

Comments

  • edited December 2014

    How did you design your experiment?

    With a loop item?

    Then these delays would be explainable, as OS uses a prepare run strategy (see: here). The computer will prepare the whole trial sequence before running it and the additional delay might be the time your computer needs for this preparation. Therefore always put everything that needs to be timed accurately within one sequence.

    In other words:
    If that's what you did, try it out again by putting everything within a trial (a.k.a. sequence item).

    Best,
    Johannes

  • riprip
    edited 2:51PM

    In other words: If that's what you did, try it out again by putting everything within a trial (a.k.a. sequence item).

    Everything is in a sequence item. The prepare and run strategy is a poor choice with
    sequence items that have only one event in them. It looks like it's a combination of the data logger and stimulus preparation that causing the unexpected onset delays.

    The problem is that this method of bulk logging and stimuli preparation is adding
    what is essentially a random jitter between each trial in the experiment. This is not going to be acceptable for most of my tasks.

  • edited December 2014

    The loop and sequence items were designed with one trial in mind. That is: one trial is represented by one cycle in the loop (allowing you to set the specific parameters in that trial with variables), and runs through a single sequence. In this light, the strategy of preparing an entire sequence at once and then running it, makes a lot of sense.

    It is true that it adds random jitter to the end/start of a trial, but in most designs this is simply not a problem (it might even be beneficial, e.g. in event-related fMRI).

    Even though using one item per sequence in some cases might be a good idea, I expect your experiment can be adapted to fit within one sequence. If you give us some specifics (that is: what do you want your experiment to do?), we might even be able to help you do it.

    BTW: I think it's a very good idea to read what the documentation pages have to say about OpenSesame's timing

  • riprip
    edited December 2014

    The loop and sequence items were designed with one trial in mind. That is: one trial is >represented by one cycle in the loop (allowing you to set the specific parameters in that >trial with variables), and runs through a single sequence.

    Yes that's how I'm using the sequence.

    In this light, the strategy of preparing an entire sequence at once and then running it, >makes a lot of sense.

    Wouldn't a more sensible approach be to load the next stimulus while the current one is being presented (a la E-Prime, Presentation, etc...)?

    It is true that it adds random jitter to the end/start of a trial, but in most designs this >is simply not a problem (it might even be beneficial, e.g. in event-related fMRI).

    In fMRI sure, but even there I would want the jitter durations to be to known ahead of time. With OpenSesame, the loading time varies depending on how much is being logged and how much preloading needs to be done. And what if I actually do want to add a more controlled jitter? It would be difficult to do that being that OS is adding additional (unpredictable) jitter already.

    And I disagree that it's "simply not a problem". I think if you told most researchers their stimulus presentation system was adding a random jitter between the trials in
    their experiment they wouldn't be very happy.

    BTW: I think it's a very good idea to read what the documentation pages have to say >about OpenSesame's timing

    I have, I'm just surprised that this is the way it works. Yikes.

  • edited December 2014

    To clarify: You are not using the sequence in the advised way. You mention in the initial post that you use a single sketchpad per sequence, and calculate the difference between the onsets of two consecutive sketchpads (in two consecutive sequences, or trials). This means you are comparing across cycles, which means you are measuring the preparation time of the next cycle. Hence my referral to the Documentation page, where this is explained.

    To solve your problem, add a blank sketchpad (Duration of 0 ms) directly after the stimulus sketchpad (Duration set to the required presentation time). Then look at the onset differences between the two to test whether the timing was accurate.

    The only issue you are then left with, is the random jitter between trials. This can be large (tens of milliseconds) if the sequence requires a lot of preparation (e.g. when using large images). However, ITI jitter is a problem only in very few studies (I should add: in my experience). If yours is one of those, I would definitely advise you to use a programming language rather than a graphical interface, as the former gives you full control over your timing.

    Would it be more sensible to load the next image during the presentation of the current? For some designs, maybe. But I think your design here is a good example of the potential issues with such an approach: your stimulus requires a longer loading time than a single refresh cycle takes. Therefore, preparation of image n+1 during the presentation of image n would render it impossible to show image n for less than two frames.

    PS: to account for non-constants trial preparation durations, add the following code to an inline_script placed at the start of your sequence:

    Prepare phase

    # timestamp of preparation start
    exp.set("trialprepstart", self.time())
    

    Run phase

    # calculate preparation duration
    exp.set("trialprepduration", self.time()-exp.get("trialprepstart"))
    
    # pause for the required amount of time, e.g. 2 seconds
    self.sleep(1990 - exp.get("trialprepduration"))
    
  • edited December 2014

    Wouldn't a more sensible approach be to load the next stimulus while the current one is being presented (a la E-Prime, Presentation, etc...)?

    That would be nice, indeed, though -- according to my experience with Presentation -- this only works well if you have really a well chosen hardware and, most of all, a huge amount of RAM (at least if you present more than just pictures). I did a dummy experiment 2 years ago with visual and auditory stimuli. I set it up both with Presentation and OpenSesame on the same, not very new, PC (with just 4GB RAM -- at that time this was already something). I had jitters between trials both in Presentation and Opensesame. Overall, delays and jitters in Presentation very even larger than in Opensesame.

    With a preload strategy, you also have to employ a really accurate and quick "freeing the RAM again" strategy, which is very dependent on how quick your overall hardware setup is (mainboard, RAM, etc...). In many cases this preloading strategy also leads to crashes (which are probably mostly due to running out of memory), which is most of the time far worse than anything else. A preload strategy is also more difficult to maintain if you develop a software that should run on different operating systems, I guess...

    Anyway, if you put everything that has to be timed accurately into the same sequence then ITI jitter is no problem. Sometimes putting everything into the same sequence is a little more work and less elegant, that's true. But so far I could always find some solution/trick with the help of the nice community here ;-)

    For this it would be necessary, though, that you post some more details of your Opensesame script and explain a little more what you want to achieve in detail.

    Best, Johannes.

  • riprip
    edited December 2014

    This means you are comparing across cycles, which means you are measuring the >preparation time of the next cycle. Hence my referral to the Documentation page, >where this is explained.

    To solve your problem, add a blank sketchpad (Duration of 0 ms) directly after the >stimulus sketchpad (Duration set to the required presentation time). Then look at the >onset differences between the two to test whether the timing was accurate.

    But that wouldn't solve my problem. It's the between cycle jitter that I'm concerned about.
    Adding another sketchpad will just push the jitter down one event. It's still going to be there.

    With a preload strategy, you also have to employ a really accurate and quick "freeing >the RAM again" strategy, which is very dependent on how quick your overall hardware >setup is (mainboard, RAM, etc...).

    Right which is why the "load while presenting" strategy is better. Preloading all the stimuli for the next trial "upfront" certainly uses more memory than preloading the next
    event's worth of stimuli during presentation no? It also removes the inter-trial jitter.

    Anyway, if you put everything that has to be timed accurately into the same sequence >then ITI jitter is no problem.

    You all keep saying this but it's simply not true. OpenSesame is inserting a fairly large, random ITI into my experiments. Suppose I'm study habituation, and I don't know that OS is inserting this jitter (which I'm going to say is more than likely for most of your users). My results are flawed, period. There are many other areas of study that would similarly be effected by this. Hoping your users have a read a paragraph hidden away in your docs and are using something like the script Edwin described to compensate is really wishful thinking no? It would be interesting to do a poll of OS users to see, maybe I'm wrong and everyone knows about this behavior.

    Well thank you for your responses. It's been very informative.

  • edited December 2014

    Hi rip,

    Well, it is true what we are saying. If you put stimuli within one sequence they are accurately timed (all shorter jitter or delay occurring there is due to your hardware then. These delays occur no matter what presentation software you use (i.e. even with Presentation and the like. The only solution to this is getting better hardware).

    A sequence is equivalent to a trial so what we wanted to say is, if you enlarge your trials in a way that everything is inside that needs to be accurately timed, then ITI jitter cannot be an issue, because you are doing a workaround. If this strategy is not what you want you can program your own solution by using inline or external scripts.

    The 'load while presenting' strategy still needs a good memory management which I sadly experienced to be not as good, stable and accurate (at least in Presentation) as it should be. That might not be a problem of Presentation itself, but of ones hardware interacting with it.

    It is true, jitter can flaw results. You should discern within-trial-jitter and ITI jitter.
    In Opensesame and all other presentation software within-trial-jitter is very small and due to your specific hardware (i.e. refresh rate of your screen, onset speed of your sound card, etc.). Although you can reduce this by the right adjustments (e.g. enabling vsync, see documentation on timing). If this jitter is an issue, depends on how you want/need to design your experiment. If you want to have really exact millisecond timing, you should either buy a complete system (software+hardware from one manufacturer) as it is e.g. available from Neuroscan. These complete solutions are very expensive, though. Another solution would be -- if you have the knowledge or you can employ a technician and buy the appropriate measuring tools -- to carefully select and build up your own hardware.
    ITI jitter is larger with all software. It is less large with software that uses some kind of preload strategy, though this is often not a perfect way out, as preloading again depends on your hardware, this reduces your overall ITI jitter but may introduce other hardware related jitter and may sometimes decrease overall stability of the experiment run.

    That said, if you put all time sensitive procedures into the same sequence (as a workaround to the ITI jitter) you are left only with timing inaccuracies that are due to your hardware (if you did not do anything wrong elsewhere in your experiment programming, e.g. 'show if' vs. 'run if'). That workaround might be not very elegant with some designs, but as I said, you could program that part as an external script.

    Hoping your users have a read a paragraph hidden away in your docs and are using something like the script Edwin described to compensate is really wishful thinking no?

    Nothing is 'hidden' in the documentation there is a separate section on timing in which all timing related things are discussed and further information is linked. Timing is a technically very complicated issue. Opensesame does not hide this. I have worked with other presentation software that claimed to have millisecond precision where this just was not documented at all. When I compared timing of OS and other software on my hardware and with my experimental designs, I found out that OS was even more accurate. So, don't believe any 'millisecond precision' advertising, it is a) related to your hardware and b) related to your programming. Loops are bad for timing in any software, you have to work around it or your software has some built-in workaround. With the latter you have to trust and believe that software, the first is more transparent and you have the possibility to control it. And if you are in doubt, you can always discuss it in this forum.

    Best, Johannes.

Sign In or Register to comment.

agen judi bola , sportbook, casino, togel, number game, singapore, tangkas, basket, slot, poker, dominoqq, agen bola. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 50.000 ,- bonus cashback hingga 10% , diskon togel hingga 66% bisa bermain di android dan IOS kapanpun dan dimana pun. poker , bandarq , aduq, domino qq , dominobet. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 10.000 ,- bonus turnover 0.5% dan bonus referral 20%. Bonus - bonus yang dihadirkan bisa terbilang cukup tinggi dan memuaskan, anda hanya perlu memasang pada situs yang memberikan bursa pasaran terbaik yaitu http://45.77.173.118/ Bola168. Situs penyedia segala jenis permainan poker online kini semakin banyak ditemukan di Internet, salah satunya TahunQQ merupakan situs Agen Judi Domino66 Dan BandarQ Terpercaya yang mampu memberikan banyak provit bagi bettornya. Permainan Yang Di Sediakan Dewi365 Juga sangat banyak Dan menarik dan Peluang untuk memenangkan Taruhan Judi online ini juga sangat mudah . Mainkan Segera Taruhan Sportbook anda bersama Agen Judi Bola Bersama Dewi365 Kemenangan Anda Berapa pun akan Terbayarkan. Tersedia 9 macam permainan seru yang bisa kamu mainkan hanya di dalam 1 ID saja. Permainan seru yang tersedia seperti Poker, Domino QQ Dan juga BandarQ Online. Semuanya tersedia lengkap hanya di ABGQQ. Situs ABGQQ sangat mudah dimenangkan, kamu juga akan mendapatkan mega bonus dan setiap pemain berhak mendapatkan cashback mingguan. ABGQQ juga telah diakui sebagai Bandar Domino Online yang menjamin sistem FAIR PLAY disetiap permainan yang bisa dimainkan dengan deposit minimal hanya Rp.25.000. DEWI365 adalah Bandar Judi Bola Terpercaya & resmi dan terpercaya di indonesia. Situs judi bola ini menyediakan fasilitas bagi anda untuk dapat bermain memainkan permainan judi bola. Didalam situs ini memiliki berbagai permainan taruhan bola terlengkap seperti Sbobet, yang membuat DEWI365 menjadi situs judi bola terbaik dan terpercaya di Indonesia. Tentunya sebagai situs yang bertugas sebagai Bandar Poker Online pastinya akan berusaha untuk menjaga semua informasi dan keamanan yang terdapat di POKERQQ13. Kotakqq adalah situs Judi Poker Online Terpercayayang menyediakan 9 jenis permainan sakong online, dominoqq, domino99, bandarq, bandar ceme, aduq, poker online, bandar poker, balak66, perang baccarat, dan capsa susun. Dengan minimal deposit withdraw 15.000 Anda sudah bisa memainkan semua permaina pkv games di situs kami. Jackpot besar,Win rate tinggi, Fair play, PKV Games