Collecting responses while stimulus remains onscreen in OSWEB

Comments

• Daniel

  August 2020

  Hi @jeboydirk ,

  The scripting functionalities of osweb are still very limited, but we are working hard to improve and extend them. We added an inline_javascript item already, which allows you to specify some basic instructions in javascript (such as interacting with the var store), but sadly collecting responses with script is not possible yet.

  If I may ask, is it simply not possible implementing the loop you describe in osweb/opensesame (because you for instance need to control for latency which the UI does not allow), or is it just more conventient to script it. I don't see any reasons yet from your description why implementing such a loop would not be possible using the GUI, which would allow things to work in osweb.

  
• jeboydirk

  August 2020

  Hi Daniel,

  I don't see any reasons yet from your description why implementing such a loop would not be possible using the GUI, which would allow things to work in osweb.

  Maybe I misunderstood, but it is critical that each image is on for exactly 800 ms, and each ISI is also exactly 200 ms. At the same time pp, should be able to hit the space bar as many times as possible and I need to record the event times of a button press. As far as I know this is not easily done via the UI.

  The approach that I have taken now is to have a loop to show each item in the sequence (see image). At the onset of each loop I record the current time stamp. Within this loop I have to separate loops from which I break if the set time elapsed (800 ms for first loop and 200 ms for second loop). Both these loops contain the same keyboard item with a timeout of 5 ms. After each keyboard I then use an inline script to check whether a response was logged and if so save the timing of that event.

  

  Unfortunately, while the stream is being presented nicely, I seem to only rarely catch a button press, and for some reason miss the majority of them. I have no idea why this is the case? Any help on how to better do this is highly appreciated :)

  

  
  
• Daniel

  August 2020 edited August 2020

  Right, I see. Your timestamp approach is a good one. Javascript works asynchronously which means that unlike Python, it never really pauses (or halts operations) but only suspends them while it does other tasks in the background. (see https://www.youtube.com/watch?v=8aGhZQkoFbQ for a really good (but quite long) explanation on how this works and what this means. That clip finally made me understand the whole big deal about asynchronous code and how it differs from the linear approach that Python has).

  As a consequence, the big difference with Python is that you don't really need a while loop to listen for keyboard responses, but use event listeners that run in the background and trigger a function whenever the event they are listening for has been detected (such as a keypress).

  So what you could do, is just specify the stimulus items in the GUI with the timings you desire and catch and process all keyboard responses yourself in a script. To do so, place this piece of code at an inline script at the start of the trial:

  vars.keyboard_listener = function(event) {    
     if (event.code === "Space") {
        vars.correct = true
     } else if (event.code === "KeyS") {
        vars.correct = false
     }
     vars.t_keypress = event.timeStamp
     // Check if vars.key_presses exist and push the new value to it if so.
     // If not, create a new array with the current event.code as the first item.
     vars.key_presses = (vars.key_presses && vars.key_presses.push(event.code)) || [event.code]
  }
  
  document.addEventListener("keydown", vars.keyboard_listener)
  

  You can also use "keypress" instead of "keydown". See https://developer.mozilla.org/en-US/docs/Web/API/Document/keydown_event. You need to place the keyboard listening function in the vars object too, since it otherwise sadly doesn't persist across scripts yet.

  To be safe, also remove the event listener in a script at the end of the trial:

  document.removeEventListener("keydown", vars.keyboard_listener)
  

  Does this make sense? I know this is quite much to process, but I hope it helps.

  
• Daniel

  August 2020 edited August 2020

  I tried my example myself because it was untested, and turns out, it didn't work 😅. So the code below did work for me in osweb:

  /* Script at the start of a trial (run phase) */
  vars.keypresses = []
  
  vars.keyboard_listener = function(event) {
      // Just some example code on how to handle keypresses specified in an event:
      if (event.code === "Space") {
        vars.correct = true
      } else if (event.code === "KeyS") {
        vars.correct = false
      }
  
      vars.keypresses.push({ 
          code: event.code,
          key: event.key,
          timestamp: event.timeStamp
      })
  }
  
  if (typeof document !== "undefined") {
      // you can use this in other run_if statements further in the experiment
      vars.in_browser = true
      document.addEventListener("keydown", vars.keyboard_listener)
  } else {
      vars.in_browser = false
  }
  

  You can of course do as you please in the keyboard_listener function

  And to clean up, add this to a script at the end of the trial:

  /* Script at the start of a trial (run phase) */
  if (vars.in_browser) {
      document.removeEventListener("keydown", vars.keyboard_listener, true)
  }
  

  The document checks are to allow the code to also run in OpenSesame. Since the document object does not exist there (it is browser specific) any code addressing the document object should be skipped. However, this is a bit moot as the experiment won't run in OpenSesame anyway since it (or Python) can't handle responses asynchronously...

  See more information here about event.timeStamp: https://developer.mozilla.org/en-US/docs/Web/API/Event/timeStamp. You probably need ot convert this to a more sensible value.

  
• jeboydirk

  August 2020

  Hi Daniel,

  Thank you very very much! It works. I will continue to finish the experiment and also upload it here in case others are interested.

  The downside of this however is that I cannot testrun using the quickrun mode with this solution and thus also not print to the debug window using console.log(). I was wondering whether there is an alternative way to print stuff to the debug window while running the experiment in browser mode to see whether things are working as intended.

  Cheers,

  Dirk
• Daniel

  August 2020 edited August 2020

  and thus also not print to the debug window using console.log()

  console.log() should work in browser mode at least. It will not work in OpenSesame.

  In quick run mode in OpenSesame (non-browser), there are sadly no ways to do this, as it is very difficult to implement or mimic the event handling sequence as you did in JavaScript.

  
• jeboydirk

  August 2020

  Hmmm, maybe I am doing something wrong or I misunderstood but I have never been able to see the output of console.log() whenever I testrun the experiment in browser mode?
• jeboydirk

  August 2020

  Never mind, did not have a look yet on the debugging instructions for OSWEB. My bad :(
• Daniel

  August 2020

  😁 it doesn't appear in OpenSesame's debug window, but in the browser's. If you are using Chrome, you can open the browser console by pressing ctrl+shift+i (non-macs) or cmd+option+j (macs)

  
• jeboydirk

  August 2020

  Yes that is what I realised after reading the debug instructions :)
• jeboydirk

  August 2020

  Hey Daniel,

  I have a working experiment now which is great! However, there are still two aspects that probably could be improved. At the start of the training loop (loop that shows all images in sequence) I have created an empty array vars.training_presses. Each and every time an event (i.e., space bar press) is detected I add the current timestamp to this array such that at the end of training I have an array that contains the timings of all keypresses. The code of the function is shown below:/

  // function to detect keypresses during stimulus presentation
  vars.keyboard_listener = function(event) {
      //check for space bar event
      if (event.code === "Space" && vars.key_press === 0) {
          var temp_list = Array(vars.training_presses)[0]
          temp_list.push(new Date().getTime())
          console.log(['???', temp_list, vars.train_tr]) 
          vars.training_presses = temp_list;
          vars.key_press = 1; // prevents the array from being reset???
      }   
  }
  

  Unfortunately this doesn't work, as what seems as at random moments, the vars.training_presses becomes an empty Array and all the previously stored time stamps are lost. See below:

  

  I can work around this by having a logger at the end of the trial sequence rather than at the end of the loop, but this will result in an unnecessarily long log file.

  Also, in the function I have build in an extra check to prevent that multiple identical events are being logged. Now each time a button press is logged I reset a variable (vars.key_press) and reset it again next time I run the function as shown below:

  // check whether experiment is run in browser and events should thus be detected
  if (typeof document !== "undefined") {
      vars.in_browser = true
      document.addEventListener("keydown", vars.keyboard_listener)
      vars.key_press = 0;
  } else {
      vars.in_browser = false
  } 
  

  While this works the downside is that only a single button press is detected per event/trial, even though subjects are allowed to press multiple times (while incorrect). This is not necessarily a problem but it would be nice it could also be fixed.

  Once again thanks for all the help!
• Daniel

  August 2020 edited August 2020

  Hi @jeboydirk ,

  The array does not become empty, but the console represents it as a foldable object. the length parameters still says 6 so when you press the expansion symbol (triangle), you should see all values again.

  I don't understand the second problem though. On the one hand you state:

  I have build in an extra check to prevent that multiple identical events are being logged

  On the other hand you state

  even though subjects are allowed to press multiple times (while incorrect)

  but the latter are bound to be multiple identical events, right? (i.e. space bar presses). Do you mean events outside the period of the response collection?

  Regarding your code:

  // function to detect keypresses during stimulus presentation
  vars.keyboard_listener = function(event) {
      //check for space bar event
      if (event.code === "Space" && vars.key_press === 0) {
          var temp_list = Array(vars.training_presses)[0]
          temp_list.push(new Date().getTime())
          console.log(['???', temp_list, vars.train_tr]) 
          vars.training_presses = temp_list;
          vars.key_press = 1; // prevents the array from being reset???
      }   
  }
  

  I don't really understand what problem you are trying to solve here (is it that the array was reset, while it wasn't ? :) ), but you can also use a closure in this case

  
  

  function spaceListener () {
     var presses = []
     return function (event) {
        if (event.code === "Space") {
          presses.push(new Date().getTime())
          vars.training_presses = presses;        
        }   
     }
  }
  
  
  // function to detect keypresses during stimulus presentation
  vars.keyboard_listener = spaceListener()
  

  The fun thing about closures is that they remember the variable in the parent function space, so everytime the keyboard_listener function is called, the presses array will retain its previous state. Read more about the construct of closures here: https://medium.com/@prashantramnyc/javascript-closures-simplified-d0d23fa06ba4

  
• jeboydirk

  August 2020

  Ah that is another thing I didn't know about JavaScript, thanks!

  I don't understand the second problem though.

  What happens is, or at least what I think happens is, because the function is constantly checking for responses in the background when a key is pressed the if statement evaluates to true multiple times in very close temporal proximity. So multiple timestamps (differing only 1 msec) are added to the array giving the 'false impression' that space bar was pressed more than once. See example output below where I removed the vars.key_press === 0 check:

  

  
  
• jeboydirk

  August 2020

  This is not really a problem obviously as it is easily fixed when analysing the data offline. I was just curious to find out how I can prevent it from happening
• Daniel

  August 2020 edited August 2020

  Hmm, maybe instead of listening to keypress try keydown or keyup . Does that fix this problem?

  EDIT: Oh wait, you already are listening to keydown. Give me a sec.

  
• Daniel

  August 2020 edited August 2020

  It may be a cleanup issue, where the listener function is not property removed. Maybe the same function is attached multiple times to the same event, see: https://stackoverflow.com/questions/26146108/addeventlistener-firing-multiple-times-for-the-same-handle-when-passing-in-argum

  What you can also do is register the event handling function once at the beginning of the experiment and then indeed switch it on or off like you did with the vars.key_press variable (or I would rather call it vars.listen_for_space_presses = {true|false} ). You do not need to detach the event handler after each trial, as long as you indeed specify the boundaries of the time period it should work in by another method.

  Can you try the approach above? You cannot use the closure any longer, because you only register the event handling function once. So at the start of each trial you need to (for example) reset the variable holding the keypresses for that trial

  // At the start of each trial
  vars.trial_presses = []
  vars.listen_for_space_presses = true
  

  and then use this in the event handling function:

  // At the start of the experiment
  if (typeof document !== "undefined") {
      document.addEventListener("keydown", function(event) {
        if (vars.listen_for_space_presses && event.code === "Space") {
          vars.trial_presses.push((new Date()).getTime())
        }   
      })
  }