Counterbalancing

penguin

I was wondering how we can ensure that e.g. in an experiment with four different lists each list is completed by exactly the same number of participants. As in principle, several participants could do the study at the same time we did not figure out so far, how we can control this, especially if we do not want a list to be counted as "used" every time someone clicks on the study link but only if the study completes and data are saved.

I was thinking that probably this could be done using the batches and somewhere in my experiment script write a condition assorting each batch number or name a list. Is this possible? How exactly could it work (using jspsych)?

elisa

Hi Penguin,

You are right: it's a bit difficult to ensure that you have exactly the same amount of participants per condition, because it's possible that as one participant is completing the task, another one is starting it. There's no real way around this. But you can get a good enough approximation.

To do this is in jsPsych, you will have to write some code in a custom function. There might be a plugin that does something similar to this, but you will anyway need to modify it to send data to JATOS.

You could do it this way:

• When data collection starts (i.e., if the batch session is empty, before your very first participant), initialize your batch session data with (say) four possible conditions, and a counter for each of them, to store the number of subjects assigned to each.
• When a new participants starts the experiment, get the condition with the minimum count and assign it.
• When participants actually finish the study, update the new subject count, using something like this:

jsPsych.init({
    experiment_structure: tasks,
    // your jsPsych init code here... 
    on_finish: function () {
        
        //update the batch session data with the new subject count once the subject finished the whole session

        jatos.endStudy();
    }
});

We've had similar questions in the google group as well, see here where I posted a (messy, but hopefully useful) example code to do this:

https://groups.google.com/forum/#!msg/jatos/0-ZNufywQm0/n7-7NNwcBwAJ

Hope this helps!

Elisa

elisa

Actually, I just remembered that we also have an example study doing this (so it's a lot better than my messy, idiosyncratic example). Check the example "Randomize Tasks Between Workers" in our example studies page: http://www.jatos.org/Example-Studies.html

penguin

Dear Elisa,

thanks for your quick reply and the link to the example study. I will try out the code!