Having trouble reliably counterbalancing in JATOS using OSweb
Hi everyone,
I’m trying to build two new experiments in which I want to counterbalance over 2 and 4 conditions respectively using the subject_nr variable. I’ve just completed two pilots that ran simultaneously (with two conditions) and I’ve found that the distribution of the two conditions is very skewed (17 vs 30!). After some digging, I think I understand what went wrong but I’m not sure how to fix it. Some questions that are on my mind:
1. Is it correct that that the formula for determining the subject_nr is: jatos.componentResultId%possible subject number? I’m basing this on the line from the generated jatos.js: ‘subject_nr = possible_subject_nrs[jatos.componentResultId % possible_subject_nrs.length]’. The 'possible subject nrs' variable should then be determined by what I put in the possible subject field in OS’ GUI (i.e. 1,2).
2. If 1 is true, this means that having two studies run simultaneously interfered with the counterbalancing procedure (because the jatos.componentResultId numbers were split among my two pilot studies). I won’t be running anything else besides this future study so that would be solved, but it would be more problematic if these numbers are divided among all users on my university’s server. Does anyone know how allocation of jatos.componentResultId works exactly? In other words: Are the jatos.componentResultId numbers incremented account wide or server wide?
3. In the case this is all wrong or I have overlooked something obvious: do you have a suggestion on how to implement counterbalancing in a reliable way? I know I could make individual links, but I would like to avoid manual labor. Not sure if this is relevant, but I’m using General multiple links.
Let me know if you need more information! Thanks in advance,
Best,
Tom
Posts I used:
https://www.jatos.org/Cross-sectional-and-longitudinal-studies.html (the example counterbalancing script does not work for me, probably because our server is still running 3.5.4. Second thing I tried was using the jatos.worker.Id but as far as I can see this increments in the same way as jatos.componentResultId)
https://forum.cogsci.nl/discussion/5930/jatos-how-to-assign-or-collect-participant-number
https://forum.cogsci.nl/discussion/6413/subject-number-as-worker-number
https://forum.cogsci.nl/discussion/5939/how-to-get-participant-number
Comments
Hi Tom,
Sorry for the delay.
To answer your question: componentResultID (and in general all IDs) increase serverwide. So it's a bad idea to use that, or even the subjectID, to randomize. I realize this is probably a recommendation from OSWeb, maybe we can raise this issue with them.
I can't really help much if you want to implement it on OSWeb (in which case it would be better to ask in their forum). Instead, I would really recommend you add another component at the beginning where you randomize your participant assignment with full flexibility.
You said that you tried the example and it doesn't work on your server, but I doubt it's because of the JATOS version. That study has been around for a while and even if it is something about version incompatibility, it should be easy to fix.
So, first question:
If you download, import, and run the example "Randomize tasks between workers" (study), do you get any error messages in the console?
Second, do you know how to combine two studies (one OSWeb, one not) so that each of them forms a single component of a larger JATOS study?
Best
Elisa
Hi Elisa,
Thanks for getting back to me! I don’t know where I got the idea that I could use the subject_nr for randomization, but I’ve used it last year for an actual study and it appeared to work perfectly fine then. I now realize I must’ve been lucky to get a good distribution.
With regard to you first question: I’m getting a 404 error when I click your link, but I think you meant to forward to https://github.com/JATOS/JATOS_examples/raw/master/examples/randomize_tasks_between_workers.jzip?
I've downloaded and imported the JATOS study, but it did not work out of the box. I'll outline the steps I tried getting the sortStartingCondition component to work below. I’m no programmer, so apologies if I miss something obvious.
1. The first problem I encountered was a greyed out ‘Continue’ button. I removed ‘disabled’ from the <button> element in the HTML. That made it clickable again.
2. The next problem was that it throws an error on line 75 (log from the console). See screenshot. There’s also tons of ‘No open batch channel’ errors but I’m not sure how relevant these are.
It seems that the function var conditions = jatos.batchSession.get("conditions") results into “conditions” being set to undefined (I’ve put in a console.log(conditions) and it says undefined in the console). Is there more input required from my side?
Regarding your second question: yes, I’ve worked with multiple component studies before so I’m confident I can stitch them together once I get sortStartingCondition started.
Best,
Tom
Hi Tom,
Very sorry again for the long (and hopefully unusual) delay. These days are really busy. I should be able to reply quicker from now on.
I downloaded JATOS v 3.5.4 (you really should convince your admins to update your server). I downloaded the study (thanks for correcting the link) and for me it did actually work out of the box. I tried on both Chrome and Safari, and didn't get any of the errors, greyed out buttons, or warnings you mentioned. Which browser are you using?
This also makes sense to me: the randomization script is pretty basic, and shouldn't depend on the JATOS version. The batch session, on which this study depends, has also been around for quite a while. Maybe those warnings you get are actually the problem. If you have admin rights, could you check what you see on the JATOS test page: http://[yourdomain]/jatos/test
In particular could you check the status of Open WebSocket?
Best
Elisa
Hi @elisa
No worries!
That is quite strange.. I have tried it in Chrome, Edge and Firefox but the box is grayed out in all of them. Unfortunately I don't have admin rights to our server so that URL doesn't work. I can try to get in touch with the admin and get him to run the command.
In the meanwhile, is there anything else I can try?
Best,
Tom
Hi, you could try a different study that also relies on the batch session. They're labelled with (basic) tags on our example page, but maybe a good one to try is the batch chat https://github.com/JATOS/JATOS_examples/raw/master/examples/study__group__and_batch_session.jzip
Hi @TIJ
i was wondering if you managed to figure this out?