Jatos, experiment won't reload because of "finished study", data not saved
Hi there,
I'm new to Jatos and I have a problem with an experiment I hope you can help me with. The experiment in question is a remote follow-up to a in-person experiment. After participants are done with the in-person experiment, I send a "general multiple, open directly" link to their e-mails. I picked that type of link out of a mix of habit (the platform I used before, cognition.run, only had the option of a general link for everyone) and laziness (I didn't want to generate and send a different link and code to every participant). However, a couple of issues have emerged which I can't figure out.
The first is that sometimes the experiment won't reload because "the study is finished". Specifically, the message is "It's not allowed to reload this component (ID: 21961). Study (ID: 14167) is finished." Note that the two IDs are always the same (even when using different browsers, computers, and networks). However, the person getting this message hasn't actually finished the study. Also, I don't get why the problem then seems to go away by itself (e.g., if I try again at a later time, the study loads with no issues).
The second problem is that for one of my participants, the data were not saved, i.e., in Jatos' results page, there is no finished study for her. That would not be a problem by itself (I know that is a possibility because of network issues) but a few things seem off: First, I'm positive that the participant completed the study as she screenshotted the end of study page. Therefore, her network must have been able to redirect her to that page, but shouldn't Jatos save the data before doing that? Second, for her as well, I have that weird not-reloading issue in the Jatos results, i.e., her study is marked as FAIL with that "study finished" message. But how could she finish the study if she got that message? Finally, for her (as well as for other participants), I have a number of Result IDs instead of just one, as if she reloaded the page a few times before getting started. Not impossible that she did do that, but I thought I should mention it.
What am I missing? I'm getting the sense that I'd be better off using personal and/or single links rather than general multiple links, however, I'd like to understand what's happening here in case it's something else. I'm using Jatos 3.8.5 and jspsych 7.3.4, i.e., the latest versions. Let me know if you need further information. Thank you in advance for your help.
Best,
Giacomo
Comments
Hi Giacomo,
I took some time to answer your long question :)
Maybe there is a general misunderstanding of how JATOS handles study runs: First you have a study link (e.g. https://example.com/publix/OZTF4BqnlWd), in your case it is of type general multiple and that means you can use this link as often as you want. Then, each time you use this link, JATOS starts a new study run (with a new study result: each line in your result page represents a study result) and redirects the participants to a different type of URL (the are much longer, e.g. https://example.com/publix/4c93ba3f-eda7-45e7-9db0-200e370dc196/15287b49-d9e1-4d20-85c8-a412c9ac2075/start). Often a JATOS study consists of multiple components - then each component has its own study run URL. Those study run URLs are not "general multiple" any more, that means, they have the normal constraints of a study run, like the other study link types (personal single, personal multiple, ...). E.g. if your component is not allowed to reload, a reload in the browser will cause JATOS to end the study with a state FAIL. Or if your study is linear, you cannot loop back to a past component.
In your case you seem to have forbidden the reloading of a component (which is often what you want) and your participant reloaded the page (for whatever reason, e.g. errors in the experiment, network problems). Then the study run got finished with a FAIL. Each consecutive reload just returned an error page from JATOS. Then your participant used the original study link again (a general multiple) and started a new study run with a new study result. That worked.
And to answer some of your questions:
The first is that sometimes the experiment won't reload because "the study is finished". Specifically, the message is "It's not allowed to reload this component (ID: 21961). Study (ID: 14167) is finished."
Your component is not allowed to be reloaded. Once it does, it ends with FAIL.
Note that the two IDs are always the same (even when using different browsers, computers, and networks).
The IDs belong to the general multiple study link.
Also, I don't get why the problem then seems to go away by itself (e.g., if I try again at a later time, the study loads with no issues).
I'm sure the person, at a later time, uses the original general multiple study link again - while before it just pressed reload in the browser.
I'm not sure I actually helped you with your issues, but I hope I could help you understanding it.
Best,
Kristian
Hi Kristian,
thank you very much for your reply. You did help, thank you, and I am now beginning to see what might have happened. All in all, I think it would be safer to use personal, single-use links in order to have a bit more control on what the participant does. There's still the question of why the data for the run the participant did finish were lost even if the participant reached the end of study component. More specifically, I'd like to understand if there's something I can do to moving forward to prevent that from happening again (it's hard to get participants for this study and I wouldn't want to lose too much data). Thanks again for your help!
Best,
Giacomo
There's still the question of why the data for the run the participant did finish were lost even if the participant reached the end of study component. More specifically, I'd like to understand if there's something I can do to moving forward to prevent that from happening again (it's hard to get participants for this study and I wouldn't want to lose too much data).
First, JATOS goes to quite some length to ensure that the sent result data reach the JATOS server. Importantly, it has a background worker (called httpLoop) running in the browser of your participants that handles all sending of result data (and other stuff) to the JATOS server. This httpLoop ensures that the order of the result data is kept (in case you sent data several times: 1 - 2 - 3 stays 1 - 2 - 3), then it automatically retries the sending if, for whatever reason, it failed (it retries up to 5 times). And finally it ensures that the study only ends or goes to the next component when all data are sent. So, it should never happen if you use JATOS (and jatos.js) correctly and there is no Internet issue that a study FINISHED and data get lost. That said, people can use jatos.js' control flow functions or functions to send data/files wrong. Can you post here the part of your code where you send your data and finish your study / go the next component? I can have a look.
Best,
K.