Problems retrieving data
Hello everyone,
I designed an experiment using OS 3.3.5, esported the experiment as a JATOS study with OSweb, and uploaded it on JATOS. I created a general multiple worker link to collect the data, since I don't need a high degree of control at this point. After distributing the link among the participants, I got 20 results. However, out of this 20 results, 3 have the state "STARTED", 3 "FAIL", 8 "DATA_RETRIEVED" and 3 "FINISHED".
Now, I'm new to all this and I sort of understand what the states mean (although I would appreciate any detailed explanation you can provide, as the one online is rather basic), but I'd like to know what I could do to prevent this from happening. I talked to some of the participants (12 of them) and all of them finished the experiment, so I don't understand why only 3 results have the "FINISHED" state. Could this have something to do with the browser?. Really, I would appreciate any help or advice to solve this.
Comments
Hi!
It seems like something is going wrong with your study: the only state that indicates a successful study run is "FINISHED". You've probably seen the state descriptions already: http://www.jatos.org/Manage-Results.html#table-columns. Sometimes "DATA_RETRIEVED" or "RESULTDATA_POSTED" (in component results) is also fine, if the result data were successful transferred and only the finishing somehow didn't happen. You should look closely into the result data of those study runs with "DATA_RETRIEVED" if they contain everything you expect. But "STARTED" and "FAIL" definitely suggest a problem.
"STARTED" is a state you rarely see. It's the first state in the life cycle of a study run. But since the first action of a study run is to retrieve initialization data it (should) immediately switch to "DATA_RETRIEVED". What happened in your study runs that are "STARTED" I can't really say. Somehow the links where clicked (the study run "STARTED") but then no code was executed in the browser.
State "FAIL" can have many reasons. E.g. one possibility is that someone reloaded the page although reloading is not permitted in this component. I'd suggest you look into JATOS' log (or ask your admin) for more information. Look for 'exception', 'error', or 'fail'.
You wrote that your talked to 12 of your 20 participant and they all said to have finished the study. That's strange, even if I add all 8 "DATA_RETRIEVED" to the 3 "FINISHED" it's only 11 participants. I'm not sure what to make of this: maybe it's not clear for your participants when your study is finished? I can't say anything without seeing the study.
And you wrote that you are using OS 3.3.6: is this the version of JATOS or OpenSesame? If JATOS, I'd recommend to update to a the current version. There are some improvements that might help you.
Best,
Kristian
Hi, Kristian!
How would you like me to share the study with you so you can see it? On the other hand, the OS 3.3.6 is the version of OpenSesame, the version of JATOS I am using is 3.5.11. Besides, you mention that I could look into the JATOS' log or talk to the admin, so could you tell me how I can do that? As I mentioned, all this is quite new to me.
Thanks for your help and comments.
Regards,
Alberto
Hi Alberto,
Ok, in your first message it wasn't clear but it sounds like you are using JATOS on the Mindprobe server. That changes things: Kristian is the official admin but sadly we don't have the capacity to look at individual cases. (If this is not the case, and you are using JATOS on e.g. your University's server, find the admin there).
Essentially, it would be important that you
Hope this helps
Elisa
Hi Elisa,
Yes, I am using the Mindprobe server. I actually thought the same thing some hours ago. I may have taken for granted the fact that participants would wait until the "this is the end" message. So, I was thinking about including more instructions and sending the link once more (although this is not the ideal solution).
I don't know how to check if the data is complete. What I have been doing to see if there is more data than the FINISHED cases is to download the results. When I do this, I only get the results of those FINISHED runs. I would really appreciate any guidance on any other method to check if data is complete in one of those DATA_RETRIEVED cases.
Regards,
Alberto
Hi alberto
That’s weird: you can’t download any of the result data that are not finished? Are you sure this is the case?
I have the feeling you might not have seen a couple of functions in the result table:
As I said I would be surprised if you can't download the other results. The only option I can think of is that those are completely empty entries (i.e. there's a row in the result table but no data have been sent back to the server, in which case I would think that participants didn't do any trials (because OSWeb typically sends data back after every trial)
Best
Elisa
Hi Elisa,
I just followed the procedure you mentioned and out of ten cases with the DATA_RETRIEVED status, only two have some data (see image 1) and the rest have nothing (as you can see in image 2). One thing that surprises me is that one of those empty results correspond to a participation that lasted 7 minutes.
I certainly do not doubt that some of the participants only opened the link and that was it or that some of them did something they shouldn't have done. However, I still don't now if this could be due to a problem of the server, a problem with the OS version I have or a mistake that I committed.
In any case, I am going to upload a new version of the experiment with very clear instructions related to some actions I believe may be interfering with the data-collection process. I will send the link to the new experiments tomorrow and I'll let you know if those instructions worked.
Regards,
Alberto
Hi Alberto,
Image 2 is odd actually. You do have data, but very weird. I don't see in which case you would have gotten just an opening (or even weirder) just a closing bracket.
That actually sounds like a client side (OSWeb/JS) issue. I'll tag @lvanderlinden in case she can give us a hint of what might have happened here.
Hi Elisa,
Thanks for all your support and help. Let's see what she can tell us to solve this problems.