Problems retrieving data

albertmrm

Hello everyone,

I designed an experiment using OS 3.3.5, esported the experiment as a JATOS study with OSweb, and uploaded it on JATOS. I created a general multiple worker link to collect the data, since I don't need a high degree of control at this point. After distributing the link among the participants, I got 20 results. However, out of this 20 results, 3 have the state "STARTED", 3 "FAIL", 8 "DATA_RETRIEVED" and 3 "FINISHED".

Now, I'm new to all this and I sort of understand what the states mean (although I would appreciate any detailed explanation you can provide, as the one online is rather basic), but I'd like to know what I could do to prevent this from happening. I talked to some of the participants (12 of them) and all of them finished the experiment, so I don't understand why only 3 results have the "FINISHED" state. Could this have something to do with the browser?. Really, I would appreciate any help or advice to solve this.

kri

Hi!

It seems like something is going wrong with your study: the only state that indicates a successful study run is "FINISHED". You've probably seen the state descriptions already: http://www.jatos.org/Manage-Results.html#table-columns. Sometimes "DATA_RETRIEVED" or "RESULTDATA_POSTED" (in component results) is also fine, if the result data were successful transferred and only the finishing somehow didn't happen. You should look closely into the result data of those study runs with "DATA_RETRIEVED" if they contain everything you expect. But "STARTED" and "FAIL" definitely suggest a problem.

"STARTED" is a state you rarely see. It's the first state in the life cycle of a study run. But since the first action of a study run is to retrieve initialization data it (should) immediately switch to "DATA_RETRIEVED". What happened in your study runs that are "STARTED" I can't really say. Somehow the links where clicked (the study run "STARTED") but then no code was executed in the browser.

State "FAIL" can have many reasons. E.g. one possibility is that someone reloaded the page although reloading is not permitted in this component. I'd suggest you look into JATOS' log (or ask your admin) for more information. Look for 'exception', 'error', or 'fail'.

You wrote that your talked to 12 of your 20 participant and they all said to have finished the study. That's strange, even if I add all 8 "DATA_RETRIEVED" to the 3 "FINISHED" it's only 11 participants. I'm not sure what to make of this: maybe it's not clear for your participants when your study is finished? I can't say anything without seeing the study.

And you wrote that you are using OS 3.3.6: is this the version of JATOS or OpenSesame? If JATOS, I'd recommend to update to a the current version. There are some improvements that might help you.

Best,

Kristian

albertmrm

Hi, Kristian!

How would you like me to share the study with you so you can see it? On the other hand, the OS 3.3.6 is the version of OpenSesame, the version of JATOS I am using is 3.5.11. Besides, you mention that I could look into the JATOS' log or talk to the admin, so could you tell me how I can do that? As I mentioned, all this is quite new to me.

Thanks for your help and comments.

Regards,

Alberto

elisa

Hi Alberto,

Ok, in your first message it wasn't clear but it sounds like you are using JATOS on the Mindprobe server. That changes things: Kristian is the official admin but sadly we don't have the capacity to look at individual cases. (If this is not the case, and you are using JATOS on e.g. your University's server, find the admin there).

Essentially, it would be important that you

• check that your data is indeed incomplete, on those cases where the status is not what you expect. Maybe you actually do have all your result data, and there's no problem.
• Consider whether your participants might be confused and quit the study before it finishes (maybe you are transferring large amounts of data - it sometimes happens with OSWeb studies) - making them think that they finished when they in fact hadn't yet.

Hope this helps

Elisa

albertmrm

Hi Elisa,

Yes, I am using the Mindprobe server. I actually thought the same thing some hours ago. I may have taken for granted the fact that participants would wait until the "this is the end" message. So, I was thinking about including more instructions and sending the link once more (although this is not the ideal solution).

I don't know how to check if the data is complete. What I have been doing to see if there is more data than the FINISHED cases is to download the results. When I do this, I only get the results of those FINISHED runs. I would really appreciate any guidance on any other method to check if data is complete in one of those DATA_RETRIEVED cases.

Regards,

Alberto

elisa

Hi alberto

That’s weird: you can’t download any of the result data that are not finished? Are you sure this is the case?

I have the feeling you might not have seen a couple of functions in the result table:

• You can expand on a result by clicking on each row's leftmost button, which contains a number and a little arrow. That will already show you the first few lines of a result
• You can select each row by clicking on the second-to-leftmost button (it's a white minus by default, turns into a blue checkmark when clicked). Then you can click on 'Export selected' and download those files. You should then be able to open them and check if your data are complete.

As I said I would be surprised if you can't download the other results. The only option I can think of is that those are completely empty entries (i.e. there's a row in the result table but no data have been sent back to the server, in which case I would think that participants didn't do any trials (because OSWeb typically sends data back after every trial)

Best

Elisa

albertmrm

Hi Elisa,

I just followed the procedure you mentioned and out of ten cases with the DATA_RETRIEVED status, only two have some data (see image 1) and the rest have nothing (as you can see in image 2). One thing that surprises me is that one of those empty results correspond to a participation that lasted 7 minutes.

Image 1.JPG

Image 2.JPG

I certainly do not doubt that some of the participants only opened the link and that was it or that some of them did something they shouldn't have done. However, I still don't now if this could be due to a problem of the server, a problem with the OS version I have or a mistake that I committed.

In any case, I am going to upload a new version of the experiment with very clear instructions related to some actions I believe may be interfering with the data-collection process. I will send the link to the new experiments tomorrow and I'll let you know if those instructions worked.

Regards,

Alberto

elisa

Hi Alberto,

Image 2 is odd actually. You do have data, but very weird. I don't see in which case you would have gotten just an opening (or even weirder) just a closing bracket.

That actually sounds like a client side (OSWeb/JS) issue. I'll tag @lvanderlinden in case she can give us a hint of what might have happened here.

albertmrm

Hi Elisa,

Thanks for all your support and help. Let's see what she can tell us to solve this problems.