Internal JATOS Error - Entity Stream Truncation
in JATOS
Hi there,
I've been running multiple lab.js studies on a JATOS server and am constantly running into an Internal JATOS Error - Entity Stream Truncation due to which there is a data loss for multiple participants.
At first, I tried running just a single study with just 1 participant from Prolific and it worked perfectly. Next, I tried it with 2 participants and that's when things went haywire. The server was able to record complete data for 1 participant but the second participant's data was partially missing.
Every time there is a similar data loss, I find this said error in the logs and would appreciate any help at this point.
Thanks,
Sid
Comments
Hi Sid,
This Entity Stream Truncation error is an infamous one indicating that something interrupted the arriving data stream, likely the sending of result data that are partially missing. The causes for this can be manifold, e.g. network problems, participant's computer problems, JATOS server low in resources (memory, CPU). It's difficult to say what causes without having more information about the experiment or your JATOS server.
Best,
Kristian
Hi Kristian,
Thank you for that information. I’ve asked the server team to pass on the details about the Server Resources. In the meantime I’m uploading the log file if that’d help finding the root cause.
It's specifically happened on more multi-participant occurrences than a single participant using it. Also, the designed study posts the data after each input the user provides, could that be a potential issue with multiple participants?
Hi Sid!
It's specifically happened on more multi-participant occurrences than a single participant using it.
Do you mean that multiple participants did the study at the same time in parallel? That should be no problem given that your JATOS server has enough resources.
Also, the designed study posts the data after each input the user provides, could that be a potential issue with multiple participants?
Maybe. I saw that you sent result data around every 3 seconds (with jatos.submitResultData). That is a lot. How big are your result data? If you send them that often it's likely you overload a server that is low in resources. I'd strongly recommend to send your result data less often. I personally have the rule of thumb once a minute, for short studies maybe more often. But if you definitely want to send the result data that often you should use jatos.appendResultData instead of jatos.submitResultData and deactivate MySQL's binary log on your server (if you use MySQL). And spread out your participants, meaning that they should not run your study all at the same time but more one-after-another.
Best,
Kristian
Hi Kristian,
Thanks for the info. Yes, we had multiple parallel participants running the study at the same time. I also got the info about our Server Resources and we have 2GB RAM with 2 - 2.7GHz CPUs which, I'm assuming, is enough for JATOS.
Today, I tested a study by incrementing the # of participants by 1 every 5-10mins and it worked well with 3 participants' data missing partially so, it does seem the better way to go.
I'll try running a study with less frequent data transfer and check. However, when we ran studies before making any edits to the study file exported from Lab.js we still ran into a similar issue with more than 50% of the participants' data not received by the server due to the said error.
Best,
Sid
Hi Sid!
2 GB memory can be easily too low when your study is demanding and multiple participants run it in parallel. You can upgrade to 4 GB - or, as you already said, ensure that only one participant runs it at the same time.
I'll try running a study with less frequent data transfer and check.
good idea
However, when we ran studies before making any edits to the study file exported from Lab.js we still ran into a similar issue with more than 50% of the participants' data not received by the server due to the said error.
I'm at a loss here: what edits do you make to lab.js' exported study file? I'm not really familiar in how lab.js send's its data back to JATOS. Maybe there is room for improvement.
Best,
Kristian