End of experiment issues: the page reload on Safari/Mac
Hi all,
I built two experiments with OSWeb 1.3.5 hosted on a server with JATOS 3.5.1. I pre-tested the experiments myself on different browsers (except Safari) and it worked fine. I now have students piloting the experiments and I had a lot of issues.
First, I have to mention that, in order to avoid experiencing data loss during the experiment (it happened some trials were missing during pre-testing phases), I use jatos.submitResultData instead of jatos.appendResultData (https://github.com/kristian-lange/opensesame-extension-osweb/blob/345b528b6cad40a6bade0b48bb9ffc6f36702add/osweb/src/js/jatos.js).
So far, I collected overall 50 results on JATOS. Nearly half are participants who tried to start the experiments but their page got stuck on OSWeb loading (a survey pointed out that those participants did not update their browser so it is not an issue). I only had 4 participants who finished the experiment without any problem. However, they said that it took a long time to get redirected to the "endPage" (~30 seconds). I had the same during my own pre-tests but I thought it was because I tested the experiment on four browsers at the same time and that a lot of windows were open. Finally, I had around 15 participants who seemed to end the experiment (they apparently got all the trials) but their page reloaded automatically due to an error before showing the endPage.html file. The automatic reload then led to the experiment to fail. I surveyed the students and it looks like when that problem occurred it was always when using Safari and/or MacOS, and those who finished the experiment without any problem have a system operating under Windows and used Chrome or Firefox.
Also, as the size of the data file is around 2MB, maybe it takes a lot of time to send the data at once and that, for a reason I don't know, it crashes with Safari/Mac and it works with Windows/Non-Safari browsers (but after some delay).
Finally, I asked some students using Safari to do the task again but with only 2 trials instead of around 150 and it worked.
Any help would be verry appreciated.
Simon
Comments
Hi Simon,
I'm sad to hear that your experiment did work out as planned. Hopefully it will get better and you will find a way to fix this. Let me divide your message and answer one by one.
Nearly half are participants who tried to start the experiments but their page got stuck on OSWeb loading (a survey pointed out that those participants did not update their browser so it is not an issue).
I don't understand. How can an not updated browser lead to a stuck OSWeb loading page?
However, they said that it took a long time to get redirected to the "endPage" (~30 seconds).
30s is too long even for the 2MB of your result data.
Finally, I had around 15 participants who seemed to end the experiment (they apparently got all the trials) but their page reloaded automatically due to an error before showing the endPage.html file. The automatic reload then led to the experiment to fail. I surveyed the students and it looks like when that problem occurred it was always when using Safari and/or MacOS, and those who finished the experiment without any problem have a system operating under Windows and used Chrome or Firefox.
You can allow reloads in the component's properties (in JATOS' GUI).
Also, as the size of the data file is around 2MB, maybe it takes a lot of time to send the data at once and that, for a reason I don't know, it crashes with Safari/Mac and it works with Windows/Non-Safari browsers (but after some delay).
2MB is a lot of data for a single experiment - but not that much for the internet. It should not overwhelm JATOS nor a browser.
Finally, I asked some students using Safari to do the task again but with only 2 trials instead of around 150 and it worked.
There can be many reasons for what you describe but I have a hunch that something is wrong with the result data sending. Is it possible for you to send me your study? I'd like to try it out on my server and have a look at the result data sending?
Best,
Kristian
Hi Kristian,
For participants getting stuck on the loading page, it is a normal behaviour as OSWeb supports only some brower versions (e.g., Safari >= version 10); it is not related to JATOS at all.
For the delay to show the endPage (for the students who experienced no issue), some students reported no specifically long delay (a few seconds) and others up to 1 minute.
When the reload option is enable on JATOS, what appened when the page is reloaded? Does it simply restart where the participant was before the reload? If yes, I would definitely do that.
For the size of the data files, I can set manually the variables to be logged to reduce the size files if it is a solution. Currently, the behaviour of the logger is to log every variables (which is the default behaviour of OpenSesame).
You'll find here attached the .zip file generated by OpenSesame that I uploaded on JATOS.
Thank you for your help!
Best,
Simon
Hi Simon,
I tried your study on my JATOS server and on my local one. I think I know the reason for the delays in the end of your study (I'm not 100% sure though). I think it is a combination of your big result data and the fact that those data are stringified and concatenated after each trial. String operations can be expensive performance-wise. Each trial produces a lot of data, this is then in method onLogHandler stringified and concatenated with the already stored jatosResultData.
I'd propose a slightly different approach: don't stringify each time, but only in the end in the onFinishedHandler. I attached a version of your study with an updated OSWeb-jatos.js.
Another possibility is to just wait for the next JATOS release. I'm working on a background sending queue for the result data that would render those OSWeb issues with result data sending obsolete. I'm expecting to have a pre-release this weekend and a release during the next week.
I hope this helps and that your experiment can continue in a better way ;)
Best,
Kristian
Hi Kristian,
Thank you very much for the file :-)
I tested it and it seems to speed up the process of sending the data a bit, but the biggest speed up improvement was when I manually determined the variables to be logged, reducing the size of the data file to 200Ko instead of 2Mo.
I had a student testing the experiment (with Safari under MacOs) with the 2Mo data file and with stringify only at the end => same problem as before. It was tested again with the 200Ko data file and with stringify only at the end => again, same problem as before.
It is strange that the problem only occurs with Safari/MacOs. With Opera, Firefox, Edge and Chrome (running on Windows) everything works fine and the loading of the endPage takes around 5 seconds with the 200Ko data file and with stringify only at the end.
Do you have another idea why this problem occurs only with Safari/MacOs?
Do you think this could be solved with the next JATOS release or is it more an OSWeb than a JATOS issue ?
Thank you for your help.
Best,
Simon
Hi Simon,
I tested it and it seems to speed up the process of sending the data a bit, but the biggest speed up improvement was when I manually determined the variables to be logged, reducing the size of the data file to 200Ko instead of 2Mo.
Weird. 2MB are a lot for experiment data but not so much for the internet (just the Amazon start page downloads 4MB of data). It's weird that it takes so long. Maybe it's your JATOS server? Where is your JATOS installed? How much CPU and RAM does it have?
Do you have another idea why this problem occurs only with Safari/MacOs?
Not really. I personally don't use Safari - but people I work with do and they never complained about Safari and JATOS. Although they never run a OSWeb study yet (only pure JATOS or jsPsych+JATOS). What version is this Safari? In the OSWeb doc they say that only Safari >= 10 is supported.
Do you think this could be solved with the next JATOS release or is it more an OSWeb than a JATOS issue ?
Maybe. I can't say, since I don't understand the problem. But the new JATOS v3.5.3 has an improved sending of result data in the background. You don't have to send all data in the end with jatos.submitResultData anymore - you can use the normal OSWeb approach with jatos.appendResultData and send it after each trail. You can try out the new version, it's in pre-release and I think I will release tomorrow (test are all okay). You can enforce an update of your current JATOS even to a version that is in pre-release with this URL: your-jatos-domain/jatos?version=v3.5.3 . Or you wait until it's released and do the normal update way.
Best,
Kristian
Hi Kristian,
JATOS is installed on servers at my university (people from the IT dealt with installing it). The server consists of 2 virtual CPUs with 4 Go of RAM and a storage capacity of 30 Go.
For Safari, OSWeb is supposed to support version >= 10. One student using Safari version 10 got stuck on the loading page but the others with more recent version had no issue before the data sending process.
I think I will update JATOS to the version 3.5.3, do a try with students using Safari and let see what happens with the OSWeb jatos.appendResultData procedure.
Again very thank for your help,
Simon
2 CPU and 4 GB of RAM should be enough (of course, it always depends on many factors like the files sizes your study downloads, the result data size your study uploads, how many participants work in parallel etc. - but JATOS with 1GB usually works for small studies - so 4 GB should be okay)
Tell me whether the update to 3.5.3 helped. I'm curious now.
Best
Hi Kristian,
After a test with Safari, everything works fine. The only weird stuff is that one trial has been logged two times, what did not occur when I tested the expeirment with other browsers.
But it is not a real issue for my study.
I think everything is in good shape. Thanks again for your help!
Best,
Simon
Nice!
(I have no idea why the trail data are logged twice. Weird.)
Good luck with your experiments,
Kristian