Combine results from two JATOS installs
Hello,
I have JATOS running smoothly on a remote server. I did an "on-site" test with my tasks at a community center using several iPads. I used the community center's guest wifi account. It was SUPER slow. My thought is to set up a LAN. Then I can go "on-site" using my laptop as the JATOS server and connect a series of iPads using the router/LAN. I have not tested this yet.
My question is, if I have two installs of JATOS, how can I combine the results? After running "on-site" testing, I want to push all the results to the main server-based JATOS install. I am using MySQL on the laptop and the server.
My first thought would be that this would require a bunch of MySQL coding. Any other ideas?
Thank you,
Jason
Comments
Hi Jason!
Sorry for answering so late. I strongly advise against changing JATOS' MySQL via the command line. It's not easy to do and it's likely you break the database.
But having JATOS installed on a laptop and do your field experiment this way is feasible.
My spontaneous idea is to buy a mobile wifi router and use this wifi instead of the slow one from the community center. Then you can use your institute's JATOS server.
Another idea is to combine the result data afterwards, not on the JATOS server, but after exporting from JATOS by using some script language. You'd might need some kind of identifier to align studies and participants.
Hope this helps,
Kristian
Hello Kristian,
Thank you for the reply.
These are great ideas. I never thought of the mobile wifi router. And you are absolutely correct that combining exported results after the fact is the most straightforward idea.
That being said, I, of course, like the approach you strongly advise me against! ;) Your warnings are noted.
My initial work demonstrates that, yes, it is not easy. I do have a production server and a development server running. I will work on the development server so I don't break my production server, and I have nightly backups of the SQL dB.
I am more than willing to share anything I make with you if you are interested.
Jason
Hi Jason,
That being said, I, of course, like the approach you strongly advise me against! ;) Your warnings are noted.
It's possible, I guess. I've never done it myself. Just a sketch: You have to export/impost the rows of the tables ComponentResult and StudyResult. And you have take care of the IDs: the ComponentResult.id and StudyResult.id always have to be unique, id you have to change the ids, you also have to change ComponentResult.studyResult_id. You might have to change StudyResult.study_id, StudyResult.batch_id, StudyResult.worker_id, and ComponentResult.componentId too and let them point to the proper objects in your database.
I still think exporting/importing the data directly on the database is not worth the effort and I still recommend doing it afterwards, on the exported result data, in some script. But on the other side why not.
Best,
Kristian
Hello Kristian,
I got it all to work. You were correct; it was NOT easy. Some interesting problems arose when moving ComponentResult.data. Since I am using jspsych, all stimuli are sent to this JSON data. When my stimuli have an apostrophe in them ( ' , like in a lot of French sentences ) there were issues when moving the data. I added a check to replace all apostrophes with double apostrophes ( ' ' ). That solved that problem. Then I had some text stimuli with HTML embedded like <p class="Instructions">Here is some text</p>. I have no idea why, but this also caused problems. I made changes in my jspsych code to only have HTML flags in the CSS files.
My code has passed initial tests, so who knows, additional errors may rear their heads later on.
Thank you,
Jason
Hi Jason,
I'm a little bit confused how you are doing the data transfer between the databases. Are you using some shell script with SQL commands? I can't really say what might have caused the issues with apostrophes and HTML tags without knowing more about your setup.
Best,
Kristian
I have done everything within python. I have one script that performs an SQL call to the LOCAL dB and saves the results as a csv file. This SQL call combines data from StudyResult, ComponentResult, Batch, and Study tables into one select call.
I then have another script that reads the csv file and splits all the pieces into separate dataFrames. Then I put StudyResults into the REMOTE dB and then all the ComponentResults. There are various checks to make sure that the StudyResult is not already in the dB. I do my checks but I also see that you have a check on the uuid that does not allow duplicate entries. For the ComponentResult I check to make sure no other data for that StudyResult has the same start and end times.
If you are interested, I can share the code with you.
Thanks,
Jason
Seems like you have everything covered. I have nothing to add then be careful with the IDs (but you are already).
Best,
Kristian