Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

Studies disappeared from the menu list after updating JATOS from to 3.5.11 to 3.7.3

Hi, we are having a troube updating JATOS from 3.5.11 to 3.7.3.

After update, the studies imported in the last 6 months or so have disappeared from the list (left side menu after login).

Running the studies from URL ends up an error ("A problem occurred:Unknown study or batch").

However, the folders and files of the studies seem still there, since when I try to import one of the study, it shows "Study import - but study assets' directory already exists."

Does anyone know how to solve this problem?


----------------------

ServerOS: CentOS Linux release 7.9.2009 (Core)

Apache: 2.4.6-97

PHP: 5.5.38-12

PEAR: 1.10.10-7

MariaDB: 5.5.68-1

Comments

  • Hi,

    I see you are using MariaDB. They said in the past MariaDB is a replacement for MySQL but since then both DBs diverted from each other (https://mariadb.com/kb/en/mariadb-vs-mysql-compatibility/). JATOS only supports MySQL (and H2). Did your JATOS work before the upgrade with MariaDB? What's in the logs (application.log, loader.log and update.log) - any errors there?

    Best,

    Kristian

  • Kristian

    Thank you.

    Yes, I use ony lab.js on this JATOS server, but it was working fine with MariaDB until this update (last experiment was run on April 18).

    I will let the engineer know that "JATOS only supports MySQL."

  • Kristian

    If we are to install the latest version of JAOTS with MySQL, on the same server that we have an old version of JATOS with MariaDB, do you see any problem?

    I want to reserve the old version since there are more than 80 studies still we can access and run on the server, and moving the all studies to the new envionment takes time, and on the other hand, we need a stable (and will be stable) environment soon.

  • Kristian

    I'm sorry that I had mislead you. My engineer told me that our JATOS has been running with H2 (out-of-box) , and not on MariaDB. We use MariaDB for Moodle.

  • edited April 2022

    Kristian

     > What's in the logs (application.log, loader.log and update.log) - any errors there?

    Could you tell me how to get those logs on JATOS Version 3.5.11? I see you can see those logs in the adminstration menu on  v3.7.4. But I cannot find a way to see the logs in Version 3.5.1.

    I tried http://your-jatos-server/jatos/log, but the result was Not Found: The requested URL /jatos/log was not found on this server.

  • Hi,

    JATOS logs are in the JATOS installation folder under logs/.

    Best,

    Kristian

  • Kristian

    Thanks! I got the log files and found that some studies had been missing before updating our web-server.

    This is a log before updating the server, when #137 study was running successfully.

    2022-04-25 11:36:27,254 [INFO] - publix_access - GET /publix/137/start?batchId=140&personalMultipleWorkerId=1323

    And this is a log obtained just now from the VPS server restored from the backup before web-server update.

    2022-05-02 11:39:03,700 [INFO] - publix_access - GET /publix/137/start?batchId=140&personalMultipleWorkerId=1323

    2022-05-02 11:39:03,844 [INFO] - c.p.w.PersonalMultiplePublix - .startStudy: studyId 137, batchId 140, workerId 1323

    2022-05-02 11:39:03,879 [INFO] - g.ErrorHandler - PublixException during call /publix/137/start?batchId=140&personalMultipleWorkerId=1323: An study with ID 137 doesn't exist.

    I thought this issue was related with the web-server update, because those studies were missing from the menu after the update, but I was wrong. The link between the JATOS system and each study was broken sometime between 2022-04-25 and 2022-04-29.

    Can you think of anything that could cause this sort of trouble? Coud you tell me what information I should look for in the logs?

  • So, to summarize, some of your studies vanished from your JATOS' H2 database but are still present in the study assets folder? That is strange. When JATOS deletes a study it always deletes both, in database and study assets. I can't really imagine what might have happened there. The logs you sent also just state the obvious: first a study with ID 137 exists and then not anymore. You sure nobody deleted it, maybe a colleague of yours with access to the study? But then why are the study assets still there. Do you have any other errors or exceptions in the application.log during that time? Any other problems on your server?

  • Yes, you  summarized correctly.

    I'm sure no one, including me, had deleted the studies.

    There were a total of 137 studies I had uploaded. 54 of them have gone missing. They were the most recent uploads (the studies uploaded after around Nov. 2021).

    The attacted is the application log file from 2022-04-25 to 2022-05-01. There seems to be lot of access by robots(?), but I can not find any security breach or obvious "delete" operation.


  • I had a look at your log and couldn't find something that helps us finding the cause of your missing studies:

    On the 2022-04-29 short after 20:00 you did a restart of the server

    There was a error about a missing study: 2022-04-29 20:06:16,525 [INFO] - g.ErrorHandler - JatosGuiException during call /jatos/84: An study with ID 84 doesn't exist.

    And a weird error about your MAC address that has most likely nothing to do with your problem: 2022-04-29 20:06:00,942 [WARN] - p.s.a.i.n.u.i.MacAddressUtil - Failed to find a usable hardware address from the network interfaces; using random bytes: 3f:dd:30:80:21:91:25:ab

    Apart from this no deletion of studies, components, batches or users (or much activity at all). And no database problems or any other problem.

    I have a silly question: is it possible that your missing studies are still there but somehow belong to a different user? You know, you can have a look in /jatos/studyAdmin and it shows you all studies, including the ones your are not member of (but you need Admin rights to access this page).

  • Thank you very much for looking into the log file.

    On the 2022-04-29 short after 20:00 you did a restart of the server

    Yes, this is the restart after we found this problem on the updated server and restore the backup image saved just before the server update.

    I have a silly question: is it possible that your missing studies are still there but somehow belong to a different user?

    https://XXX.XXX.XXX.XXX/jatos/studyAdmin (XXX parts IP address as raw numbers) returns "bad request." And our Admin menu does not show which users own which studies. So, I logged in as each of the two users besides me (there are only 3 users including me), but there was no mis-connection of missing studies to these other users.


  • Only newer versions have this studies admin page. But you found your own way, fine, no other user has your studies. I'm getting out of ideas.

    Do you have older log files from before 25th? If the studies didn't disappear, according to the log, after 25th, maybe before we can find something. But even then it would just explain what happened and not give you back the studies. If you don't have older backups I don't think there is much hope.

  • edited May 2022

    Do you have older log files from before 25th? If the studies didn't disappear, according to the log, after 25th, maybe before we can find something.

    The attached is the log files between April 4th and 30th.

     If you don't have older backups I don't think there is much hope.

    I have all the studies and data backed up locally. Now with the JATOS updated, I don't need to rescure missing studies and data on the old server.

    But I wish to know, if there is a way to find out, what happened and why, just to prevent it happens again.

    I'm running three studies on JATOS server as practicum in a psychology lab course this semester,I want to avoid the worst case senario that students could not complete the experiments during the class time.

    So, any information about precautions, things I should do beforehand, would be greatly appreciated.


  • Do you have older log files from before 25th? If the studies didn't disappear, according to the log, after 25th, maybe before we can find something.

    The attached is the log files between April 4th and 30th.

     If you don't have older backups I don't think there is much hope.

    I have all the studies and data backed up locally. Now with the JATOS updated, I don't need to rescure missing studies and data on the old server.

    But I wish to know, if there is a way to find out, what happened and why, just to prevent it happens again.

    I'm running three studies on JATOS server as practicum in a psychology lab course this semester,I want to avoid the worst case senario that students could not complete the experiments during the class time.

    So, any information about precautions, things I should do beforehand, would be greatly appreciated.

  • But I wish to know, if there is a way to find out, what happened and why, just to prevent it happens again.

    This is exactly why I'm looking closing at this issue. I don't want JATOS loosing studies randomly.

    I had a look at your logs from April 4th to April 30th and could not find anything they could explain the missing studies. Actually everything looks rather normal:

    • A lot of failed authentication requests by bots - normal for an internet-facing server
    • A lot of bots failed in exploiting known vulnerabilities in common server applications - again normal for an internet-facing server
    • Some scrapers/bots/spiders trying to access robot.txt or sitemap.xml - normal

    But most importantly there is not a single DELETE request. Every deleted study, batch or user would leave a line with 'DELETE' in the log. But there is not a single of those. There is now way around it (okay, a malicious person could edit the logs, if they have server access - but why would they?).

    Leaves the possibility that somehow the H2 database or the way JATOS deals with the H2 database had a failure that somehow lead to some (but not all) of your studies being deleted. But this would be the first failure of this kind in JATOS that I have heard of.

    Another idea that just came to my mind. Again a stupid question: Is it possible that you or someone with access to the JATOS server deleted/renamed/moved the database folder that is in the JATOS installation folder. In this folder H2 stores its files and when you somehow delete/rename/move it H2 will just start with a brand new one. This would mean that all studies, users and results up to this point are gone. Maybe you added studies afterwards and the now missing one are from before? Just an idea.

  • Thank you very much for looking into the details of the log files.

    Leaves the possibility that somehow the H2 database or the way JATOS deals with the H2 database had a failure that somehow lead to some (but not all) of your studies being deleted.

    Just to make sure, the study files are still in /study_assets_root direcotry. I don't know how to check if they are intact, but it seems there are enough files for each study.

    Another idea that just came to my mind. Again a stupid question: Is it possible that you or someone with access to the JATOS server deleted/renamed/moved the database folder that is in the JATOS installation folder.

    The database folder is named "database" and its time stamp is 2020/05/12.

    This would mean that all studies, users and results up to this point are gone. Maybe you added studies afterwards and the now missing one are from before? Just an idea.

    Missing studies are the ones that were uploaded after the ones that are not missing.


    BTW, I run a study with more than 30 students at once and did ti twice. I went successfully, and all the studies are not missing in the new server and on the latest JATOS.


    This is exactly why I'm looking closing at this issue. I don't want JATOS loosing studies randomly.

    Would you like to log in the server and see if you can find any thing by yourself? I'm happy to give you an account to investigate this issue, if that may help you in any way to improve JATOS.

  • Just to make sure, the study files are still in /study_assets_root direcotry. I don't know how to check if they are intact, but it seems there are enough files for each study.

    If the folder of the study exists it should be fine.

    The database folder is named "database" and its time stamp is 2020/05/12.

    That still leaves the possible that somehow database was deleted/renamed/moved. H2 would just re-create the folder and start with an empty database.

    Missing studies are the ones that were uploaded after the ones that are not missing.

    Okay, that pretty much rules out the deleted/renamed/moved idea. Unless someone did a backup of the database folder and later on moved back into JATOS.

    BTW, I run a study with more than 30 students at once and did ti twice. I went successfully, and all the studies are not missing in the new server and on the latest JATOS.

    That is reassuring.

    Would you like to log in the server and see if you can find any thing by yourself? I'm happy to give you an account to investigate this issue, if that may help you in any way to improve JATOS.

    Actually yes, I'd like to have a glimpse into the server to rule out that JATOS or H2 somehow misbehaves. Let's move this exchange to email to have it more private: lange.kristian@gmail.com.

  • Thanks! I'll email to you.

  • Kristian

    Have you received my e-mail, sent on May 11th? Just checking.

  • Hi simamune, I just answered.

  • Thank you very much for your full support!

Sign In or Register to comment.

agen judi bola , sportbook, casino, togel, number game, singapore, tangkas, basket, slot, poker, dominoqq, agen bola. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 50.000 ,- bonus cashback hingga 10% , diskon togel hingga 66% bisa bermain di android dan IOS kapanpun dan dimana pun. poker , bandarq , aduq, domino qq , dominobet. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 10.000 ,- bonus turnover 0.5% dan bonus referral 20%. Bonus - bonus yang dihadirkan bisa terbilang cukup tinggi dan memuaskan, anda hanya perlu memasang pada situs yang memberikan bursa pasaran terbaik yaitu http://45.77.173.118/ Bola168. Situs penyedia segala jenis permainan poker online kini semakin banyak ditemukan di Internet, salah satunya TahunQQ merupakan situs Agen Judi Domino66 Dan BandarQ Terpercaya yang mampu memberikan banyak provit bagi bettornya. Permainan Yang Di Sediakan Dewi365 Juga sangat banyak Dan menarik dan Peluang untuk memenangkan Taruhan Judi online ini juga sangat mudah . Mainkan Segera Taruhan Sportbook anda bersama Agen Judi Bola Bersama Dewi365 Kemenangan Anda Berapa pun akan Terbayarkan. Tersedia 9 macam permainan seru yang bisa kamu mainkan hanya di dalam 1 ID saja. Permainan seru yang tersedia seperti Poker, Domino QQ Dan juga BandarQ Online. Semuanya tersedia lengkap hanya di ABGQQ. Situs ABGQQ sangat mudah dimenangkan, kamu juga akan mendapatkan mega bonus dan setiap pemain berhak mendapatkan cashback mingguan. ABGQQ juga telah diakui sebagai Bandar Domino Online yang menjamin sistem FAIR PLAY disetiap permainan yang bisa dimainkan dengan deposit minimal hanya Rp.25.000. DEWI365 adalah Bandar Judi Bola Terpercaya & resmi dan terpercaya di indonesia. Situs judi bola ini menyediakan fasilitas bagi anda untuk dapat bermain memainkan permainan judi bola. Didalam situs ini memiliki berbagai permainan taruhan bola terlengkap seperti Sbobet, yang membuat DEWI365 menjadi situs judi bola terbaik dan terpercaya di Indonesia. Tentunya sebagai situs yang bertugas sebagai Bandar Poker Online pastinya akan berusaha untuk menjaga semua informasi dan keamanan yang terdapat di POKERQQ13. Kotakqq adalah situs Judi Poker Online Terpercayayang menyediakan 9 jenis permainan sakong online, dominoqq, domino99, bandarq, bandar ceme, aduq, poker online, bandar poker, balak66, perang baccarat, dan capsa susun. Dengan minimal deposit withdraw 15.000 Anda sudah bisa memainkan semua permaina pkv games di situs kami. Jackpot besar,Win rate tinggi, Fair play, PKV Games