Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

"Cannot read result data." after long "Working"

Hi everyone,

We have about 1500 results and we can't inspect them through the JATOS administrator dashboard.

We have a rather powerful virtual machine and a good internet connection, and we didn't have this issue with a smaller amount of results.

Anyone having a suggestion on how to download the results?


Comments

  • Hi Kian,

    I've seen this before. I guess it has to do with your MySQL database setup. By default it limits the amount of data that can be loaded at once. JATOS' docs has a chapter about it:

    Basically you (or your admin) have to adjust the max_allowed_packet to a higher value in your mysql.cnf. E.g. to allow 512MB use:

    max_allowed_packet=512MB
    

    I also try to find a solution for this problem in JATOS but this will take a couple of days.

    Best,

    Kristian

  • @kri

    Thank you for the quick reply Kristian.

    We did all the steps mentioned by you (and our MySQL is not older than 8.xx) but didn't work.

    We ended up downloading the results directly from the database.

    Now we are experiencing the same issue with 800 responses.

    We will go through the documentation again to check if we missed something while setting up the server and the database, but wanted to reply here in case you have a solution for it.

    Thanks again for your time.

  • I'm sorry to hear you had to download your data directly from the database. This is annoying and JATOS is intended to make this unnecessary. May I ask how big your result dataset is expected to be altogether? I'm asking because there is an additional issue in MySQL that it by default loads all data into memory at once. If you have very large datasets then even a setting of max_allowed_packet won't help because it has a maximum of 1GB.

    It's planned for JATOS to change the way it queries MySQL so it does not load all data at once into memory. But I can't say yet when this will be released.

    Best,

    Kristian

  • @kri

    Hi Kristian,

    For a study that we have with 871 participants, the data size we are getting from the table ComponentResult is only 522 MB (we are only using the ComponentResult because this is where our data is. We know that normally JATOS is also querying some of the other tables as well thus trying to show in the dashboard more than 522MB).

    We see that JATOS has pagination implemented in mind when we try to see the result via the drop-down menu "Show [10, 25, 50, All] entries", but it doesn't look like it is fetching the selected amount from the Database, and then when you go to the next page, the next amount and so on, but it appears that JATOS is trying to SELECT everything from the Database and then trying to visualize it on the admin dashboard.

    Thank you very much for the hints. We hope to see that it is coming in near future.

  • Hi Kian,

    Sorry for the late reply - some holidays came in the way.

    For a study that we have with 871 participants, the data size we are getting from the table ComponentResult is only 522 MB (we are only using the ComponentResult because this is where our data is. We know that normally JATOS is also querying some of the other tables as well thus trying to show in the dashboard more than 522MB).

    522MB is big for a database query, but should be possible with enough memory in the server and the receiving computer. Did you try to set max_allowed_packet to 1GB?

    We see that JATOS has pagination implemented in mind when we try to see the result via the drop-down menu "Show [10, 25, 50, All] entries", but it doesn't look like it is fetching the selected amount from the Database, and then when you go to the next page, the next amount and so on, but it appears that JATOS is trying to SELECT everything from the Database and then trying to visualize it on the admin dashboard.

    You are correct, the pagination setting changes only the number of results shown at once in the UI - but does not change the way JATOS queries the database. JATOS queries all result data at once which is actually a bug in Hibernate (a library used by JATOS; more about the bug). But I'm thinking about fetching the results in a different way which would circumvent the bug. This would also be paginated. Then the results are send via chunked HTTP to the client side and displayed. It's working beautifully in my dev version. But I can't yet say when this will be finished and go into a release.

    Best,

    Kristian

  • Thanks Kristian for the reply.

    Yes, we also tried the max_allowed_packet to be 1GB.

    Great, looking forward to having it as a release :)

    Is there any chance that you could share your dev version with us?


    Thanks again for all the support,

  • Hi Kian,

    The updated result pagination changes are currently mixed in together with other stuff that not yet ready for release. But as soon as I extracted the result pagination changes I'll prepare a release and let you know. But the current Corona lockdown in the country were I live makes it difficult to work for me. So it might take a couple weeks.

    Best,

    Kristian

  • Hi,

    I'm experiencing a similar problem, but with only 63 responses. Doesn't make sense that memory is an issue. Any idea?


    Thanks!

    Yoav

  • Solved by resizing the DigitalOcean droplet.

  • Thank you, Kristian,

    We will happily be waiting for the new release.

    Thanks again for all the support.

    Best regards,

    Kian

  • Hi Guys,

    I have inherited looking a system with version 3.5.8 of Jatos using postgres and I have a user with this issue. I have tried increasing the shared buffers and work mem but that hasn't helped so far. Any tips on what else to try or is there possibly a configuration change required in Jatos itself?

    Thanks,

    Nathan

  • Hi @kri,

    I did manage to get the system working with the buffer increases and increasing the number of connections, not sure which fixed the issue exactly or if it was both. I now have the same user with the problem again, and I am wondering if you can offer any tips or advice as the users number of results is much lower than those mentioned above (around 400).

    Thanks,

    Nathan

  • So by increasing the connections and buffers, plus restarting postgres and jatos, it is working again. Unfortunately I didn't think to just try restarting the services first. The dataset for this study is about 1.8GB, the same user was having an issue with other smaller datasets so I'm wondering if a restart would have fixed that.

  • Stopped working for this user again, would allow me to load results if I restarted Jatos, but then a non-descript error when trying to export all.

  • Hi Nathan,

    First a hint, questions are more visible if you post each new question in its own forum thread. Don't add something to a question in 2024 that is from 2021.

    Second,  3.5.8 is pretty old. I know it can be challenging to update a running system, but you should consider it anyways.

    Third, JATOS isn't compatible with Postgres. You sure you aren't using a MySQL (or MariaDB)? Or the embedded H2 database.

    Fourth, you just write "I have inherited looking a system with version 3.5.8 of Jatos using postgres and I have a user with this issue.". Can you please describe your issue again. It's also my time and going through the whole thread here and what has been written 3 years ago, is quite a drag.

    Best,

    Kristian

Sign In or Register to comment.

agen judi bola , sportbook, casino, togel, number game, singapore, tangkas, basket, slot, poker, dominoqq, agen bola. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 50.000 ,- bonus cashback hingga 10% , diskon togel hingga 66% bisa bermain di android dan IOS kapanpun dan dimana pun. poker , bandarq , aduq, domino qq , dominobet. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 10.000 ,- bonus turnover 0.5% dan bonus referral 20%. Bonus - bonus yang dihadirkan bisa terbilang cukup tinggi dan memuaskan, anda hanya perlu memasang pada situs yang memberikan bursa pasaran terbaik yaitu http://45.77.173.118/ Bola168. Situs penyedia segala jenis permainan poker online kini semakin banyak ditemukan di Internet, salah satunya TahunQQ merupakan situs Agen Judi Domino66 Dan BandarQ Terpercaya yang mampu memberikan banyak provit bagi bettornya. Permainan Yang Di Sediakan Dewi365 Juga sangat banyak Dan menarik dan Peluang untuk memenangkan Taruhan Judi online ini juga sangat mudah . Mainkan Segera Taruhan Sportbook anda bersama Agen Judi Bola Bersama Dewi365 Kemenangan Anda Berapa pun akan Terbayarkan. Tersedia 9 macam permainan seru yang bisa kamu mainkan hanya di dalam 1 ID saja. Permainan seru yang tersedia seperti Poker, Domino QQ Dan juga BandarQ Online. Semuanya tersedia lengkap hanya di ABGQQ. Situs ABGQQ sangat mudah dimenangkan, kamu juga akan mendapatkan mega bonus dan setiap pemain berhak mendapatkan cashback mingguan. ABGQQ juga telah diakui sebagai Bandar Domino Online yang menjamin sistem FAIR PLAY disetiap permainan yang bisa dimainkan dengan deposit minimal hanya Rp.25.000. DEWI365 adalah Bandar Judi Bola Terpercaya & resmi dan terpercaya di indonesia. Situs judi bola ini menyediakan fasilitas bagi anda untuk dapat bermain memainkan permainan judi bola. Didalam situs ini memiliki berbagai permainan taruhan bola terlengkap seperti Sbobet, yang membuat DEWI365 menjadi situs judi bola terbaik dan terpercaya di Indonesia. Tentunya sebagai situs yang bertugas sebagai Bandar Poker Online pastinya akan berusaha untuk menjaga semua informasi dan keamanan yang terdapat di POKERQQ13. Kotakqq adalah situs Judi Poker Online Terpercayayang menyediakan 9 jenis permainan sakong online, dominoqq, domino99, bandarq, bandar ceme, aduq, poker online, bandar poker, balak66, perang baccarat, dan capsa susun. Dengan minimal deposit withdraw 15.000 Anda sudah bisa memainkan semua permaina pkv games di situs kami. Jackpot besar,Win rate tinggi, Fair play, PKV Games