File Sizes Too Large

jpbodet

Hi all,

I've got a bit of a conundrum here -- I'm working with sound recordings (actually, a lot of sound recordings -- around 600 total) that equal ~200mb worth of data once all collected, not to mention the multiple .csv file outputs that accompany them. All told, I'll need 300mb of space per study run to be sure all the data is collected. The max, though, is 50mb.

As far as I can tell, I have two options: install MySQL to handle the data, or adjust the environment variables when installing the docker. My JATOS installation is in a Docker on a DigitalOcean server, so the latter seems simpler.

To adjust the environment variables, I'd need to know how to adjust [jatos.resultUploads.limitPerStudyRun] when I install the Docker on a new server. Since my Docker is on a secured server, I'm not sure where I'd throw the -e command into this (the User Data field provided on the JATOS website):

#!/bin/bash

DOMAIN_NAME="my.domain.name"
EMAIL="my.email@foo.com"

curl https://raw.githubusercontent.com/JATOS/JATOS/master/deploy/docker-compose.yaml > /root/docker-compose.yaml
curl https://raw.githubusercontent.com/JATOS/JATOS/master/deploy/traefik.toml > /root/traefik.toml

sed -i "s/<DOMAIN_NAME>/${DOMAIN_NAME}/g" /root/docker-compose.yaml
sed -i "s/<DOMAIN_NAME>/${DOMAIN_NAME}/g" /root/traefik.toml
sed -i "s/<EMAIL>/${EMAIL}/g" /root/traefik.toml

touch /root/acme.json
chmod 600 /root/acme.json
docker network create proxy
docker-compose -f /root/docker-compose.yaml up -d

nor do I know exactly what the command would look like. The example is

JATOS_RESULT_UPLOADS_PATH="PATH"

so I figure it might be

-e JATOS_RESULT_UPLOADS_LIMITPERSTUDYRUN=300

But I can't find the exact wording online anywhere.

If changing the environmental variables isn't an option, I'd really appreciate some help setting up the MySQL server! The instructions I've found in the jsPsych literature (link) says it needs to be done in the same server as the experiment is running, but with JATOS sitting in a docker in there, I don't want to fiddle around too much and mess everything up.

Please and thank you in advance for your help!

kri

Hi!

You are correct, the size for file uploads is limited in JATOS. By default those values are 30 MB for one file and 50 MB for all files in one study run.

# Max file size
jatos.resultUploads.maxFileSize = 30MB
# Limit of all files per study run
jatos.resultUploads.limitPerStudyRun = 50MB

But the only option you have to increase the max file size is changing JATOS configuration - setting up a MySQL database won't help. All uploaded result files are stored in your server's file system directly and not in the database.

Unfortunately there is no environment variable for those fields (yet). So you have to change them directly in your JATOS' production.conf. Since you run JATOS in a docker it's a bit more difficult and there are several ways: I would open a shell in your running JATOS container, e.g. with

docker exec -it my-jatos-docker-container-id /bin/bash 

and then edit the production.conf which is located in your JATOS folder in the 'conf' folder: change those fields jatos.resultUploads.maxFileSize and jatos.resultUploads.limitPerStudyRun. And don't forget to restart your JATOS docker container afterwards.

Best,

Kristian

kri

I had to edit my previous answer: both config fields have the prefix jatos: jatos.resultUploads.maxFileSize and jatos.resultUploads.limitPerStudyRun

jpbodet

Thanks so much, Kristian! I've been trying to access production.conf in the 'conf' folder this morning and can't seem to get to it. Using the command line code you provided (replacing with my container ID) in the server's console returned this error:

In case the picture is too small, it says:

OCI runtime exec failed: exec failed: container_linux.go:345: starting container process caused "exec: \"bin/bash\": stat /bin/bash: no such file or directory": unknown

I've looked into how to access bin/bash in the DigitalOcean and Docker documentation and played around with other approaches to no avail. I'm not sure how to get into the JATOS' files within the docker to edit production.conf, so I'm not sure where to start. My inexperience with server management is shining; I would appreciate any help you might be able to offer on this!!

Thank you!

kri

Hi!

Just tried it with cortex.jatos.org which is also on DigitalOcean and it worked:

root@docker-s-1vcpu-1gb-fra1-01:~# docker ps
CONTAINER ID        IMAGE                COMMAND               CREATED             STATUS              PORTS                                      NAMES
75bbf28836c4        jatos/jatos:cortex   "./loader.sh start"   9 hours ago         Up 9 hours          9000/tcp, 9443/tcp                         jatos
48e4152eca16        traefik:latest       "/traefik --docker"   9 hours ago         Up 9 hours          0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp   traefik
root@docker-s-1vcpu-1gb-fra1-01:~# docker exec -it 75bbf28836c4 /bin/bash
daemon@75bbf28836c4:/opt/docker$ vim conf/production.conf 

I can't see why it is not working with you. The error you got points out that the file /bin/bash does not exist in the JATOS docker. But that is not possible since I set up this Docker image and just tried it out. Are you sure you used the correct Docker container ID and not an Docker image ID? And from the correct container?

Best

Kristian

jpbodet

Kristian,

That's it! I'm sorry -- I thought the "./loader.sh start" Container was for my local JATOS setup. I was using the wrong Container ID.

For anyone else who has this need, once I got in using the code Kristian provided above in DigitalOcean's console for my server, I entered the file with

vim conf/production.conf

Edited it by pressing i and saved my changes by ESC from edit mode and pressing :x . For some reason, :wq didn't let me save. I'm about to test the experiment and make sure it runs smoothly now. Thank you SO MUCH for your help!