JATOS service no longer working?

ekm

Hello! I found out today that JATOS has been giving a 503 error since last Thursday; I went and started it manually on the server (just using ./loader.sh restart) and it seemed fine, but when I tried to start the service that runs it in the background, I had no success.

Miscellaneous facts:

• I checked on memory, since that may have been a culprit, but it seems ok (MemTotal: 1863068 kb, MemFree: 333808 kB, MemAvailable: 1492248 kB)
• In application.log, this is the only error I could find: date, time, ... [ERROR] - c.g.Home - Couldn't request latest JATOS update info.
• Running the service my predecessor had set up for JATOS goes as follows:

$ sudo systemctl start [jatosnickname].service
$ sudo systemctl status [jatosnickname].service
● [jatosnickname].service - Onlinexp JATOS local Java
   Loaded: loaded (/etc/systemd/system/[jatosnickname].service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2021-03-01 13:06:37 PST; 38min ago
  Process: 43815 ExecStopPost=/bin/rm -f /opt/onlinexp/RUNNING_PID (code=exited, status=0/SUCCESS)
  Process: 43813 ExecStop=/bin/kill $MAINPID (code=exited, status=1/FAILURE)
 Main PID: 43818 (loader.sh)
   CGroup: /system.slice/[jatosnickname].service
           ├─43818 /bin/bash /opt/onlinexp/loader.sh start
           └─43972 sleep infinity

Mar 01 13:06:37 [server] systemd[1]: [jatosnickname].service holdoff time over, scheduling restart.
Mar 01 13:06:37 [server] systemd[1]: Stopped Onlinexp JATOS local Java.
Mar 01 13:06:37 [server] systemd[1]: Started Onlinexp JATOS local Java.
Mar 01 13:06:52 [server] loader.sh[43818]: Starting JATOS... started
Mar 01 13:06:52 [server] loader.sh[43818]: To use JATOS type 127.0.0.1:9000 in your browser's address bar
Mar 01 13:08:24 [server] loader.sh[43818]: JATOS stopped

Is the service the issue? If I restart the whole server, do I risk losing any JATOS data? Thanks!

kri

JATOS service status looks fine.

What does JATOS' log say, are there any errors? It's in JATOS installation directory in logs/application.log.

ekm

The error that shows up is ``c.g.Home - Couldn't request latest JATOS update info.``

If you want the rest of the error log for context, I've attached it here:

application.log

Thanks for your help!

kri

Your log looks fine too.

This 'Couldn't request latest JATOS update info.' means that JATOS couldn't query GitHub for new version of JATOS. Maybe there is some kind of network problem?

And I see that your JATOS binds to port 9000 on localhost. So you probably have a reverse proxy in front, e.g. Apache, Nginx? Maybe this one is broken? Did you check this service?

ekm

Some good news, I tried again just running systemctl stop and then systemctl start for the jatos service, and that did the trick! Something about how restartwas configured must be off, but thankfully it won't be a huge issue going forward. Anyway, the original issue is resolved!

Now that I'm aware of this JATOS update lag, I'm inclined to ask about that too. We're on JATOS 3.5.5 right now, but the latest release is 3.5.8, and it'd be good for the automatic update check to work in future too.

My uni is trying to keep all connections to servers maintained by ITS pretty buttoned down, so the update query is going to need a proxy for the outgoing connection. (just fyi the server is running Red Hat Enterprise Linux release 7.6 (Maipo) on vmware with 1 processor) Right now we have a .bak file under /etc/httpd/conf.d/ relevant to JATOS, that was configured pretty much exactly as recommended here

Do you have advice for where to add the needed proxies for outgoing cxns? Am I even looking in the right place? (this is my first time tinkering with the jatos setup, another tech was the one who put it in place)

kri

Nice, that a restart solved your original problem (good old turning off and back on).

Then about the JATOS automatic update: It needs an outgoing port 80 to connect to GitHub to check if there are new releases of JATOS. But I'm not a network specialist and never worked with ITS. I just say that you (or someone) has to configure the firewall to let port 80 through. I don't think you have to tinker with Apache's config, but I might be mistaken there. Did you check if your server has a local firewall like uwf? And if nothing works: you can always update JATOS manually.

ekm

Turn it off/on is a cliche for a reason.

Thanks for confirming what JATOS needs for that GitHub connection. I'll check with our ITS department to see if that's something their configuration will actually allow, and as you say, just update manually if needed! Cheers :)