OpenSesame 3.1 extremely slow to start the first time, and run experiments the first time
Over the last 6 months, we (me and a few colleagues) have used OpenSesame 3.1.1 in one of our Bachelor Psychology courses. In this course, around 400 students learn to build experiments using OpenSesame. About 2/3 of the students work on our machines (Windows 7), 1/3 of the students work on their own laptops (Mac OS X, Windows 7, Windows 10).
The major issue we encounter is that in almost all of the cases, OpenSesame is extremely slow to start (both the installed and the portable version, both Mac and Windows) when opening for the first time. This can take anywhere from 1 to even 10 minutes (yes, we have timed this..). The same holds for the first time an experiment is run.
Preferably, we'd like OpenSesame to start faster. More importantly, we would like to see some feedback that OpenSesame is in fact loading the software, or loading the experiment. Our students get frustrated not knowing what happens (and click OpenSesame.exe a number of times), and we cannot easily verify that OpenSesame is in fact loading, and has not crashed (or something of the sorts).
Our courses start again in November, meaning that before the summer we will decide whether to use OpenSesame for our courses in the coming year.
Comments
Hi Roy,
Good to hear that you used it!
Slow startup is a known issue on Mac OS (±15 seconds on an older iMac), and it can be a little slow on Windows too (but much less so).
But the numbers you give, from 1 to 10 minutes, is something else. Here's my guess: The computers that show this are university computers that are remotely managed. When you start OpenSesame, at least for the first time, the software is first downloaded from a server. This can take a long time. Could that be it?
Cheers,
Sebastiaan
Check out SigmundAI.eu for our OpenSesame AI assistant!
Hi Sebastiaan,
The students working on university computers run OpenSesame portable from a USB stick (sometimes one of their own, but mostly one we provide; Kingston 8GB DataTraveler 100 USB2.0)
Students working on their own systems sometimes use a portable version too (as we explicitly explain the advantages of having portable software, e.g. during internships / theses), otherwise an installed version. Of course we cannot completely control what laptops they use, but I do state that on their own system it may be slower if the laptop isn't too fast.
Do you reckon this might be solved if we were to switch to USB3 sticks? When we were running OpenSesame 2.7 we did not encounter any timing problems such as this with our current USB sticks.
cheers
Hi Sebastiaan,
Jeroen and I did some more research into the problem.
Starting OpenSesame 3.1 portable from USB2 stick on our student computers takes ~30 seconds cold start, and ~7 seconds warm start. From USB3 strick ~20 / 6 seconds (on USB2 port), ~18 / 5 seconds (on USB 3 port).
This is remarkably different from the times we observed during the courses. The major difference is that the room was pretty quiet now. Consequently, I think you may be right in that a remote server is involved. We're going to look into it with the IT department. If anything else comes up, we'll let you know.
Thanks!
Hi Roy,
I have the luxury that I only have machine's with SSDs at my disposal so I can't really test the opening speeds on data carriers with lower throughput capacities such HDDs. The fact that you run OpenSesame from a USB drive may definitely be one of the factors. USB2 is very/too slow for the requirements of most apps nowadays. OpenSesame has become a very complex program which loads a lot of auxiliary modules to function correctly. This of course involves a lot of read operations, which may be a bottleneck. Furthermore, every once in a while (I don't really know exactly how often), python recompiles .py files that it uses to .pyc or .pyo files, which involves a lot of write operations and these are even slower on most usb drives! Besides, there are a lot of different usb drives on the market which have a large variation in performance. I can imagine loading programs can take a couple of minutes from a slower type of USB stick.
I suggest to not run OpenSesame from USB drives, or use one of at least USB3 or better. I bet that if you run OpenSesame from a faster usb drive or from the hard drive/ssd of the main machine, you get far better results.
As a sidenote: we hope to make the switch from Qt4 to Qt5 (the GUI platform OpenSesame is made with) in the near future and this should also improve startup times.
Hi Roy,
I think I have found the problem of the slow startup -- I experienced it myself yesterday!
What made this difficult to find is that it only occurs when the machine that OpenSesame is running on doesn't have an Internet connection; how many times does that still happen nowadays?
The culprit is the OSF extension: on startup of OpenSesame, it tries to download the latest credentials for connection to the Open Science Framework from cogsci.nl (to make sure older versions of OS don't suddenly stop working if these credentials ever change). However, opposed to all other connection requests to the OSF which occur asynchronously (they don't block processes), I still use a library for this that works synchronously. This means that when this library can't connect to cogsci.nl (due the lack of an Internet connection), it keeps trying until a certain timeout, which can be very long as you experienced, and blocks OpenSesame from starting up until it is finished. This is of course a major problem that you only encounter when you're offline, but nevertheless we have to think of a solution for this.
I have created an issue for this at https://github.com/dschreij/opensesame-extension-osf/issues/14
Hi Daniel,
Yes, I've had this as well, for example when working on a train. But what makes you think that this was the issue for Roy?
Cheers,
Sebastiaan
Check out SigmundAI.eu for our OpenSesame AI assistant!
I am not entirely certain, but I think it is the most likely explanation for delays as long as 10 minutes that Roy experienced. For me it wasn't that long, it only took a minute or 2 before OpenSesame popped up, but I can imagine that on slower computers it will take even more time. It's a wild guess, but I think we should follow up this lead.
Hi Daniel, Sebastiaan,
(took me a while to respond, apparently the notifications were popping up in my spam folder)
Good to hear that a possible issue has been identified. We are going to conduct another (more expansive) test with the IT department tomorrow; running OS 3.1 on about 40-60 PC's simultaneously from USB stick. If anything interesting pops up here I will let you know. I will also make sure to mention the connection to cogsci.nl to them, maybe they can zoom in on that specifically.
Hi guys,
Last week we conducted another test, and unfortunately, the results were worse this time, an overview of the tests we've conducted so far:
1 PC:
cold start max 30 seconds, warm start max 7 seconds
12 PCs:
cold start 1:40 - 2:40, warm start 0:40-1:40;
42 PCs:
cold start: first OS opened after 2:30, most of them after 6:00-6:30, last on at 7:30.
warm start: all < 0:30
These tests we're conducted in the same computer room at our faculty. During the last test (the one with 42 PCs), one computer was monitored by the IT department. Nothing weird came up on their end. The network-monitoring file is available here (WireShark log): https://surfdrive.surf.nl/files/index.php/s/Y5pFWwefLh7iFYt
@Daniel perhaps you could check whether you see any reference to the OSF credentials, I couldn't locate it on my first try; but I'm not sure what I'm looking for.
Another option (though I'm not sure if feasible) would be to let us test out a compiled version without the OSF extension, so we can check to see whether that solves the issue.
There are three extensions that do network stuff. All three can be disabled through Tools → Plug-in and extension manager.
Just a sanity check: If you start another large problem on 42 PCs at the same time, do you get the same trouble?
Check out SigmundAI.eu for our OpenSesame AI assistant!
That's good to know, I assume that info is stored somewhere for the portable version too?
We haven't had this with other software (we ran OS 2.7 for a long time without any issues like this); nor have we had it recently with other software that I know of.
Yes, and each extension corresponds to a folder in (if I remember correctly)
Lib\site-packages\share\opensesame_extensions. If you don't want an extension, you can simply delete its folder.Check out SigmundAI.eu for our OpenSesame AI assistant!
We've run another test today. ~40 PC's with OpenSesame 3.1 from USB sticks without the internet plugins. Start-up times seem to be reduced to 2:00-2:30 minutes cold start. When we run a local version (not from USB) start-up times are around 1:30 (which still seems kinda steep though).
We can't figure out what else to try to reduce start up times.. Any ideas?