Can't export/import study
I know this is a vague issue, but something seems to be going wrong when I am exporting my study. First, it takes a while (on a local host) for a jzip file to get created. The logs don't show any errors:
2022-09-09 17:13:39,728 [INFO] - gui_access - GET /jatos/2/export (admin)
However, the exported file is only 4 bytes (I am expecting larger size). Finally, when I try to import the jzip file into my remote JATOS server, I get a "Import of study failed" message in the GUI and the following error in the application logs:
2022-09-09 15:20:55,639 [WARN] - s.g.ImportExportService - .unzipUploadedFile: unzipping failed
java.util.zip.ZipException: zip END header not found
at java.base/java.util.zip.ZipFile$Source.zerror(ZipFile.java:1607)
at java.base/java.util.zip.ZipFile$Source.findEND(ZipFile.java:1497)
at java.base/java.util.zip.ZipFile$Source.initCEN(ZipFile.java:1504)
at java.base/java.util.zip.ZipFile$Source.<init>(ZipFile.java:1308)
at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1271)
at java.base/java.util.zip.ZipFile$CleanableResource.<init>(ZipFile.java:733)
at java.base/java.util.zip.ZipFile$CleanableResource.get(ZipFile.java:850)
at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:248)
at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:177)
at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:191)
at utils.common.ZipUtil.unzip(ZipUtil.java:43)
at services.gui.ImportExportService.unzipUploadedFile(ImportExportService.java:331)
at services.gui.ImportExportService.importStudy(ImportExportService.java:85)
at controllers.gui.ImportExport.importStudy(ImportExport.java:109)
at gui.Routes$$anonfun$routes$1$$anonfun$applyOrElse$105$$anonfun$apply$151$$anonfun$apply$152.apply(Routes.scala:3060)
at gui.Routes$$anonfun$routes$1$$anonfun$applyOrElse$105$$anonfun$apply$151$$anonfun$apply$152.apply(Routes.scala:3060)
at play.core.routing.HandlerInvokerFactory$$anon$5.resultCall(HandlerInvoker.scala:147)
at play.core.routing.HandlerInvokerFactory$$anon$5.resultCall(HandlerInvoker.scala:146)
at play.core.routing.HandlerInvokerFactory$JavaActionInvokerFactory$$anon$10$$anon$2$$anon$1.invocation(HandlerInvoker.scala:112)
at play.core.j.JavaAction$$anon$1.call(JavaAction.scala:128)
at play.mvc.Action.lambda$call$0(Action.java:89)
at java.base/java.util.Optional.map(Optional.java:265)
at play.mvc.Action.call(Action.java:81)
at play.http.DefaultActionCreator$1.call(DefaultActionCreator.java:33)
at play.mvc.Action.call(Action.java:66)
at controllers.gui.actionannotations.GuiAccessLoggingAction.call(GuiAccessLoggingAction.java:37)
at controllers.gui.actionannotations.AuthenticationAction.call(AuthenticationAction.java:138)
at play.mvc.Action.lambda$call$0(Action.java:89)
at java.base/java.util.Optional.map(Optional.java:265)
at play.mvc.Action.call(Action.java:81)
at play.db.jpa.TransactionalAction.lambda$call$0(TransactionalAction.java:36)
at play.db.jpa.DefaultJPAApi.lambda$withTransaction$5(DefaultJPAApi.java:302)
at play.db.jpa.DefaultJPAApi.withTransaction(DefaultJPAApi.java:207)
at play.db.jpa.DefaultJPAApi.withTransaction(DefaultJPAApi.java:298)
at play.db.jpa.TransactionalAction.call(TransactionalAction.java:35)
at play.core.j.JavaAction$$anonfun$10.apply(JavaAction.scala:188)
at play.core.j.JavaAction$$anonfun$10.apply(JavaAction.scala:188)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at play.core.j.HttpExecutionContext$$anon$2.run(HttpExecutionContext.scala:77)
at play.api.libs.streams.Execution$trampoline$.execute(Execution.scala:70)
at play.core.j.HttpExecutionContext.execute(HttpExecutionContext.scala:69)
at scala.concurrent.impl.Future$.apply(Future.scala:31)
at scala.concurrent.Future$.apply(Future.scala:494)
at play.core.j.JavaAction.apply(JavaAction.scala:189)
at play.api.mvc.Action$$anonfun$apply$2.apply(Action.scala:95)
at play.api.mvc.Action$$anonfun$apply$2.apply(Action.scala:88)
at scala.concurrent.Future$$anonfun$flatMap$1.apply(Future.scala:253)
at scala.concurrent.Future$$anonfun$flatMap$1.apply(Future.scala:251)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:92)
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:92)
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:92)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:49)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2022-09-09 15:20:55,640 [INFO] - g.ErrorHandler - JatosGuiException during call /jatos-server/jatos/import/study: Import of study failed
I've been working on the study for a while and was able to export/import it before. Not sure what went wrong.
@kri Any ideas?
Comments
Hi,
Something seem to go wrong when you export the study. You are right the jzip should be more the 4 byte and the import error in your log just reflects that the zip (the jzip is just a zip archive with a non-standard file suffix) is somehow faulty. Weird thing is that there is no error when you export the study. Can you export other studies without problems? And what's the size of your study's study assets folder?
Best,
Kristian
A stupid non-solution: have you tried exporting it with a different browser?
Thanks, @elisa and @kri for your input! Using a different browser did not work because I suspect my `study_assets` folder is too big. The size is due to a large `.git` directory. I know why it is so big, and I think it could be smaller, but is there a way to export my study while ignoring certain files/directories like `.git`?
If it's problematic, I'll just have to wipe out my git history. It wouldn't be the end of the world, but if I could keep it, I would.
JATOS doesn't have a way to ignore certain files in your study assets. But the .git folder is just like any other folder. You can temporarily move it somewhere else and try exporting the study then. Afterwards you move the .git back into your study assets folder. But it is important to make no changes to your study assets in the mean time since they wouldn't be reflected by git (exporting a study is fine, it doesn't make changes to the files).
But I'm curious: how big is your study assets together with the .git folder? And another question: what's the version of JATOS you are using?
Best
Kristian
I'm using JATOS 3.7.4.
The git folder got too big because I adopted a stupid workflow. Basically, I have my study assets outside the jatos directory as a git repository (I wanted to document my work). But git usual workflows are not fully compatible with jatos because I can't just clone this study assets repo into another jatos installation (e.g., on my remote server) and test my study -- I would also need to create the components anew, add json data etc.
As a workaround, I thought I would just export a jzip into my repo, so that the study can be simply imported into another JATOS instance by cloning the repo and then using the GUI to import. My dumb mistake was in exporting the study without deleting the old jzip, so the old file was added to the new export. It does not take many exports to make the jzip really big. Now I don't do this anymore but I guess the .git folder still has snapshots of these huge files from the past (my .git folder is over 4GB right now :P).
I see.
Basically, I have my study assets outside the jatos directory as a git repository (I wanted to document my work).
Good idea. I've seen people doing this.
But git usual workflows are not fully compatible with jatos because I can't just clone this study assets repo into another jatos installation (e.g., on my remote server) and test my study -- I would also need to create the components anew, add json data etc.
Yes. There you can only do an export/import in JATOS. Maybe in the future we will do a better git integration.
As a workaround, I thought I would just export a jzip into my repo, so that the study can be simply imported into another JATOS instance by cloning the repo and then using the GUI to import. My dumb mistake was in exporting the study without deleting the old jzip, so the old file was added to the new export.
Ah, and now the git repo is full of jzips and maybe even jzips containing other jzips and it got rather big :/ Hm, I'm not a git expert, but isn't there a way to clean up a git history by discarding some commits?
Best,
Kristian
Ah, and now the git repo is full of jzips and maybe even jzips containing other jzips and it got rather big :/ Hm, I'm not a git expert, but isn't there a way to clean up a git history by discarding some commits?
Surely, there is a way to do that, yes. And I will try to do it someday. Just, messing around with git history (e.g. reverting, resetting, etc.) has always been a terrifying experience for me, so I try to do it as little as possible -- it's my last resort xD.