Participant data visible in individual result but missing from bulk CSV export
Hi,
I am running a behavioural experiment on a JATOS server, with the experiment implemented in PsychoPy/PsychoJS.
For each participant, I generated a unique JATOS link. When generating each link, I entered the participant's participant code and name in the Comment field so that I could identify which result belonged to which participant.
For most participants, the data appears normally in the bulk results export. However, for some participants who have definitely completed the task, their results are not appearing in the combined/bulk exported data.
I have checked the individual results on the JATOS server, and the situation is interesting:
- The individual result is present on the JATOS server.
- When I select the individual result and download/extract the result as a TXT file, the task data is clearly present in the TXT file.
- However, when I convert this TXT data to CSV, the resulting CSV does not contain the expected data.
- I have also noticed that some fields/variables may be missing or formatted differently in these problematic files, although the actual trial data appears to be present.
- Additionally, the Participant Code that was entered in the JATOS link/comment is not appearing in the final exported raw dataset, making it difficult to associate these results with participants.
I would therefore like to understand:
- Is the Comment field associated with each generated JATOS link stored in the result metadata, and is there a way to include/export this Comment field together with the corresponding result data?
- Why might a result that is visible and contains data when downloaded individually as TXT not appear correctly in the bulk CSV export?
- Is there a recommended way to recover/export these individual results while preserving all the trial-level data and metadata?
- Could differences in the structure or data types of individual
data.txtfiles cause the CSV export/conversion to omit the data? - Is there a recommended JATOS export format or procedure for recovering these results without losing any trial data?
I am particularly concerned because the individual result files appear to contain the data, so I would like to make sure that the data are not being lost simply during the export/conversion process.
I can provide a sample of the problematic .txt result file and/or the JATOS results export if that would help diagnose the issue.
Thank you!
Comments
Hi,
Yes, it would be helpful if you could include the JATOS results export. But it sounds like this is the expected behaviour, and that there is an easy solution
The participant code that you entered in JATOS (when creating the link) is something that happens in the backend. It will not be automatically included in the data that the experiment creates and submits to the server (this is the result data, what you see on the .csv). (This is important: JATOS will not modify your data unless you tell it to do so in the front end, using functions from the jatos.js library).
So you are trying to merge metadata into your result data. What I tend to do is write a simple script (e.g. in R or python) that reads in both the metadata and the result data and merges them together.
Hope this helps
Elisa
If I understand your issue correctly, some completed study results are missing or empty in the bulk CSV export, even though the corresponding individual results exist in JATOS and their downloaded
data.txtfiles contain the trial data. Can you give us an example of the data that are missing in CSV but exists in the data.txt? How do you convert from TXT -> CSV?Best,
K.