reading .xpd file

Comments

• fladd

  June 2017

  Hi Sarah,

  it seems that you forgot to add the data variable names to the data file.
  You have to define them once at the beginning of your experiment (see here for an example).

  In your case that would be:

  exp.data_variable_names = ["Response", "RT"]
  

  Best,
  Florian

  Warning There was an error rendering this rich post.
• Sarah

  June 2017 edited June 2017

  I had exp.data_variable_names in my code but still I cant see the name of the variables
  here is the code:

      exp.add_data_variable_names = ["button", "rt"]
      #### start experiment############
      expyriment.control.start()
  
  
      for rule, block in zip(rule_list, exp.blocks)
  
  
          for trial in block.trials:
  
  
              st=trial.stimuli[0].present()
  
              button, rt = exp.keyboard.wait(keys=response_keys,duration=reaction_time-st)
              if rt is not None:
                 exp.clock.wait(reaction_time-rt) 
              exp.data.add([button,rt])
              exp.data.save()  
  
          bt=blank.present()
          exp.clock.wait(blank_page_time-bt)
  

  any comment?
• fladd

  June 2017

  Hi Sarah,

  you have add_data_variable_names in your code, which is a method, not an attribute, like data_variable_names which I suggested to use.

  Hence, either of the following two will work:
  1. exp.add_data_variable_names(["response_keys", "RT"])
  2. exp.data_variable_names = ["response_keys", "RT"]

  Best,
  Florian

  Warning There was an error rendering this rich post.
• Sarah

  June 2017 edited June 2017

  Here is the result of the task in brief:

      subject_id,block.name,button,rt
      3,A4,275,2
      3,A4,275,121
      3,A4,None,None
      3,A4,None,None
  

  Is there any way of programming that I could open .xpd file in a way that I could above result in an array? I think working with an array is much easier for further processing. Unfortunately, The short code that I paste below did not do the job

      data_folder='./data'
      file_name='./name'
      agg = data_preprocessing.Aggregator(data_folder, file_name, suffix='.xpd')
      #
      agg.set_subject_variables(["subject_id","b.name","button", "rt"])
      agg.set_computed_variables(["button", "rt")]
  

  BTW: In my code, I only saved 4 variables but in the .xpd file I could see a long list of block name is saved:
  Is there any way to remove these on .xpd file?

          #e     block factors: 
      #e     n trials: 7
      #e     trial factors: 
      #e Block 1: AS6
      #e     block factors: 
      #e     n trials: 7
      #e     trial factors: 
      #e Block 2: T6
      #e     block factors: 
      #e     n trials: 7
      #e     trial factors: 
      #e Block 3: T3
      #e     block factors: 
      #e     n trials: 7
      #e     trial factors: 
      #e Block 4: A8
  

  Thanks
• fladd

  June 2017

  The Expyriment log files are all saved in a standard comma separated value (csv) format, and it should hence be relatively straight forward to convert them into any other data structure or parse them with external tools if necessary.
  The block names (the whole design actually) are entered in the log files as a csv comment, so it will not interfere with your data, when processing it further outside of Expyriment.

  Concerning the code snippet of the Expyriment preprocessing, it seems that filename is not the correct value. It should be set to a string that represents how the data files begin. For instance, if my experiment is called "MyExp", then I would also set filename in the Aggregator to "MyExp", since all the data files will start with this. For more information on the data preprocessing Aggregator, please see: http://docs.expyriment.org/expyriment.misc.data_preprocessing.Aggregator.html

  Best,
  Florian

  Warning There was an error rendering this rich post.
• Sarah

  June 2017

  Dear Flad, Thanks for your reply. I am sorry but I don't why in the output csv file I only could see the *first rows od the data * ( instead of 20 rows). It's very starnge... Here is the code:

      data_folder='./data'
      file_name='nback'
      agg = data_preprocessing.Aggregator(data_folder, file_name, suffix='.xpd')
  
      agg.set_subject_variables(["b.name","button", "rt"])
              agg.aggregate(output_file="out.csv")
  
  
  
      import csv
      import codecs
  
      filename= "out.csv"
  
      data = []
      with codecs.open(filename) as f:
          for line in f:
              data.append(line.strip().split(","))    
  

  Do you have any idea why this happens?
  Best
• fladd

  June 2017

  Hi Sarah,

  so you are saying you only get the aggregated data for the first subject? Are the data files of the remaining 19 subjects named in the same way (i.e. starting with nback)?

  I also don't understand why you define "button" and "rt" as between-subject factors. From your last examples it seemed that they are within-subject factors. Maybe you could explain in more detail what kind of experiment you were running and what exactly you would like to achieve?

  Best,
  Florian

  Warning There was an error rendering this rich post.
• Sarah

  June 2017 edited June 2017

  Hi Florian,

  Thank you for your email. In only run one experiment. Therefore, in the folder, there was only one file associated with one participant. In this single .xpd file, I had 20 trials. When I open the .xpd file manually, I could see the results of those 20 trials. In the next step, I would like to have all those 20 trials ( for one participant) in a .csv file and I used the code below. When I open the created .csv file I noticed I only could see the information for first trial ( only first row) so the other 19 trials are missing. Please let me know what is your thoughts: here is the code:

      from __future__ import print_function
      from expyriment.misc import data_preprocessing, constants
  
  
      data_folder='./data'
      file_name='nback'
      agg = data_preprocessing.Aggregator(data_folder, file_name, suffix='.xpd')
          agg.aggregate(output_file="out.csv")
  
  
  
      #import csv
      #import codecs
  
      filename= "out.csv"
  
      data = []
      with codecs.open(filename) as f:
          for line in f:
              data.append(line.strip().split(","))
  

  Thanks
• fladd

  June 2017

  Dear Sarah,

  I think there is some confusion about what the Aggregator in the data_preprocessing package does. Its purpose is to aggregate data from several participants into a single file by creating per-subject averages for all factor-level combinations, which you can then use to run a repeated-measures ANOVA on (for instance in SPSS). If there is only data from a single subject, this data will be averaged into a single line.

  You further state that your goal is to create a csv file with all 20 trials for your participant. You already have this: This is exactly what the .xpd file should be, if you add the data to it after every trial (which I understood you do, given your earlier posts). So there is no need to transform the .xpd file into csv format - it already is in csv format. If you are using Windows and some other software there expects your csv files to end with .csv for some reason, you can just rename them to end with .csv, instead of .xpd, but in either case, the content of the file fulfills the csv specification: https://en.wikipedia.org/wiki/Comma-separated_values.

  I hope this helps. Let me know if you have any further questions.

  Warning There was an error rendering this rich post.