Welcome!

Sign in with your CogSci, Facebook, Google, or Twitter account.

Or register to create a new account.

We'll use your information only for signing in to this forum.

Supported by

Datamerger

NikkieKNikkieK Posts: 13

Hello Daniel,
I am unable to install Datamerger on my system. I am using Ubuntu 16.04 and i followed the steps mentioned. It says "unable to locate package datamerger". How do i install it?

Thank you!

Comments

  • sebastiaansebastiaan Posts: 2,667

    Hi Nikkie,

    The DataMerger is quite outdated by now. Below is a simple script that allows you to do more or less the same thing. The way it works:

    • Add all .csv files that you want to merge to the OpenSesame file pool
    • Add this script as an inline_script item
    • In the inline_script item, select all text, and click on the 'Run selected text' button. (So don't run the experiment itself!)
    • You will then find the merged files in the file pool

    I'll see if I can turn this into a simple extension to make it easier.

    Cheers!
    Sebastiaan

    import os
    from datamatrix import io, DataMatrix
    
    # This points toward the file pool if you run the script from within OpenSesame
    # by selecting the script and clicking on the 'Run selected text' button.
    # This will *not* work when running as an experiment, or as a standalone script.
    DIRNAME = opensesame.experiment.pool_folder
    
    dm = DataMatrix()
    # Loop through all files in the source directory
    for basename in os.listdir(DIRNAME):
        # Skip files that aren't csv or start with merged (because that's the
        # output file)
        if not basename.endswith('.csv') or basename.startswith('merged'): 
            continue
        try:
            filedm = io.readtxt(os.path.join(DIRNAME, basename))
        except Exception as e:
            print('Failed to open %s (Exception: %s)' % (basename, e))
        else:
            # Add a source column, and then concatenate the input file and the full
            # DataMatrix
            print('Read %s' % basename)
            filedm.source = basename
            dm <<= filedm
    # Save to the source directory as both csv and xlsx
    io.writetxt(dm, os.path.join(DIRNAME, 'merged.csv'))
    io.writexlsx(dm, os.path.join(DIRNAME, 'merged.xlsx'))
    

    There's much bigger issues in the world, I know. But I first have to take care of the world I know.
    cogsci.nl/smathot

  • NikkieKNikkieK Posts: 13
    edited April 26

    Hello again...
    So this happened when I ran the script...what went wrong?

     /usr/lib/python2.7/dist-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building  the font cache using fc-list. This may take a moment.
       warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
     /usr/lib/python2.7/dist-packages/datamatrix/_datamatrix/_basecolumn.py:33: UserWarning: Install  fastnumbers for better performance
       warnings.warn('Install fastnumbers for better performance')
     Read subject-8.csv
     Read subject-9.csv
     Read subject-13.csv
     Read subject-6.csv
     Read subject-5.csv
     Read subject-14.csv
     Read subject-11.csv
     Read subject-7.csv
     Read subject-10.csv
     Read subject-15.csv
     Read subject-1.csv
     Read subject-3.csv
     Read subject-12.csv
     Read subject-2.csv
     Read subject-4.csv
     /usr/lib/python2.7/dist-packages/datamatrix/_datamatrix/_numericcolumn.py:35: UserWarning: Install  fastnumbers for better performance
       warnings.warn('Install fastnumbers for better performance')
    ---------------------------------------------------------------------------
    CellCoordinatesException                  Traceback (most recent call last)
    <ipython-input-1-d06b02b6e2e2> in <module>()
         26 # Save to the source directory as both csv and xlsx
         27 io.writetxt(dm, os.path.join(DIRNAME, 'merged.csv'))
    ---> 28 io.writexlsx(dm, os.path.join(DIRNAME, 'merged.xlsx'))
    
    /usr/lib/python2.7/dist-packages/datamatrix/io/_excel.pyc in writexlsx(dm, path)
         90         ws = wb.active
         91         for colnr, colname in enumerate(dm.column_names):
    ---> 92                 ws[chr(ord('A')+colnr)+'1'] = colname
         93         for rownr, row in enumerate(dm):
         94                 for colnr, (colname, value) in enumerate(row):
    
    /usr/lib/python2.7/dist-packages/openpyxl/worksheet/worksheet.pyc in __setitem__(self, key, value)
        337 
        338     def __setitem__(self, key, value):
    --> 339         self[key].value = value
        340 
        341 
    
    /usr/lib/python2.7/dist-packages/openpyxl/worksheet/worksheet.pyc in __getitem__(self, key)
        333         if ":" in key:
        334             return self.iter_rows(key)
    --> 335         row, column = coordinate_to_tuple(key)
        336         return self._get_cell(row, column)
        337 
    
    /usr/lib/python2.7/dist-packages/openpyxl/utils/__init__.pyc in coordinate_to_tuple(coordinate)
        160     Convert an Excel style coordinate to (row, colum) tuple
        161     """
    --> 162     col, row = coordinate_from_string(coordinate)
        163     return row, _COL_STRING_CACHE[col]
        164 
    
    /usr/lib/python2.7/dist-packages/openpyxl/utils/__init__.pyc in coordinate_from_string(coord_string)
         40     if not match:
         41         msg = 'Invalid cell coordinates (%s)' % coord_string
    ---> 42         raise CellCoordinatesException(msg)
         43     column, row = match.groups()
         44     row = int(row)
    
    CellCoordinatesException: Invalid cell coordinates ([1)`
    
  • sebastiaansebastiaan Posts: 2,667

    Ow right. There used to be a bug when writing .xlsx files with a large number of columns. You can work around this by either:

    • commenting out the line where the .xlsx file is written. You'll still have the .csv, so that's probably fine; or:
    • update python-datamatrix. You can do this with pip install python-datamatrix --upgrade.

    There's much bigger issues in the world, I know. But I first have to take care of the world I know.
    cogsci.nl/smathot

Sign In or Register to comment.