Importing special data types

Some beamlines write out data in a form that IFEFFIT is not capable of reading without help. To address this problem, DEMETER offers a simple plugin architecture. A plugin is a bit of perl code which transformed the problematic file into a form that IFEFFIT can handle. Plugins are small and hopefully quick and easy to write.

Filetype plugins

NSLS beamline X15B uses an antiquated data acquisition system which saves data in a quirky binary format. One of the plugins for file type conversion that ships with DEMETER knows how to interpret this quirky format. Here is how data from that beamline are imported into a DEMETER program:

#!/usr/bin/perl
use Demeter;

my $filetype = Demeter::Plugins::X15B->new(file=>'file_from_x15b.001');
if ($filetype->is) {
  my $converted = $filetype->fix;
  my $data = Demeter::Data->new(file => $converted,
                                $filetype -> suggest("fluorescence")
                               );
};

At line 4, a new Plugin object is made and pointed at the problematic file. At line 4, a check is made to verify that the file actually is of the X15B type. When recognized as such, the problematic file is converted into an IFEFFIT friendly form at line 6. A normal Data object is then created at line 7 and 8 using the converted file and the Plugin's suggestion for how to form fluorescence data from the columns in the converted file.

Every plugin must offer three methods:

is

This method identifies a file as being of that type and returns a boolean value. It is quite important that this method be fast. A program might need to check a file against many different plugins. If this method is slow, than any program using it will seem unresponsive to the user.

fix

This method transforms the file into a form easily imported by IFEFFIT and writes the transformed data to a transitional file. This transitional file is usually placed in the stash directory (link to section explaining the stash directory), but that is not a strict requirement. The fully resolved name of the transitional file is returned. This method can use any tools available, ranging from straight perl to IFEFFIT to serious math packages such as the perl data language or the Cephes library.

suggest

This method returns an array of suggestions for forming transmission or fluorescence data from the transformed file. Specifically, this returns an array containing the energy, numerator, denominator, and ln attributes. See the section on column data.

Plugins that ship with Demeter

X10C

Convert files from NSLS beamline X10C. These files have headers which confuse IFEFFIT and sometimes fail to have white space separating numbers among the data columns. The plugin comments out the headers and corrects the problem with white space between columns.

X15B

Convert files from NSLS beamline X15B. This beamline uses an ancient data acquisition system which writes files in a cryptic binary format. This plugin converts these data to a simple column data file, saving only the scalars containing the XAS-relevant measurement channels.

PFBL12C

Convert files from Photon Factory XAS beamlines. These files have headers which will confuse IFEFFIT's file import and store data as a function of monochromator angle. This plugin comments the header and converts mono angle to energy using information about the crystal type contained in the header. The plugin name makes specific reference to beamline 12C for historical reasons. It will actually work on XAS data from any Photon Factory beamline.

SSRLB

Convert SSRL binary data file. Yes, SSRL does provide a program for converting these binary files to column ASCII data. This plugin does the same chore, yielding a file easily read by IFEFFIT.

SSRLA

Convert SSRL ASCII data file. Presumably, these ASCII files are the result of the SSRL conversion program. These ASCII files are unreadable by IFEFFIT. This plugin, comments out the header lines, constructs a column label line out of the Data: section, moves the first column (real time clock) to the third column, and swaps the requested and acheived energy columns.

SSRLmicro

Sam Webb's microprobe data acquisition program writes files with lots of columns and with a header structure that cannot be easily used by IFEFFIT. This plugin massages that file format into a form more easily ready by IFEFFIT, keeping only the ROI columns. (Note that this plugin could be modified quite easily to perform a simple ICR/OCR deadtime correction.)

HXMA

Files from the HXMA beamline at the Canadian Light Source are readable by IFEFFIT, but the columns are labeled in a way that IFEFFIT is unable to use. This plugin restructures the header for IFEFFIT's convenience and keeps only the columns containing the ion chambers and the corrected (presumably by a simple ICR/OCR deadtime correction) ROI signals from the multi-element detector.

CMC

Files from APS beamline 9BM (CMC-XOR) are single-record Spec files. As a result, these data files contain lots of useless column (for example, each file inexplicably saves h, k, and l values). This plugin discard all the useless columns, keeping only those from the ion chambers and the multi-element detector. It also discards the problematic “logi0i1” column, which can result in NaN entries in the case of zero signal on the transmission detector.

X23A2MED

Data measured using the Vortex silicon drift detector at NSLS X23A2 are imported and deadtime corrected using the point-by-point iterative algorithm developed and implemented by Joe Woicik and Bruce Ravel and described in J. Woicik, et al., J. Synchrotron Rad., 17, (2010) p. 409. The output data file contains columns for each corrected detector channel as well as columns for the various ion chambers. This is an example of a file type plugin which uses IFEFFIT dirrectly.

DUBBLE

Files from the DUBBLE beamline (BM26) at ESRF. This plugin converts monochromator angle into from millidegrees to energy and (as needed) disentangles the confusing layout of data from the multi-element detector, writing out a file that can easily be imported by Athena.

Lytle

Import files from the Lytle database. This plugin imports those data that are recorded by encoder value and which have headers that start with the word NPTS and have the mono d-spacing and steps-per-degree in the second line. There is another common file format in the Lytle database (the header begins with CUEDGE and does not record the mono parameters) that is not handled by this plugin. See question 3 at http://cars9.uchicago.edu/ifeffit/FAQ/Data_Handling.