3. Data import

ATHENA is very versatile in how she reads in data files. Pretty much any data in the form of columns of numbers can be successfully read. With a few exceptions, ATHENA relies upon IFEFFIT's read_data() command to handle the details of data import. IFEFFIT is clever about recognizing which part of a file is columns of numbers and which part is not. In the following, I'll explain how the read_data() command interprets files, explain the limits on its and ATHENA's abilities to interpret a data file, and discuss the kinds of manipulations of data that can and cannot be performed by ATHENA as data are imported.

ATHENA expects data of one of a few types. Column data in which the columns represent such things as the energy grid and the scalars measured during the experiment are the most common sort of data that most people use import into ATHENA. ATHENA's column selection dialog is used to convert the raw scalars into μ(E) data. Other common kinds of data files that might be read into ATHENA are files that contain μ(E) or χ(k) data in columns or the output files from FEFF, xmu.dat and chi.dat.

Here is an example of a data file that will make ATHENA quite happy. There are some header lines, followed by a line of dashes, followed by a line of column labels, followed by lines containing columns of data.

# X15B  project: MT 9/23/04
# original file: STD1.001
# unpacked from original data as a sequence of 4-byte floats
# -----------------------------------------------------------
#   energy           I0          narrow        wide
    2400.0020    60183.3008       38.5000       83.0000
    2401.5088    60241.0508       41.5000       82.0000
    2403.0078    60347.5508       40.0000       83.7500
    2404.5039    60531.0508       42.2500       78.2500
  ... etc ...

In this example of a well formatted file, the header lines, the line of dashes, and the column labels line are all preceded by a hash (#) mark. IFEFFIT is thus able to recognize these as header lines. Since IFEFFIT recognizes them as such, ATHENA will store them in the project along with the data. Because there is a line of dashes and because it is followed immediately by column labels, ATHENA is able to use these labels in the column selection dialog. A few other common US keyboard symbols, such as % or \*, will also be understood as marking header lines.

The numbers in the columns can be integers, floats (such as 1.234, -1.234, .1234 or -.1234), or exponentials (such as 1.23e45 or -1.23E-45). Anything interpretable as a number in the C programming language will be interpretable in this context. The columns of numbers go to the end of the file. There is no text following the data.

When data is recorded as described above, it will be fully utilized by ATHENA. The headers will be recorded, the column labels will be used, the data will be interpreted. ATHENA can, however, accommodate significant deviation from the format described above.

  • If the header lines are not marked by a # or some other recognizable marking character, IFEFFIT will not be able to recognize headers or column labels. As long as no text follows the data, the columns will still be understood as columns of data and the data can be imported by ATHENA.
  • If the line of dashes is missing, again the headers and column labels will not be recognized, but the columns of data will be.
  • If no headers are in the file, the columns of data will still be understood as data.

ATHENA expects that the data are recorded as a function of energy and that one of the columns contains energy values. The assumption is that the first column is the energy column, but that can be changed in the column selection dialog. ATHENA works in eV. If data are recorded in keV, there is a menu in the column selection dialog that must be set accordingly.

Here are some operations that can be performed as data is imported.

  1. Data from a multi-element detector can be summed on the fly.
  2. Data from a multi-element detector can be imported such that each detector channel is imported into its own data group.
  3. Data can be negated, i.e. multiplied by -1, or multiplied by an arbitrary constant
  4. A reference channel can be read from the the same file.
  5. Data can be preprocessed. That is, data can be truncated, deglitched, aligned to a standard, and have its parameters constrained to a standard

Here are some operations that can be handled using the Plugin architecture.

  1. Data can be imported as a function of pixel position on an area or linear detector.
  2. Conversion from wavelength to energy.
  3. Conversion from encoder reading or motor steps to energy.
  4. Conversion of data in a binary format
  5. Dead-time corrections using columns from the data file.
  6. Any math expression more complicated than sums of columns in the numerator and denominator, e.g. plugins allow you to multiply the If column by 7 and divide by the sine of the I0 column, if that's what you really want.

If some of the criteria for the data file format are not met, for example if there is text following the data columns or if you need to perform one of the operations not yet supported, you will need to process you data before trying to import into ATHENA.

There are examples of data files that ATHENA will process before sending off to IFEFFIT for import. An example is the data file format from beamline X10C at NSLS. Files from that beamline cannot be imported as written by IFEFFIT's read_data() command. ATHENA will recognize such a file and process it as needed before importing it. This can be done with other beamlines. You should contact Bruce if you are the beamline scientist or a frequent user of some beamline which writes data in a way that read_data() cannot import.

As a final comment, I would encourage beamline scientists and the authors of data acquisition software to consider their users when designing data file formats. While I certainly will not say that beamlines should be required to accommodate ATHENA or even that beamline staff have any obligation to recommend ATHENA to their users, the truth is that ATHENA is widely used in the XAS community. A format that serves ATHENA well is likely a fine format that can be imported by a very wide variety of XAS software, plotting software, spreadsheets, and other programs. Your users would be well served by your adopting it.



DEMETER is copyright © 2009-2016 Bruce Ravel – This document is copyright © 2016 Bruce Ravel

This document is licensed under The Creative Commons Attribution-ShareAlike License.

If DEMETER and this document are useful to you, please consider supporting The Creative Commons.