Command disposal and command templates

DEMETER uses a highly abstract system for communicating with FEFF, IFEFFIT, and the plotting . The Text::Template module is used along with a large number of small, external template files. The idea behind Text::Template is quite simple. Perl code is interspersed with normal text. A pass over the text is made and the perl code is evaluated. In this way, a bit of text is made that is specifically relevant to a particular situation. As an example, here is the template that DEMETER uses to construct the IFEFFIT command used to normalize the μ(E) data associated with a Data object.

{ # -*- ifm -*-
  # pre-edge template
  #   {$D->group} returns the ifeffit group name
  #   {$D->parameter} returns the value of that parameter
}

pre_edge("{$D->group}.energy+{$D->bkg_eshift}", 
         {$D->group}.xmu, 
         e0         = {$D->bkg_e0},
         pre1       = {$D->bkg_pre1}, 
         pre2       = {$D->bkg_pre2}, 
         norm_order = {$D->bkg_nnorm},
	 norm1      = {$D->bkg_nor1}, 
         norm2      = {$D->bkg_nor2})

In this example, $D represents the Data object. Thus IFEFFIT's pre-edge command is filled in ith attribute values apprporiate to the current Data object.

The text generated from the processing of the template is then disposed of via one or more avenues, one of which may include sending the text to IFEFFIT for processing.

This section explains all of the disposal and templating options available to a DEMETER program.

---

 

Command disposal

Once commands have been processed using the templating system, there are seevral avenues for disposing of those commands. The most obvious avenue is to send the commands to IFEFFIT so that the actual data can be processed. However, several other options are possible.

The dispose method is a method of the base class and inherited by all objects. It is used all throughout DEMETER to get things done and is available for explicit use in DEMETER programs. The syntax is quite simple:

  $any_object -> dispose($text);
  $any_object -> dispose($text, 'plotting');

The first line is used for any data processing command. The second form, with the second argument to the method, is used to identify command text which is explicitly used to make plots. When using the PGPLOT plotting back end, there is not actually a significant difference between plotting and non-plotting commands since IFEFFIT is used to send command to PGPLOT. However, when using other plotting backends, it is essential that plotting commands be apprpriately flagged. Plotting backends are discussed in more detail in the user interface chapter.

The $text given as an argument to the dispose method typically comes from the evaluation of a template, but can be any text generated in any fashion. Thus, it is a completely generic way for a program to communicate with DEMETER's backends.

The targets of the dispose method are set using the set_mode method, another method of the base class which is inherited by all objects. The syntax of set_mode is consistent with other methods in DEMETER:

  $any_object -> set_mode(screen=>1, backend=>1);

Any command can be sent to multiple targets. The disposal targets which can be set using set_mode are:

ifeffit

When true, commands will be sent to IFEFFIT. It is often useful to turn this disposal target off when debugging DEMETER programs.

screen

When true, commands will be sent to standard output (usually the screen). Turning this disposal target on is often useful when debugging DEMETER programs.

plotscreen

When true, plotting commands will be sent to standard output (usually the screen). Turning this disposal target on is ofetn useful when debugging DEMETER programs.

repscreen

When true, the reprocessed commands (discussed below) will be sent to standard output (usually the screen).

file

When set to a string value, that string will be interpretted as a file name to be opened for writing and the commands will then be written to that file. To append text to a file, the file mode string should begin with the > character.

plotfile

When set to a string value, that string will be interpretted as a file name to be opened for writing and the plotting commands will then be written to that file. To append text to a file, the file mode string should begin with the > character.

repfile

When set to a string value, that string will be interpretted as a file name to be opened for writing and the reprocessed commands (discussed below)will then be written to that file. To append text to a file, the file mode string should begin with the > character.

buffer

When set to an array reference, commands will be pushed onto that array. When set to a scalar reference, commands will be concatinated to the end of the strings held by the scalar.

plotbuffer

When set to an array reference, plotting commands will be pushed onto that array. When set to a scalar reference, plotting commands will be concatinated to the end of the strings held by the scalar.

callback

When set to a code reference, the text of the command will be sent to that code reference as the sole argument. This is useful for user interfaces that want to post-process the commands. For example, this disposal mode is used by ARTEMIS to display colorized text in its command buffer.

plotcallback

When set to a code reference, the text of the plotting command will be sent to that code reference as the sole argument. This is useful for user interfaces that want to post-process the commands. For example, this disposal mode is used by ARTEMIS to display colorized text in its plotting buffer.

feedback

When set to a code reference, the text of IFEFFIT's response to commands will be sent to that code reference as the sole argument. This is useful for user interfaces that want to post-process the commands. For example, this disposal mode is used by ARTEMIS to display colorized text in its command buffer.

---

 

Reprocessed commands

DEMETER tries to use IFEFFIT as efficiently as possibly. On one hand, IFEFFIT the one of the things that makes DEMETER go and so is indispensible. On the other hand, the business of communicating between perl code and the IFEFFIT library is (with one exception) always the slowest thing that DEMETER does. One of the optimizations implemented by DEMETER is the reprocessing of commands targeted for disposal to IFEFFIT.

Command strings in IFEFFIT can be quite long -- up to 2048 characters as it is normally compiled. A command that is split over multiple lines, as the example at the beginning of this section is, will be processed much faster if DEMETER pre-processes the command to remove unnecessary line breaks. Basically this means that everything between parentheses will be sent to IFEFFIT as a single string. This is accomplished within the dispose method via the application of a few regular expressions. The reprocessed string is then sent to IFEFFIT.

As a small example of how reprocessing works, this human-friendly command:

pre_edge("data0.energy+0",
         data0.xmu,
         e0         = -9999999,
         pre1       = -150,
         pre2       = -30,
         norm_order = 3,
         norm1      = 150,
         norm2      = 1800)                                                                                            

will be reprocessed into this one-line command before being shuffled off to IFEFFIT.

pre_edge("data0.energy+0", data0.xmu, e0=-9999999, pre1=-150, pre2=-30, norm_order=3, norm1=150, norm2=1800)

which, when summed over dozens or hundreds of IFEFFIT commands, results in a substantial performance improvement.

The repscreen and repfile disposal targets are provided to debug the behavior of this optimization. Reprocessing is quite well tested. However, if you suspect that reprocessing is damaging the commands sent to IFEFFIT, use one of those disposal channels to see the text that is actually being sent.

---

 

Command templates

DEMETER ships with a lot of templates. Each template encapsulates a small bit of functionality and DEMETER does many things. The templates are organized into “template sets”, which are written for specific backend targets, and “template groups” which, are groups of templates which serve related functions. All template sets must have a complete representation of template groups to be fully functional.

The templates are found in lib/templates/ directory underneath the installation location of the DEMETER package. One of the reasons for explaining the templating system in this level of detail is to underscore that it is quite possible to add new template sets. By following the model of the existing template sets, new output types can be created for DEMETER. Indeed, when finally makes its appearence, it should be relatively simple to extend DEMETER to use it simply by creating an apprporiate template set.

Choosing between template sets is one of the topics of the next section.

---

 

Template sets

Template sets describe backend targets for disposed commands. There are four different categories of template sets:

1. Data processing commands

2. Plotting commands

3. FEFF input templates

4. ATOMS input templates

DEMETER currently ships with four different sets in the data processing category.

1. ifeffit, templates which write the syntax of IFEFFIT is a compact form.

2. iff_columns, templates which write the syntax of IFEFFIT is a more human-readable form.

3. feffit, templates which write the syntax of input files for the old FEFFIT program.

4. demeter, templates which write out perl syntax using DEMETER.

The demeter category might seem a bit strange. Its purpose is, indeed, to allow DEMETER programs to write DEMETER programs. The intent is to allow a GUI to export a file containing a DEMETER program that can be used to make a fit using the same fitting model that was created using the GUI.

The possibility of having these different output targets is the main reason for using a templating system. Having command creation containined in these small template files separate from the code may seem like an unnecessary layer of abstraction and misdirection, but it offers DEMETER a lot of flexibility and power. This is even more evident for the plotting backends.

DEMETER currently ships with three different sets in the plotting category. More information about plotting backends can be found in the user interface chapter.

1. pgplot, templates which write the syntax of IFEFFIT plotting commands, which talk directly to PGPLOT.

2. gnuplot, templates which write Gnuplot plotting scripts. Using Gnuplot involves writing lots of temporary files which contain the data to be plotted. It also requires that Gnuplot be installed on your computer, which is something that you have to do separate from the installation of DEMETER.

3. singlefile, this set of templates is used to export the data to be plotted to a single column file. The main use of this is in a GUI to exprt a file that can be used to replicate an interesting plot -- with offsets, energy shifts, and scaling factors -- in an external plotting program.

In the future, I would like to add more plotting backends to DEMETER. Certainly, any of the plot creation tools from CPAN (such as GD or Chart) would be possible, as would something like Grace, which uses a text file as its input.

DEMETER currently ships with two different sets in the FEFF input template category, one for FEFF6 and one for FEFF8. (Actually the FEFF8 set has not yet been written at the time this document is being written. In fact, DEMETER's FEFF8 interface has not yet been started.

There is only one set in the ATOMS input template category. It seems unlikely that other sets will actually be required.

---

 

Template groups

Template groups define related chores. These chores are

1. process: all data processing chores that do not involve fitting χ(k) data or doing any other sort of data analysis.

2. fit: all chores associated with fitting χ(k) data.

3. analysis: all analysis chores other than those associated with fitting χ(k) data. This might include things like linear combination fitting. principle component analysis, or log-ration/phase-difference analysis.

4. plot: all chore associated with plotting data

The first four groups must be provided completely by any template set. Although if a template is missing from a template set, DEMETER will fall back to using the template for that chore found in the ifeffit set.

---

 

The template method

When the template method is called, a number of variables are set for use in the template. These variables are set appropriately for the contect in which the template method is called. You can see one example of this in the example at the beginning of this section. The $D variable represents the Data object relevant to the context in which that template is evaluated. Some more examples will be seen below.

Here is the complete list of these special variables.

$S

This is the “self” object, i.e. the object that called the template method.

$D

The is the Data object of the calling object. When a Data object is the caller, $S and $D are the same thing. For a Path object as $S, $D is its associated Data object.

$P

This is the Plot object.

$C

This is the Config object.

$F

This is the currently active Fit object.

$DS

This is the currently active Data standard.

$T

This is the currently active Feff (i.e. theory) object.

$PT

This is the currently active Path object.

The syntax of the template method is relatively simple. The method takes two arguments, the first identifying the template group, the second identifying the chore that the template performs. Identifying the specific template also requires the template set, which is an attribute of the Mode group.

my $string = $self->template("process", "fft");
$self->dispose($string);

In this example, the command to make a forward Fourier transform using the current template set is generated by evaluating the appropriate template. The text of this command is then passed to the dispose method.

Some templates require data that is not normally available from any attribute of any object. There are two ways of addressing that situation. One is to store an arbitrarily named attribute in the Config object. This is done like so:

$config->set(conv_type  => $args{type},
             conv_width => $args{width},
             conv_which => $args{which},
            );
my $string = $self->template("process", "convolve");
$self->dispose($string);

Here three scalars related to data covolution are set in the Config object. Here is how those scalars are used:

{
  $x = ($C->get("conv_which") eq 'xmu') ? 'energy' : 'k';
  $type = 'gconvolve';
  ($type = 'lconvolve') if (lc($C->get("conv_type")) =~ m{\Al});
  q{}
}
##|
##| convolution {$D->group}
set {$D->group}.{$C->get("conv_which")} = {$type}({$D->group}.{$x}, {$D->group}.{$C->get("conv_which")}, {$C->get("conv_width")})
##|

Note that this example uses the $C special template variable to access the Config object and the Config object's get method to oibjtain the values of these arbitrarily named scalars.

The other approach to passing arbitrary data to a template is to provide a hash reference as the third argument of the template method.

$command = $self->template("plot", "marker", { x => $xx, 'y'=> $y });

These user-defined parameters are then accessed by name in the template. This example also shows the use of the $P special variable to make reference to the Plot object.

plot_marker({$x}, {$y}, marker={$P->markertype}, color={$P->markercolor})