.. Athena document is copyright 2016 Bruce Ravel and released under The Creative Commons Attribution-ShareAlike License http://creativecommons.org/licenses/by-sa/3.0/ Command disposal and command templates ====================================== :demeter:`demeter` uses a highly abstract system for communicating with :demeter:`feff`, :demeter:`ifeffit`, and the plotting backend. The `Text::Template `__ module is used along with a large number of small, external template files. The idea behind `Text::Template `__ is quite simple. Perl code is interspersed with normal text. A pass over the text is made and the perl code is evaluated. In this way, a bit of text is made that is specifically relevant to a particular situation. As an example, here is the template that :demeter:`demeter` uses to construct the :demeter:`ifeffit` command used to normalize the |mu| (E) data associated with a Data object. :: { # -*- ifm -*- # pre-edge template # {$D->group} returns the ifeffit group name # {$D->parameter} returns the value of that parameter } pre_edge("{$D->group}.energy+{$D->bkg_eshift}", {$D->group}.xmu, e0 = {$D->bkg_e0}, pre1 = {$D->bkg_pre1}, pre2 = {$D->bkg_pre2}, norm_order = {$D->bkg_nnorm}, norm1 = {$D->bkg_nor1}, norm2 = {$D->bkg_nor2}) In this example, ``$D`` represents the Data object. Thus :demeter:`ifeffit`'s ``pre-edge`` command is filled in with attribute values appropriate to the current Data object. The text generated from the processing of the template is then disposed of via one or more avenues, one of which may include sending the text to :demeter:`ifeffit` for processing. This section explains all of the disposal and templating options available to a :demeter:`demeter` program. Command disposal ---------------- Once commands have been processed using the templating system, there are seevral avenues for disposing of those commands. The most obvious avenue is to send the commands to :demeter:`ifeffit` so that the actual data can be processed. However, several other options are possible. The ``dispose`` method is a method of the base class and inherited by all objects. It is used all throughout :demeter:`demeter` to get things done and is available for explicit use in :demeter:`demeter` programs. The syntax is quite simple: .. code-block:: perl $any_object -> dispose($text); $any_object -> dispose($text, 'plotting'); The first line is used for any data processing command. The second form, with the second argument to the method, is used to identify command text which is explicitly used to make plots. When using the PGPLOT plotting back end, there is not actually a significant difference between plotting and non-plotting commands since :demeter:`ifeffit` is used to send command to PGPLOT. However, when using other plotting backends, it is essential that plotting commands be apprpriately flagged. Plotting backends are discussed in more detail in `the user interface chapter <../ui.html>`__. The ``$text`` given as an argument to the ``dispose`` method typically comes from the evaluation of a template, but can be any text generated in any fashion. Thus, it is a completely generic way for a program to communicate with :demeter:`demeter`'s backends. The targets of the ``dispose`` method are set using the ``set_mode`` method, another method of the base class which is inherited by all objects. The syntax of ``set_mode`` is consistent with other methods in :demeter:`demeter`: .. code-block:: perl $any_object -> set_mode(screen=>1, backend=>1); Any command can be sent to multiple targets. The disposal targets which can be set using ``set_mode`` are: ``backend`` When true, commands will be sent to :demeter:`ifeffit` or :demeter:`larch`. It is often useful to turn this disposal target off when debugging :demeter:`demeter` programs. ``ifeffit`` and ``larch`` are aliases for ``backend``. ``screen`` When true, commands will be sent to standard output (usually the screen). Turning this disposal target on is often useful when debugging :demeter:`demeter` programs. ``plotscreen`` When true, plotting commands will be sent to standard output (usually the screen). Turning this disposal target on is often useful when debugging :demeter:`demeter` programs. ``repscreen`` When true, the reprocessed commands (`discussed below <#reprocessed-commands>`_) will be sent to standard output (usually the screen). ``file`` When set to a string value, that string will be interpretted as a file name to be opened for writing and the commands will then be written to that file. To append text to a file, the ``file`` mode string should begin with the ``>`` character. ``plotfile`` When set to a string value, that string will be interpretted as a file name to be opened for writing and the plotting commands will then be written to that file. To append text to a file, the ``file`` mode string should begin with the ``>`` character. ``repfile`` When set to a string value, that string will be interpretted as a file name to be opened for writing and the reprocessed commands (`discussed below <#reprocessed-commands>`_) will then be written to that file. To append text to a file, the ``file`` mode string should begin with the ``>`` character. ``buffer`` When set to an array reference, commands will be pushed onto that array. When set to a scalar reference, commands will be concatinated to the end of the strings held by the scalar. ``plotbuffer`` When set to an array reference, plotting commands will be pushed onto that array. When set to a scalar reference, plotting commands will be concatinated to the end of the strings held by the scalar. ``callback`` When set to a code reference, the text of the command will be sent to that code reference as the sole argument. This is useful for user interfaces that want to post-process the commands. For example, this disposal mode is used by :demeter:`artemis` to display colorized text in its command buffer. ``plotcallback`` When set to a code reference, the text of the plotting command will be sent to that code reference as the sole argument. This is useful for user interfaces that want to post-process the commands. For example, this disposal mode is used by :demeter:`artemis` to display colorized text in its plotting buffer. ``feedback`` When set to a code reference, the text of :demeter:`ifeffit`'s (or :demeter:`larch`'s) response to commands will be sent to that code reference as the sole argument. This is useful for user interfaces that want to post-process the commands. For example, this disposal mode is used by :demeter:`artemis` to display colorized text in its command buffer. Reprocessed commands -------------------- :demeter:`demeter` tries to use :demeter:`ifeffit` as efficiently as possibly. On one hand, :demeter:`ifeffit` the one of the things that makes :demeter:`demeter` go and so is indispensible. On the other hand, the business of communicating between perl code and the :demeter:`ifeffit` library is (`with one exception <../feff/pathfinder.html>`__) always the slowest thing that :demeter:`demeter` does. One of the optimizations implemented by :demeter:`demeter` is the reprocessing of commands targeted for disposal to :demeter:`ifeffit`. Command strings in :demeter:`ifeffit` can be quite long |nd| up to 2048 characters as it is normally compiled. A command that is split over multiple lines, as the example at the beginning of this section is, will be processed much faster if :demeter:`demeter` pre-processes the command to remove unnecessary line breaks. Basically this means that everything between parentheses will be sent to :demeter:`ifeffit` as a single string. This is accomplished within the ``dispose`` method via the application of a few regular expressions. The reprocessed string is then sent to :demeter:`ifeffit`. As a small example of how reprocessing works, this human-friendly command: .. code-block:: text pre_edge("data0.energy+0", data0.xmu, e0 = -9999999, pre1 = -150, pre2 = -30, norm_order = 3, norm1 = 150, norm2 = 1800) will be reprocessed into this one-line command before being shuffled off to :demeter:`ifeffit`. .. code-block:: text pre_edge("data0.energy+0", data0.xmu, e0=-9999999, pre1=-150, pre2=-30, norm_order=3, norm1=150, norm2=1800) which, when summed over dozens or hundreds of :demeter:`ifeffit` commands, results in a substantial performance improvement. The ``repscreen`` and ``repfile`` disposal targets are provided to debug the behavior of this optimization. Reprocessing is quite well tested. However, if you suspect that reprocessing is damaging the commands sent to :demeter:`ifeffit`, use one of those disposal channels to see the text that is actually being sent. .. note:: :demeter:`larch` does not use reprocessed commands. When :demeter:`larch` is the backend, the reprocessing step is skipped. Command templates ----------------- :demeter:`demeter` ships with **a lot** of templates. Each template encapsulates a small bit of functionality and :demeter:`demeter` does many things. The templates are organized into :quoted:`template sets`, which are written for specific backend targets, and :quoted:`template groups` which, are groups of templates which serve related functions. All template sets must have a complete representation of template groups to be fully functional. The templates are found in :file:`lib/templates/` directory underneath the installation location of the :demeter:`demeter` package. One of the reasons for explaining the templating system in this level of detail is to underscore that it is quite possible to add new template sets. By following the model of the existing template sets, new output types can be created for :demeter:`demeter`. Indeed, when finally makes its appearence, it should be relatively simple to extend :demeter:`demeter` to use it simply by creating an apprporiate template set. Choosing between template sets is one of the topics of `the next section `__. Template sets ~~~~~~~~~~~~~ :demeter:`demeter` currently ships with five different sets in the data processing, XANES analysis, EXAFS fitting, and plugin categories: #. ``ifeffit``, templates which write the syntax of :demeter:`ifeffit` in a compact form #. ``larch``, templates which write the syntax of :demeter:`larch` #. ``iff_columns``, templates which write the syntax of :demeter:`ifeffit` is a more human-readable form #. ``feffit``, templates which write the syntax of input files for the old :demeter:`feffit` program. *incomplete* #. ``demeter``, templates which write out perl syntax using :demeter:`demeter`. *incomplete* The ``demeter`` category might seem a bit strange. Its purpose is to allow :demeter:`demeter` programs to write :demeter:`demeter` programs. The intent is to allow a GUI to export a file containing a :demeter:`demeter` program that can be used to make a fit using the same fitting model that was created using the GUI. The possibility of having these different output targets is the main reason for using a templating system. Having command creation contained in these small template files separate from the code may seem like an unnecessary layer of abstraction and misdirection, but it offers :demeter:`demeter` a lot of flexibility and power. This is even more evident for the plotting backends. :demeter:`demeter` currently ships with four sets in the plotting category. More information about plotting backends can be found in `the user interface chapter <../ui.html>`__. #. ``gnuplot``, templates which write :program:`Gnuplot` plotting scripts. This is the default plotting backend. Using :program:`Gnuplot` involves writing lots of temporary files which contain the data to be plotted. It also requires that :program:`Gnuplot` be installed on your computer, which is something that you have to do separate from the installation of :demeter:`demeter`. #. ``pgplot``, templates which write the syntax of :demeter:`ifeffit` plotting commands, which talk directly to :program:`PGPLOT`. #. ``singlefile``, this set of templates is used to export the data to be plotted to a single column file. The main use of this is in a GUI to exprt a file that can be used to replicate an interesting plot |nd| with offsets, energy shifts, and scaling factors |nd| in an external plotting program. #. ``demeter``, templates which write out perl syntax using :demeter:`demeter`. *incomplete* In the future, there might be more plotting backends. A backend for :demeter:`larch`'s matplotlib interface would make sense. :demeter:`demeter` currently ships with two different sets in the :demeter:`feff` input template category, one for :demeter:`feff6` and one for :demeter:`feff8`. (Actually the :demeter:`feff8` set has not yet been written at the time this document is being written. In fact, :demeter:`demeter`'s :demeter:`feff8` interface has not yet been started. Support for :demeter:`feff8` will use `feff85exafs `_. There is only one set in the :demeter:`atoms` input template category. It seems unlikely that other sets will actually be required. Template groups ~~~~~~~~~~~~~~~ Template groups define related chores. These chores are #. ``process``: all data processing chores that do not involve fitting |chi| (k) data or doing any other sort of data analysis. #. ``fit``: all chores associated with fitting |chi| (k) data. #. ``analysis``: XANES analysis chores. This includes things like linear combination fitting. principle component analysis, or log-ratio/phase-difference analysis. #. ``plot``: all chores associated with plotting data #. ``report``: generation of textual reports #. ``plugin``: data processing chores performed by filetype or other plugins #. ``feff``: :demeter:`feff` input templates #. ``atoms``: :demeter:`atoms` input templates The first four groups must be provided completely by any template set. Although if a template is missing from a template set, :demeter:`demeter` will fall back to using the template for that chore found in the ``ifeffit`` set. The template method ~~~~~~~~~~~~~~~~~~~ When the ``template`` method is called, a number of variables are set for use in the template. These variables are set appropriately for the contect in which the ``template`` method is called. You can see one example of this in the example at the beginning of this section. The ``$D`` variable represents the Data object relevant to the context in which that template is evaluated. Some more examples will be seen below. Here is the complete list of these special variables. ``$S`` This is the :quoted:`self` object, i.e. the object that called the ``template`` method. ``$D`` The is the Data object of the calling object. When a Data object is the caller, ``$S`` and ``$D`` are the same thing. For a Path object as ``$S``, ``$D`` is its associated Data object. ``$P`` This is the Plot object. ``$C`` This is the Config object. ``$F`` This is the currently active Fit object. ``$DS`` This is the currently active Data standard. ``$T`` This is the currently active Feff (i.e. theory) object. ``$PT`` This is the currently active Path object. The syntax of the ``template`` method is relatively simple. The method takes two arguments, the first identifying the template group, the second identifying the chore that the template performs. Identifying the specific template also requires the template set, which is an attribute of the `Mode group `__. .. code-block:: perl my $string = $self->template("process", "fft"); $self->dispose($string); In this example, the command to make a forward Fourier transform using the current template set is generated by evaluating the appropriate template. The text of this command is then passed to the ``dispose`` method. Some templates require data that is not normally available from any attribute of any object. There are two ways of addressing that situation. One is to store an arbitrarily named attribute in the Config object. This is done like so: .. code-block:: perl $config->set(conv_type => $args{type}, conv_width => $args{width}, conv_which => $args{which}, ); my $string = $self->template("process", "convolve"); $self->dispose($string); Here three scalars related to data covolution are set in the Config object. Here is how those scalars are used: :: { $x = ($C->get("conv_which") eq 'xmu') ? 'energy' : 'k'; $type = 'gconvolve'; ($type = 'lconvolve') if (lc($C->get("conv_type")) =~ m{\Al}); q{} } ##| ##| convolution {$D->group} set {$D->group}.{$C->get("conv_which")} = {$type}({$D->group}.{$x}, {$D->group}.{$C->get("conv_which")}, {$C->get("conv_width")}) ##| Note that this example uses the ``$C`` special template variable to access the Config object and the Config object's ``get`` method to oibjtain the values of these arbitrarily named scalars. The other approach to passing arbitrary data to a template is to provide a hash reference as the third argument of the ``template`` method. .. code-block:: perl $command = $self->template("plot", "marker", { x => $xx, 'y'=> $y }); These user-defined parameters are then accessed by name in the template. This example also shows the use of the ``$P`` special variable to make reference to the Plot object. :: plot_marker({$x}, {$y}, marker={$P->markertype}, color={$P->markercolor})