DOKK / manpages / debian 12 / esorex / esorex.1.en
ESOREX(1) User Commands ESOREX(1)

esorex - ESO Recipe Execution Tool

esorex [esorex-options] recipe [recipe-options] sof

EsoRex is the ESO Recipe Execution Tool. It can list, configure and execute CPL-based recipes from the command line.

One of the features provided by the CPL is the ability to create data-reduction algorithms that run as plugins (dynamic libraries). These are called recipes and are one of the main aspects of the CPL data-reduction development environment.

As these recipes are dynamic libraries, it is not possible to run them directly from the command line. However, ESO provides several tools to do this, thus saving recipe developers the need to write such an application themselves. One of these is GASGANO (a GUI-based tool) and the other is EsoRex (which runs from the command line) and is described here.

Display this help and exit. If a recipe name is also given, then help will be given for it as well. [TRUE]
Display version information and exit. [FALSE]
When TRUE, all the input files must exist and be readable before calling the recipe. This option may also be set using the environment variable ESOREX_CHECK_SOF_EXIST. [FALSE]
Configuration file to be used for EsoRex. This option may also be set using the environment variable ESOREX_CONFIG. []
Creates a configuration file for Esorex. If set to TRUE a config file 'esorex.rc' is created in the '.esorex' directory in $HOME of the user.If a filename is specified, a config file will be created accordingly. If a recipe is specified in the command line, then the configuration file will be created for the recipe instead (called 'recipename.rc') Note that an existing file will be overwritten, but a backup file will be copied to 'filename.rc.bak' in the same directory. If the filename ends with extension .json then a machine-readable JSON format will be used. This option may also be set using the environment variable ESOREX_CREATE_CONFIG. [FALSE]
The directory in which a symbolic link to each of the product files should be written. The enable/disable switch to control whether the link is actually made is the '--suppress-link' option. This option may also be set using the environment variable ESOREX_LINK_DIR. [/tmp]
Directory where to place the logfile. This option may also be set using the environment variable ESOREX_LOG_DIR. [.]
Filename of logfile. This option may also be set using the environment variable ESOREX_LOG_FILE. [esorex.log]
Controls the severity level of messages that will be printed to the logfile. This option may also be set using the environment variable ESOREX_LOG_LEVEL. <debug | info | warning | error | off> [info]
Disables the computation of the MD5 data hash for FITS product files. This option may also be set using the environment variable ESOREX_NO_DATAMD5. [FALSE]
Disables the computation of the standard FITS product checksums. This option may also be set using the environment variable ESOREX_NO_CHECKSUM. [FALSE]
Display a manual page for the specified recipe, and then exit. Note that this option only applies to recipes, and that it does nothing for esorex by itself. See also the '--help' option. This option may also be set using the environment variable ESOREX_MAN_PAGE. [FALSE]
Report on memory status at completion of recipe execution. This option may also be set using the environment variable ESOREX_MEM_CHECK. [FALSE]
Controls the severity level of messages that will be printed to the terminal. This option may also be set using the environment variable ESOREX_MSG_LEVEL. <debug | info | warning | error | off> [info]
The directory where the product files should be finally moved to (all products are first created in the current dir). This option may also be set using the environment variable ESOREX_OUTPUT_DIR. [.]
Prefix applied to any output file. For example, specifying 'pre' would translate 'filename.fits' to 'pre_0000.fits'. See also the '--suppress-prefix' option. This option may also be set using the environment variable ESOREX_OUTPUT_PREFIX. [out]
When TRUE, any existing product files in the specified output directory will be set to read-only, for user, group and other. If FALSE, then EsoRex will use the default permissions for that account/directory. destroy any pre-existing files. This option exists for the Paranal operations environment. This option can additionally be used to prevent EsoRex from overwriting pre-existing files. This option may also be set using the environment variable ESOREX_OUTPUT_READONLY. [FALSE]
Configuration file for creation of PAF files. This option may also be set using the environment variable ESOREX_PAF_CONFIG. []
List the input parameters and their current settings (whether from the command line or a configuration file) for the esorex application. Parameters are labelled using the parameter's alias. If a recipe is also specified, then the list of its parameters will also be generated in the same way. This option may also be set using the environment variable ESOREX_PARAMS. [FALSE]
Output file which contains the FITS files created by the recipe. If the filename ends with extension .json then a machine-readable JSON formatwill be used. This option may also be set using the environment variable ESOREX_PRODUCTS_SOF. []
Display a list of all available recipes (that are available in the directory tree specified with '--recipe-dir'). This option may also be set using the environment variable ESOREX_RECIPES. [FALSE]
Configuration file for any selected recipe. This option may also be set using the environment variable ESOREX_RECIPE_CONFIG. []
Directory containing recipe libraries. Note that esorex will recursively search not only the specified directory, but all sub-directories below it as well. Multiple directory heads may be specified, by separating the starting paths with colons (:). This option may also be set using the environment variable ESOREX_PLUGIN_DIR. [.]
When TRUE recipe parameters which are declared as hidden by the recipe are shown in the output of help and man page options. [FALSE]
When TRUE, no symbolic link is created to the output product. However, if FALSE, then a symbolic link is created in the directory specified with the option '--link-dir' for each product that is created by the recipe. This option may also be set using the environment variable ESOREX_SUPPRESS_LINK. [TRUE]
When TRUE, the original name of the output product, as produced by the recipe, is maintained. If FALSE, then the name of the output file is changed to the "prefix_number" format. The prefix can be altered using the '--output-prefix' option. This option may also be set using the environment variable ESOREX_SUPPRESS_PREFIX. [FALSE]
Measure and show the recipe's execution time. This option may also be set using the environment variable ESOREX_TIME. [FALSE]
When TRUE, the plugin is unloaded after execution. If FALSE, the plugin is not unloaded after processing, so that a software like, e.g. valgrind, can be used for debugging the executed recipe. This option may also be set using the environment variable ESOREX_UNLOAD_PLUGIN. [TRUE]

All options can be set as environment parameters as well. See the previous paragraph for details.

Default configuration files

The configuration file contains the EsoRex options, less the `--´ switch, but prefixed with `esorex.caller.´. Blank lines are ignored and lines beginning with `#´ are treated as comments.

Here is an example configuration file.


# Example EsoRex configuration file
#
esorex.caller.recipe-dir=/home/username/EsoRex/Plugins
esorex.caller.log-dir=.
esorex.caller.log-file=esorex.log
esorex.caller.log-file=esorex.log
esorex.caller.output-dir=.
esorex.caller.output-prefix=out_

A sof file contains a list of the input data. This data is specified in an sof file (which is just a text file), where each input file is specified with its associated classification and category. The format of each line in the sof file is as follows:


full-path-to-file classification

Optionally, a third column may be provided. Permitted values are either RAW or CALIB. This is for when a recipe does not identify the type of input file, but as all ESO recipes are required to do so, this column is typically not needed.

An example sof file, for the mythological "ZIMOS" instrument, might look like this:


/data/mos/ZIMOS.03-12-26T01:05:06.fits MOS_SCIENCE
/data/mos/ZIMOS.03-12-26T01:26:00.fits MOS_SCIENCE
/data/mos/ZIMOS.03-12-26T01:47:04.fits MOS_SCIENCE
/data/cal/master_bias4.fits MASTER_BIAS
/data/cal/grs_LR_red.3.tfits GRISM_TABLE
/data/gasgano/extract_table2.fits EXTRACT_TABLE
/data/cal/badpixel.3.tfits CCD_TABLE

For an concrete example for a specific instrument, check the documentation for that instrument.

When a recipe is used with the --suppress-prefix option, and the --output-dir is set to the current working directory, then the first execution of a recipe will work correctly, but subsequent executions may fail. This is due to output products being given `read-only´ permission (to avoid the potential inadvertent loss of products). The recipe itself is unable to modify the permissions, and thus it fails when attempting to create the file. The solution (other than using a different output directory or prefixes) is to change the permission of these output files or delete them prior to any subsequent execution of that recipe.

This problem is less likely to occur in EsoRex v2+, due to the replacement of the --output-overwrite option with the --output-readonly (which is disabled by default). However, a determined user can still reach this situation, in which case the non-readable products must have their permissions changed, as described above.

When creating configuration files, if the the recipe is provided on the command line, then EsoRex will generate the configuration file for this recipe. If no recipe name is given, then EsoRex will generate a configuration file for EsoRex itself. All configuration files are written in the $HOME/.esorex/ directory.

It is possible to get EsoRex to check for memory leaks in the recipe that it is running, by enabling the --mem-check option. Then, at the conclusion of the recipe execution, and after memory deallocation, a list of all remaining allocated memory will be printed to screen. If there are no memory leaks, then no addition output is displayed.

Report bugs to <cpl-help@eso.org>.

Copyright 2003-2018, European Southern Observatory.

May 2022 esorex