Local variables in easyconfig files¶
This page discusses the use of local variables in easyconfig files.
For more general information on writing easyconfig files, please see Writing easyconfig files: the basics.
Motivation & context¶
When composing easyconfig files, it can sometimes make sense to use one or more local variables to define easyconfig parameters in a cleaner way, for example to avoid copy-pasting values that are used multiple times, or to avoid too long lines (longer than the maximum width of 120 characters as specified in Code style). A local variable within the context of easyconfig files is any variable that does not correspond to a known easyconfig parameter (either general of easyblock-specific).
One thing that is easily overlooked is the importance of how these local variables are named: you should try and avoid that the names of local variables will match with the name of an easyconfig parameter that may get introduced in future EasyBuild versions. If that would happen, the semantics of the easyconfig file will change which may either result in a broken installation or perhaps a different installation than was intended (which could be worse than a broken one).
Avoiding that names of local variables are close to the name of (future) easyconfig parameters is also important, because of the "typo detection" feature that EasyBuild has: if the name of a local variable is too close to a known easyconfig parameter, EasyBuild will assume you have made a mistake, and will report a typo error, for example:
ERROR: Failed to process easyconfig example.eb:
You may have some typos in your easyconfig file: configopt -> configopts
Changes in EasyBuild v4.0 w.r.t. local variables in easyconfig files¶
Some changes were made in EasyBuild v4.0 to discourage the use of poorly named local variables:
- a Recommended naming scheme for local variables in easyconfig files was defined;
- a mechanism was implemented to detect the use of local variables in easyconfig files, which will print a Warning for local variables that do not follow the recommended naming scheme
- EasyBuild can be configured to report an error for local variables
that do not follow the recommended naming scheme; see
Specifying what should be done when non-confirming local variables are found via
--local-var-naming-check
- using
eb --fix-deprecated-easyconfigs
, the names of local variables can be changed automatically to match the recommended naming scheme; see Renaming local variables to match the recommended naming scheme usingeb --fix-deprecated-easyconfigs
Recommended naming scheme for local variables in easyconfig files¶
To ensure that local variables used in easyconfig files do not clash with easyconfig parameters that get added in future EasyBuild versions, we have defined a recommended naming scheme for local variables.
Names of local variables should either:
- start with
local_
, to make it explicit that it is a local variable (examples:local_example
,local_var
) - consist of a single letter (examples:
f
,l
,x
); this is typically only done within the context of a list comprehension - start with a single underscore (
_
), which matches the common convention in Python code that these variables are only for "internal use" (examples:_example
,_local_var
)
Note
A check was added in EasyBuild v4.0 to ensure that the names of known easyconfig parameters do not conform to any of these rules, to ensure that local variables can always be discriminated from known easyconfig parameters.
Warning for local variables that do not follow the recommended naming scheme¶
By default, EasyBuild will produce a clear warning when one or more local variables are used that do not conform to the Recommended naming scheme for local variables in easyconfig files.
For example, when using an easyconfig file that includes a local
variable named var
:
$ eb example.eb
...
WARNING: Use of 1 unknown easyconfig parameters detected in example.eb: var
If these are just local variables please rename them to start with 'local_', or try using --fix-deprecated-easyconfigs to do this automatically.
To get rid of this warning, you can either:
- rename the local variable, either manually to something like
local_var
or usingeb --fix-deprecated-easyconfigs example.eb
(see also Renaming local variables to match the recommended naming scheme usingeb --fix-deprecated-easyconfigs
) - configure EasyBuild to only log the warning (not print it), via
--local-var-naming-check=log
(see also Specifying what should be done when non-confirming local variables are found via--local-var-naming-check
; note that silencing the printed warning is not recommended, see the motivation in Motivation & context
Specifying what should be done when non-confirming local variables are found via --local-var-naming-check
¶
Using the --local-var-naming-check
configuration option, you can
specify what should be done when one or more local variables are found
that do not follow the
Recommended naming scheme for local variables in easyconfig files:
--local-var-naming-check=error
: fail with an error if any easyconfig file that was parsed includes local variables that do not follow the recommended naming scheme;--local-var-naming-check=log
: only log a warning (but do not print it) if any easyconfig file that was parsed includes local variables that do not follow the recommended naming scheme;--local-var-naming-check=warn
[default]: print a warning if any easyconfig file that was parsed includes local variables that do not follow the recommended naming scheme;
The default is set such that non-confirming local variables are only reported through a printed warning, but do not result in cancelling the installation (since they're usually not actually problematic).
Renaming local variables to match the recommended naming scheme using eb --fix-deprecated-easyconfigs
¶
To fix one or more easyconfig files that includes local variables that
do not follow the
Recommended naming scheme for local variables in easyconfig files, eb --fix-deprecated-easyconfigs
can be used.
For example:
eb --fix-deprecated-easyconfigs bzip2.eb zlib.eb
== temporary log file in case of crash /tmp/eb-Z7r_KJ/easybuild-dHtPY4.log
* [1/2] fixing /tmp/example/bzip2.eb... FIXED!
(changes made in place, original copied to /tmp/example/bzip2.eb.orig_20190815180106_53972)
* [2/2] fixing /tmp/example/zlib.eb... FIXED!
(changes made in place, original copied to /tmp/example/zlib.eb.orig_20190815180106_53972)
All done! Fixed 2 easyconfigs (out of 2 found).
== Temporary log file(s) /tmp/eb-Z7r_KJ/easybuild-dHtPY4.log* have been removed.
== Temporary directory /tmp/eb-Z7r_KJ has been removed.
There are a couple of caveats here though...
While --fix-deprecated-easyconfigs
can be very useful, it's certainly
not perfect since all it does is simple search and replace of the names
of non-conforming local variables (as whole words) to prefix them with
local_
.
This means that it may make some unintended changes, so make sure to always double check which changes were made!
In addition, it sometimes make more sense to simply eliminate the
local variable rather than renaming it, for example when it wasn't
really needed at all: maybe it was only actually used once, or maybe
using a template like %(pyver)s
or (pyshortver)s
(see also
Available template values for easyconfigs) renders it
obsolete.