Partial installations¶
Several ways of performing partial installations are supported. These may be useful when debugging a particular issue with the installation procedure being performed by EasyBuild, updating existing software installations or module files, or after changing the EasyBuild configuration (e.g., switching to module files in Lua syntax or a different module naming scheme).
Stopping the installation procedure after a step using -s/--stop¶
To stop the installation procedure after a specific step in the installation procedure, the -s/--stop
command line option can be used; the name of the step must be supplied as an argument.
The following step names are recognized (listed in execution order): fetch, ready, source, patch,
prepare, configure, build, test, install, extensions, package, postproc,
sanitycheck, cleanup, module, testcases.
Example usage:
$ eb GCC-11.3.0.eb --stop configure
== temporary log file in case of crash /tmp/eb-X2Z0b7/easybuild-mGxmNb.log
== processing EasyBuild easyconfig /home/example/GCC-11.3.0.eb
== building and installing GCC/11.3.0...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== COMPLETED: Installation STOPPED successfully
== Results of the build can be found in the log file /dev/shm/example/GCC/11.3.0/dummy-dummy/easybuild/easybuild-GCC-11.3.0-20230613.091644.log
== Build succeeded for 1 out of 1
== temporary log file(s) /tmp/eb-X2Z0b7/easybuild-mGxmNb.log* have been removed.
== temporary directory /tmp/eb-X2Z0b7 has been removed.
Fetching sources with --fetch¶
It may be useful to be able to batch-download sources on a machine where no modules tool is installed. The
--fetch option, which is equivalent with --stop fetch --ignore-osdeps, addresses this requirement.
Example usage:
$ eb GCCcore-6.2.0.eb --fetch
== Temporary log file in case of crash /tmp/eb-23iu2fpm/easybuild-ss0wcidi.log
== processing EasyBuild easyconfig /home/example/GCCcore-11.3.0.eb
== building and installing GCCcore/11.3.0...
== fetching files...
== COMPLETED: Installation STOPPED successfully (took 1 secs)
== Results of the build can be found in the log file(s) /tmp/example/easybuild/build/GCCcore/11.3.0/system-system/easybuild/easybuild-GCCcore-11.3.0-20230613.090338.log
== Build succeeded for 1 out of 1
== Temporary log file(s) /tmp/example/eb-23iu2fpm/easybuild-ss0wcidi.log* have been removed.
== Temporary directory /tmp/example/eb-23iu2fpm has been removed.
Note
--fetch can be used in conjunction with the --robot and --robot-path options to download sources of
the whole dependency tree of an easyconfig (see Enabling dependency resolution, --robot / -r and --robot-paths).
Note
Sources will be downloaded in the default location (see Source path (--sourcepath)),
unless EasyBuild is configured via the --sourcepath option.
Installing additional extensions using -k/-skip¶
For software applications that may include Extensions, it is often required to install one or more additional
extensions without having to reinstall the software package (and all extensions) from scratch.
For this purpose, the -k/--skip command line option is available.
To actually skip an existing software installation and all installed extensions, a corresponding module must be
available already; if not, the installation procedure will be performed from scratch.
To trigger the installation of missing extensions, --rebuild ( or --force, see Forced reinstallation) must be used as well; without it, the installation
procedure will be skipped as a whole (since the module is already available).
When --skip is combined with --rebuild, EasyBuild will:
i) ensure that all (extension) sources are available (and try to fetch them if needed); ii) prepare the build environment; iii) check which extensions have not been installed yet; iv) install the missing extensions; v) run the sanity check (which includes checking that all extensions are available) vi) regenerate the module file (since it contains a list of installed extensions)
Example usage:
$ eb Python-3.10.4-GCCcore-11.3.0.eb --skip
...
== Python/3.10.4-GCCcore-11.3.0 is already installed (module found), skipping
== No easyconfigs left to be built.
== Build succeeded for 0 out of 0
$ eb Python-3.10.4-GCCcore-11.3.0.eb --skip --rebuild
...
== building and installing Python/3.10.4-GCCcore-11.3.0...
...
== configuring [skipped]
== building [skipped]
== testing [skipped]
== installing [skipped]
== taking care of extensions...
...
== sanity checking...
== cleaning up...
== creating module...
== COMPLETED: Installation ended successfully
Note
Upgrading of extensions to a newer version does not work (yet) using --skip, because the way in which extensions
are checked for availability, i.e. the extensions filter, is (usually) version-agnogstic.
Note
The 'skipsteps' easyconfig parameter has a different purpose, i.e. to specify which installation steps should
always be skipped when the installation of a particular software package is performed, no matter whether the
software or corresponding module is already available or not.
Note
When --skip is used, a backup is created for all existing module files that are regenerated.
To disable backing up of module files, use --disable-backup-modules (see also Backing up of existing modules (--backup-modules)).
Only (re)generating (additional) module files using --module-only¶
Since EasyBuild v2.1, it is possible to only (re)generate the module file that matches the specifications in the
easyconfig file, using --module-only. Depending on the use case, additional options should be supplied.
Usually --rebuild is also required, either to ignore the existing module file (if the module is available under the
same name as the one being (re)generated), or to skip the sanity check that verifies the software installation (if no
software installation is available).
Combining --module-only with --installpath-modules is also a common use case, to generate the module file in
a (test) location other than the software installation prefix (see Software and modules install path (--installpath, --installpath-software, --installpath-modules)).
Note
Although --module-only was already supported in EasyBuild v2.1.0, we strongly recommend to use EasyBuild
v2.1.1 or a more recent version, because of some critical bug fixes with respect to --module-only.
Use cases:
Note
When --module-only is used, a backup is created for all existing module files that are regenerated.
To disable backing up of module files, use --disable-backup-modules (see also Backing up of existing modules (--backup-modules)).
Only (re)generating (existing) module file¶
To only generate a module file (i.e., skip actually building and installing the software), or to regenerate an
existing module file, --module-only can be used.
In the former case, enabling --rebuild is required because the sanity check step that verifies whether the
installation produced the expected files and/or directories is not skipped unless forced.
In the latter case, --rebuild must be used to make EasyBuild ignore that the module is already available
according to the modules tool.
Example usage:
-
only generate module file:
$ module avail GCC ---------------------------- /home/example/.local/modules/all ---------------------------- GCC/11.2.0 $ eb GCC-11.3.0.eb --module-only --rebuild ... == building and installing GCC/11.3.0... == fetching files [skipped] ... == configuring [skipped] == building [skipped] == testing [skipped] == installing [skipped] ... == sanity checking [skipped] == cleaning up [skipped] == creating module... == COMPLETED: Installation ended successfully ... $ module avail GCC ---------------------------- /home/example/.local/modules/all ---------------------------- GCC/11.2.0 GCC/11.3.0 -
regenerate existing module file:
$ module avail GCC/11.3.0 ---------------------------- /home/example/.local/modules/all ---------------------------- GCC/11.3.0 $ ls -l /home/example/.local/modules/all/GCC/11.3.0 -rw-rw-r-- 1 example example 1002 Jun 13 08:35 /home/example/.local/modules/all/GCC/11.3.0 $ eb GCC-11.3.0.eb --module-only --rebuild ... == building and installing GCC/11.3.0... ... == sanity checking [skipped] == creating module... == COMPLETED: Installation ended successfully ... $ ls -l /home/example/.local/modules/all/GCC/11.3.0 -rw-rw-r-- 1 example example 1064 Jun 13 10:54 /home/example/.local/modules/all/GCC/11.3.0
Generating additional module files¶
Generating an additional module file, next to the one(s) already available, is also supported. This can achieved by
combining --module-only with additional configuration options that apply to the module generation.
Examples:
-
to generate a module file in Lua syntax, next to an already existing module file in Tcl syntax,
--module-only --module-syntax=Luacan be used:$ module avail GCC/11.3.0 ---------------------------- /home/example/.local/modules/all ---------------------------- GCC/11.3.0 $ ls -l /home/example/.local/modules/all/GCC/11.3.0* -rw-rw-r-- 1 example example 1064 Jun 13 08:54 /home/example/.local/modules/all/GCC/11.3.0 $ eb GCC-11.3.0.eb --modules-tool=Lmod --module-only --module-syntax=Lua --rebuild ... == building and installing GCC/11.3.0... ... == sanity checking [skipped] == creating module... == COMPLETED: Installation ended successfully ... $ ls -l /home/example/.local/modules/all/GCC/11.3.0* -rw-rw-r-- 1 example example 1064 Jun 13 08:54 /home/example/.local/modules/all/GCC/11.3.0 -rw-rw-r-- 1 example example 1249 Jun 13 08:56 /home/example/.local/modules/all/GCC/11.3.0.luaNote
Since only Lmod can consume module files in Lua syntax, it must be used as a modules tool; see also Module files syntax (
--module-syntax).Only changing the syntax of the module file does not affect the module name, so Lmod will report the module as being available. Hence,
--rebuildmust be used here as well. -
to generate a module file using a different naming scheme,
--module-onlycan be combined with--module-naming-scheme:$ eb --installpath-modules=$HOME/test/modules --module-only --module-naming-scheme=HierarchicalMNS --rebuild ... == building and installing Core/GCC/11.3.0... ... == sanity checking [skipped] == creating module... == COMPLETED: Installation ended successfully $ module unuse $HOME/.local/modules/all $ module use $HOME/test/modules/all $ module avail ---------------------------- /home/example/test/modules/all ---------------------------- Core/GCC/11.3.0Note
Modules that are generated used different module naming schemes should never be mixed, hence the use of
--installpath-modules, see also Direct options:--installpath-softwareand--installpath-modules.Note
The modules files generated using the specified module naming scheme will most likely not be tied to an existing software installation in this case (unless the software installation was already there somehow), since the name of the subdirectory of the software installation prefix is also governed by the active module naming scheme. This is also why
--rebuildmust be used in the example above (to skip the sanity check that verifies the software installation).Thus, this is only useful to assess how the module tree would look like under a particular module naming scheme; the modules themselves are useless since they point to empty installation directories.
To tie a module file generated using to an existing software installation that was performed under a different module naming scheme, a simple module naming scheme can be implemented that mixes two modules naming schemes, by providing the name of the software installation subdirectory using one scheme, and the module names (and other metadata for module files) with the other.
An example of such a module naming scheme is
MigrateFromEBToHMNS, which allows to generate module files using the hierarchical module naming scheme implemented byHierarchicalMNSfor the software installed in subdirectories following the default EasyBuild module naming schemeEasyBuildMNS. TheMigrateFromEBToHMNSmodule naming scheme is available since EasyBuild v2.2.0.