Frequently Asked Questions

How does the module command work?

We know that the child program inherits the parents’ environment but not the other way around. So it is very surprising that a command can change the current shell’s environment. The trick here is that the module command is a two part process. The module shell function in bash is:

$ type module
module() { eval $($LMOD_CMD bash "$@") }

Where $LMOD_CMD points to your lmod command (say /apps/lmod/lmod/libexec/lmod). So if you have a module file (foo/1.0) that contains:

setenv("FOO", "BAR")

then “$LMOD_CMD bash load foo/1.0” the following string written to stdout:

export FOO=BAR
...

The eval command read that output from stdout and changes the current shell’s environment. Any text written to stderr bypasses the eval and written to the terminal.

What are the environment variables _ModuleTable001_, _ModuleTable002_, etc doing it in the environment?

The module command remembers its state in the environment through these variables. The way Lmod does it is through a Lua table called ModuleTable:

ModuleTable = {
   mT = {
     git = { ... }
   }
}

This table contains quotes and commas and must be store in environment. To prevent problems the various shells, the table is encoded into base64 and split into blocks of 256 characters. These variable are decoded at the start of Lmod. You can see what the module table contains with:

$ module --mt

How does one debug a modulefile?

There are two methods. Method 1: If you are writing a Lua modulefile then you can write messages to stderr with and run the module command normally:

local a = "text"
io.stderr:write("Message ",a,"\n")

Method 2: Take the output directly from Lmod. You can put print() statements in your modulefile and do:

$ $LMOD_CMD bash load *modulefile*

Why doesn’t % module avail |& grep ucc work under tcsh and works under bash?

It is a bug in the way tcsh handles evals. This works:

% (module avail) |& grep ucc

However, in all shells it is better to use:

% module avail ucc

instead as this will only output modules that have “ucc” in their name.

Why does Lmod require a static location of lua? Why shouldn’t a site allow Lmod to use the lua found in the path?

The short answer is that it is possible but for general use it is not a good idea. If you change the first line of the lmod script to be:

"#!/usr/bin/env lua"

and all the other Lmod executable scripts to do the same then all the scripts would use the lua found in the user’s $PATH. There are other things that would have to change as well. Lmod carefully sets two env. vars: LUA_PATH and LUA_CPATH to be compatible with the Lua that was used to install lmod.

So why does Lmod go to this much trouble? Why doesn’t Lmod just use the lua found in the path and not set LUA_PATH and LUA_CPATH. Earlier version of Lmod did not. The answer is users. Lmod has to protect itself from every user out there. What if a user installs their own Lua? What if your system install of Lua is version 5.1 but your user wants to install the latest version of Lua. The libraries that Lmod depends on change with version. What if a user installs their own version lua but doesn’t install the required libraries for Lmod to work (lua-posix, lfs) or they install their own library called lfs but it does something completely different. Lmod would fail with very strange errors.

To sum up. Lmod is very careful to use the Lua that was used to install it and the necessary libraries. Lmod is also very careful to set LUA_PATH and LUA_CPATH internally so that user changes to those env. variables don’t affect how Lmod runs.

Can I disable the pager output?

Yes, you can. Just set the environment variable LMOD_PAGER to none.

Why are messages printed to standard error and not standard out?

The module command is an alias under tcsh and a shell routine under all other shells. There is an lmod command which writes out commands such as export FOO=”bar and baz” and messages are written to standard error. The text written to standard out is evaluated so that the text strings make changes to the current environment. See next question for a different way to handle Lmod messages.

Can I force the output of list, avail and spider to go to stdout instead of stderr?

Bash and Zsh user can set the environment variable LMOD_REDIRECT to yes. Sites can configure Lmod to work this way by default. However, no matter how Lmod is set-up, this will not work with tcsh/csh due to limitations of this shell.

How can I use grep easily with the module command?

If your site doesn’t send the output of stdout, you can still use this trick when you need to grep the output of module command. Here are some examples:

$ module -t --redirect avail       | grep foo
$ module --raw --redirect show foo | grep bar
$ module -t --redirect spider      | grep baz

Can I ignore the spider cache files when doing module avail?

Yes you can:

$ module --ignore_cache avail

or you can set:

$ export LMOD_IGNORE_CACHE=1

to make Lmod ignore caches as long as the variable is set.

I have created a module and “module avail” can’t find it. What do I do?

Assuming that the modulefile is in MODULEPATH then you have an out-of-date cache. Try running:

$ module --ignore_cache avail

If this does find it then you might have an old personal spider cache. To clear it do:

$ rm -rf ~/.lmod.d/.cache

If “module avail” doesn’t find it now, then the system spider cache is out-of-date. Please ask your system administrator to update the cache. If you are the system administrator then please read System Spider Cache and User Spider Cache

Why doesn’t the module command work in shell scripts?

It will if the following steps are taken. First the script must be a bash script and not a shell script, so start the script with #!/bin/bash. The second is that the environment variable BASH_ENV must point to a file which defines the module command. The simplest way is having BASH_ENV point to /opt/apps/lmod/lmod/init/bash or wherever this file is located on your system. This is done by the standard install. Finally Lmod exports the module command for Bash shell users.

How do I use the initializing shell script that comes with this application with Lmod?

New in Lmod 8.6+, a modulefile can contain source_sh (“shell”, “shell_script arg1 arg2 …”) to source a shell script by automatically converting it into module commands. Sites can use $LMOD_DIR/sh_to_modulefile to convert the script once. See Shell scripts and Lmod for details.

Why is the output of module avail not filling the width of the terminal?

If the output of module avail is 80 characters wide, then Lmod can’t find the width of the terminal and instead uses the default size (80). If you do module --config, you’ll see a line:

Active lua-term true

If it says false instead then lua-term is not installed. One way this happens is to build Lmod on one computer system that has a system lua-term installed and the package on another where lua-term isn’t installed on the system.

Why isn’t the module defined when using the screen program?

The screen program starts a non-login interactive shell. The Bash shell startup doesn’t start sourcing /etc/profile and therefore the /etc/profile.d/*.sh scripts for non-login interactive shells. You can patch bash and fix /etc/bashrc (see Issues with Bash for a solution) or you can fix your ~/.bashrc to source /etc/profile.d/*.sh

You may be better off using tmux instead. It starts a login shell.

Why does LD_LIBRARY_PATH get cleared when using the screen program?

The screen program is a guid program. That means it runs as the group of the program and not the group associated with the user. For security reason all of these kinds of program clear LD_LIBRARY_PATH. This unsetting of LD_LIBRARY_PATH is done by the Unix operating system and not Lmod.

You may be better off using tmux instead. It is a regular program.

How can you write TCL files that can be safely used with both Lmod and Tmod?

For example the hide-version command only works Lmod and could be found in ~/.modulerc. This could be read by both Tmod and Lmod. You can prevent Tmod from executing Lmod only code in the following way:

#%Module
global env
if { [info exists $env(LMOD_VERSION_MAJOR)]} {
   hide-version CUDA/8.8.8
}

Lmod defines the environment variable LMOD_VERSION_MAJOR during its execution. This trick can also be used in a TCL modulefile to set the family function:

#%Module
...
global env
if { [info exists $env(LMOD_VERSION_MAJOR)]} {
   family compiler
}

As of Lmod 8.4.8+ you can also use the TCL global variable ModuleTool:

#%Module
...
if ( [ info exists ::ModuleTool ] && $::ModuleTool == "Lmod" } {
   family compiler
}

How can I get the shell functions created by modules in bash shell scripts such as job submission scripts?

First, please make sure that shell functions and alias works correctly in bash interactive sub-shells. If they don’t then your site is not setup correctly.

Once that works then change the first line of the shell script to be:

#!/bin/bash -l

Note that is a minus ell not minus one. This will cause the startup scripts to be sourced before the first executable statement in the script.

Why do modules get sometimes get loaded when I execute module use <path>?

A main principal is that when $MODULEPATH changes, Lmod checks all the currently loaded modules. If any of thoses modules would not have been chosen then each is swapped for the new choice.

How to use module commands inside a Makefile?

A user might wish to use module commands inside a Makefile. Here is a generic way that would work with both Tmod and Lmod. Both Lmod and Tmod define MODULESHOME to point to the top of the module install directory and both tools use the same initialization method to define the module command.

Here is an example Makefile that shows a user listing their currently loaded modules:

module_list:
       source $$MODULESHOME/init/bash; module listt

What to do if new modules are missing when doing module avail?

If your site adds a new modulefile to the site’s $MODULEPATH but are unable to see it with module avail?

It is likely that your site is having an spider cache issue. If you see different results from the following commands then that is the problem:

$ module --ignore_cache avail
$ module                avail

If you see a difference between the above two commands, delete (if it exists) the user’s spider cache:

$ rm -rf ~/.lmod.d/.cache ~/.lmod.d/__cache__

and try again. If that still leads to a difference then there is a out-of-date system spider cache. Please see System Spider Cache on how to setup and update a system spider cache. This issue can happen with a user’s personal spider cache. Please see User Spider Cache for more details.

How to edit a modulefile?

Lmod does not provide a way to directly edit modulefiles. Typically modulefiles are owned by the system so cannot be editted by users. However, Lmod does provide a convenient way to locate modules which could be used for a bash/zsh shell function:

function edit_modulefile () {
   $EDITOR $(module --redirect --location $1)
}

If my startup shell is bash or tcsh and I start zsh, why do I get a message that is like: “/etc/zsh/zshrc:48: compinit: function definition file not found”

Lmod supports both zsh and ksh. Both these shell use shell var. FPATH but in very different ways. The issue is that some bash or tcsh users run ksh scripts and need access to the module command. In the K-shell, the env. var. FPATH is exported and is the path to where the module shell function is found. Z-shell also uses FPATH to point to tool like compinit and others. By exporting FPATH, Z-shell does not change the value of FPATH which means that the zsh user can not find all the functions that make it work.

The solution is to add “unset FPATH” in your startup files before “exec zsh”.

Why doesn’t setting CC in compiler and mpi modulefiles work always work reliably in a software hierarchy?

When Lmod sees changes to $MODULEPATH it immediately checks to see whether all the currently loaded modules should still be loaded. This means that setting CC with a pushenv() before modifying $MODULEPATH is unsafe. The correct way to modify $MODULEPATH is in the beginning of the modulefile. So for the gcc modulefile you might have:

prepend_path("MODULEPATH","...")
pushenv("CC","gcc")

and not the other way around.