New features — Modules documentation (original) (raw)

This document describes the major changes occurring between versions of Modules. It provides an overview of the new features and changed behaviors that will be encountered when upgrading.

v5.5

This new version is backward-compatible with previous version 5 release. It fixes bugs but also introduces new functionalities that are described in this section. See the 5.5 release notes for a complete list of the changes between Modules v5.4 and v5.5.

Logging activity

The ability to log module command activity is now available out of the box. It could be enabled on previous versions by using a specificsiteconfig.tcl configuration script as described in theLog module command cookbook recipe.

Integrated logging feature relies on two configuration options:

logger option relies on the logger command by default, which is usually available and already installed on most systems.

The logged_events option is empty by default, so no content is sent to logs by default. It recognizes the following events:

$ module config logged_events +requested_cmd:requested_eval $ ml av --------------- /path/to/modulefiles --------------- bar/1.0 foo/1.0 qux/1.0 $ module load bar Loading bar/1.0 Loading requirement: qux/1.0 $ module purge

In the above example the module command and modulefile evaluations directly requested by users are sent to the system log:

$ journalctl -q -t modules Apr 29 07:47:42 hostname modules[3777797]: user="username" command="avail" arguments="" Apr 29 07:48:10 hostname modules[3777876]: user="username" command="load" arguments="bar" Apr 29 07:48:10 hostname modules[3777876]: user="username" mode="load" module="bar/1.0" specified="bar" modulefile="/path/to/modulefiles/bar/1.0" requested="1" Apr 29 07:48:17 hostname modules[3777914]: user="username" command="purge" arguments="" Apr 29 07:48:17 hostname modules[3777914]: user="username" mode="unload" module="bar/1.0" specified="bar/1.0" modulefile="/path/to/modulefiles/bar/1.0" requested="1"

Some messages can also be sent during modulefile evaluation by using thelog channel of the puts command:

$ cat /path/to/modulefiles/foo/1.0 #%Module puts log {some message sent to log} $ ml foo/1.0 $ journalctl -q -t modules Apr 29 07:47:51 hostname modules[3777835]: user="username" command="ml" arguments="foo/1.0" Apr 29 07:47:51 hostname modules[3777835]: some message sent to log Apr 29 07:47:51 hostname modules[3777835]: user="username" mode="load" module="foo/1.0" specified="foo/1.0" modulefile="/path/to/modulefiles/foo/1.0" requested="1"

PowerShell support

Support for PowerShell (pwsh) is added. This shell may be used with Modules on either Unix or Windows platforms. All main kind of environment changes are supported.

A envmodule shell function is configured on PowerShell, as module is a reserved keyword on this shell.

The Windows-specific distribution zipball of Modules has been updated to also contain the required files to setup PowerShell support.

Conflict unload

Automated module handling is enhanced with the addition of a new mechanism:Conflict Unload. When loading a module, this new mechanism automatically unloads conflicting modules and their dependent.

$ module load foo/1 Loading foo/1{-debug} Loading requirement: qux/1 bar/1 $ module load foo/2 Loading foo/2 ERROR: Module cannot be loaded due to a conflict. HINT: Might try "module unload foo/1" first. $ module config conflict_unload 1 $ module load foo/2 Loading foo/2{-debug} Unloading conflict: foo/1{-debug}

As seen in the above example, Conflict Unload is controlled by theconflict_unload configuration option. It is disabled by default as it changes behaviors of the automated module handling mode. But everyone is encouraged to enable this new option to benefit from an highly automated experience. This option can be changed at installation time with --enable-conflict-unloadconfigure script option.

Conflict Unload handles all kind of conflicts: conflicts defined by already loaded modules, conflict declared by loading module through conflict,family or module unload commands. It also supports the unload of a module that is loaded again but with different variant values.

$ module load foo/2 +debug Loading foo/2{+debug} Unloading conflict: foo/2{-debug}

Along with Conflict Unload, the Useless Requirement Unload mechanism is added to load evaluation. With it, auto loaded requirements of modules unloaded by Conflict Unload mechanism are automatically unloaded.

$ module load foo/3 Loading foo/3{-debug} Unloading conflict: foo/2{+debug} bar/1 Loading requirement: bar/2 Unloading useless requirement: qux/1

Dependent modules of Conflict Unload modules are considered Dependent Reload modules. They are unloaded prior unloading the _Conflict Unload_module that depend on them. Reload occurs after loading main module and only if the loaded environment satisfies the dependencies of the module to reload.

$ module load bar/3 Loading bar/3 Loading requirement: qux/2 $ module load qux/1 Loading qux/1 Unloading dependent: bar/3 Unloading conflict: qux/2 Reloading dependent: bar/3

Improved automated module handling mechanisms

Along with the introduction of Conflict Unload mechanism, some of the existing automated module handling mechanisms have been adapted and enhanced.

Useless Requirement Unload (UReqUn) mechanism is moved during aswitch sub-command from the end of the unload phase to the end of the load phase. Both UReqUn modules coming from unload and load phases are this way treated jointly. If UReqUn modules resulting from the switch unload phase are detected as conflict to the switched-on module, it is advised to enable the conflict_unload mechanism to automatically handle them.

$ module switch foo/3 foo/4 Loading foo/4 ERROR: Module cannot be loaded due to a conflict. HINT: Might try "module unload bar/2" first.

Switching from foo/3{-debug} to foo/4 ERROR: Load of switched-on foo/4 failed $ module config conflict_unload 1 $ module switch foo/3 foo/4 Switching from foo/3{-debug} to foo/4 Unloading conflict: bar/2

Dependent Reload (DepRe) modules that also are UReqUn modules are now unloaded during the DepRe unload phase instead of during the _UReqUn_process. As a consequence these modules are now unloaded prior main module action, as they are a dependent of this main module. This kind of _UReqUn_modules may not be a dependency of a module unloaded in the current processing.

Modules from Dependent Unload (DepUn) and Dependent Reload (DepRe) mechanisms are now mixed together to proceed to their unload. They are this way unloaded in their reverse loading order. So if a module is part of the_DepUn_ process and one of its requirement is part of the DepRe process, the requirement is now unloaded after its dependent.

Reload of Dependent Reload (DepRe) module is now skipped if this module is not found loadable in environment. Either because its requirements are not loaded or a conflict is spotted. Such module is considered _Dependent Unload_rather Dependent Reload. If DepRe module is sticky, reload is attempted anyway.

Reload of all Dependent Reload (DepRe) modules now occurs after main module evaluation rather after each sub module evaluation that triggered the_Dependent Reload_ unload phase.

When a loading module requires a Dependent Unload (DepUn) module, the unload of this module was forced when it was triggered by a Conflict Unload or a switched-off module. The unload of such DepUn module is not forced anymore by default and an error is raised if a loading module requiring the _DepUn_module is detected.

When unloading Dependent Reload modules, do not force by default this unload unless asked with --force option. As a result an error is obtained when trying to reload Dependent Reload modules that conflict with other loaded modules. This error may be by-passed with --force command-line option.

Specific modulepath for requirements

A --modulepath option is introduced on the always-load,depends-on, prereq, prereq-all andprereq-any modulefile commands. This option indicates that requirement should be specifically searched in the provided list of modulepaths. Such mechanism helps to ensure loaded requirements are those expected and they are not coming from an unexpected module tree.

In the following example, foo module requires bar. It expects thebar module from the same modulepath than foo. User's environment has enabled another modulepath that also provides a bar module. Use of--modulepath option guides the requirement load mechanism to the expected module.

$ module avail --------------------- /path/to/othermfiles --------------------- bar/1

--------------------- /path/to/modulefiles --------------------- bar/1 foo/1 foo/2 $ module show foo/1

/path/to/modulefiles/foo/1:

prereq --modulepath .. bar/1

$ module load foo/1 Loading foo/1 Loading requirement: bar/1 $ echo $LMFILES /path/to/modulefiles/bar/1:/path/to/modulefiles/foo/1

Specific modulepath set may guide to a directory not necessarily enabled (i.e., not defined in MODULEPATH environment variable):

$ module show foo/2

/path/to/modulefiles/foo/2:

prereq --modulepath /path/to/alternatefiles bar/2

$ module load foo/2 Loading foo/2 Loading requirement: bar/2 $ module list Currently Loaded Modulefiles:

  1. bar/2 2) foo/2 $ echo $LMFILES /path/to/alternatefiles/bar/2:/path/to/modulefiles/foo/2

If a required module is already loaded from a modulepath that does not correspond to the requirement definition, an error is obtained. However if the Conflict Unload mechanism described above is enabled, the conflicting module will automatically be unloaded.

$ module purge $ module load bar/1 $ echo $LMFILES /path/to/othermfiles/mp2/bar/1 $ module load foo/1 Loading bar/1 ERROR: Module already loaded from a different modulepath

Loading foo/1 ERROR: Load of requirement bar/1 (specific path) failed $ module config conflict_unload 1 $ module load foo/1 Loading foo/1 Unloading conflict: bar/1 Loading requirement: bar/1 $ echo $LMFILES /path/to/modulefiles/bar/1:/path/to/modulefiles/foo/1

v5.4

This new version is backward-compatible with previous version 5 release. It fixes bugs but also introduces new functionalities that are described in this section. See the 5.4 release notes for a complete list of the changes between Modules v5.3 and v5.4.

Purging sticky modules

The sticky_purge configuration option is added to define the behavior of purge sub-command when unloading a sticky or super-sticky module.

By default an error is raised. sticky_purge can be changed to emit a warning message instead or to be silent.

$ module purge Unloading foo/1.0 ERROR: Unload of sticky module skipped $ module config sticky_purge warning $ module purge Unloading foo/1.0 WARNING: Unload of sticky module skipped $ module config sticky_purge silent $ module purge $ module list Currently Loaded Modulefiles:

  1. foo/1.0

Key: sticky

Specific modulepath labels

modulepath-label command is introduced to define a label to use to designate modulepath in module avail output. This new command should be used in global or modulepath-specific rc files.

$ cat /path/to/modulefiles/.modulerc #%Module modulepath-label . Tools $ module avail foo ---------------------- Tools ---------------------- foo/1.0 foo/2.0

Unique module name loaded

The configuration option unique_name_loaded is introduced to allow only one module loaded per module name. It is similar to the _One name rule_feature introduced by the Lmod project.

When enabled, unique_name_loaded produces a conflict definition at the start of modulefile evaluation for each actual and alternative module root name. For instance when loading a module named foo/sub/1.0 with aqux/1.0 alias, it produces a conflict against foo and qux.

When loading a module that shares a name with an already loaded module, an error is raised due to the conflict definition. This error aborts the module load evaluation.

$ cat /path/to/modulefiles/bar/1.0 #%Module $ cat /path/to/modulefiles/bar/2.0 #%Module $ module config unique_name_loaded 1 $ module load bar/1.0 $ module load bar/2.0 Loading bar/2.0 ERROR: Module cannot be loaded due to a conflict. HINT: Might try "module unload bar" first.

unique_name_loaded is disabled by default. It can be changed with module config sub-command or at installation time with--enable-unique-name-loaded configure script option.

Cache sourced files

You may have gathered common Tcl code files used by all your modulefiles in Tcl files that are sourced with source(n) Tcl command. When loading multiple modulefiles at once, these sourced files may be read for every loaded modules, which may induce some I/O load.

The configuration option source_cache is introduced to cache files evaluated through source Tcl command in order to only read them once during a modulecmd.tcl execution. When this option is enabled, sourced file read also benefits from optimized I/O calls of Modules Tcl extension library.

In the following example, 20 modules are loaded, each of them sources the same Tcl file:

$ syscall_list=close,fcntl,ioctl,newfstatat,openat,read,readlink $ strace -f -e syscalllist−csyscall_list -c syscalll​ist−cMODULES_CMD bash load foo/20 % time seconds usecs/call calls errors syscall

31.48 0.000781 1 466 read 28.58 0.000709 5 131 16 openat 14.19 0.000352 2 142 3 newfstatat 9.71 0.000241 1 124 close 8.02 0.000199 0 212 211 readlink 4.55 0.000113 1 67 47 ioctl 3.47 0.000086 1 60 fcntl

100.00 0.002481 2 1202 277 total

When sourced file is cached, it is only read once which saves a substantial amount of I/O operations:

$ module config source_cache 1 $ strace -f -e syscalllist−csyscall_list -c syscalll​ist−cMODULES_CMD bash load foo/20 % time seconds usecs/call calls errors syscall

34.41 0.000523 4 112 16 openat 27.37 0.000416 0 428 read 11.84 0.000180 1 122 3 newfstatat 11.51 0.000175 1 105 close 7.11 0.000108 2 47 27 ioctl 5.99 0.000091 0 92 91 readlink 1.78 0.000027 0 40 fcntl

100.00 0.001520 1 946 137 total

Abort on error

When an error occurs during the evaluation of a modulefile, several module sub-commands, like load or unload, continue their processing. It means these sub-commands do not stop if they have multiple modulefiles to evaluate. This is named the continue on error behavior.

The abort_on_error configuration option is introduced to abort, for the sub-commands listed in its value, the evaluation of multiple modulefiles if one fails to evaluate.

The abort on error behavior is applied by default on ml command,reload sub-command and the unload phase of switchsub-command. This behavior may also be turned on for load,mod-to-sh, purge, switch, try-load andunload sub-commands. If a command is removed fromabort_on_error's value, it applies the _continue on error_behavior.

In the following example, abort on error behavior is applied toload sub-command to stop evaluation instead of loading the remaining modulefile in the sequence if first modulefile fails to evaluate.

$ module load foo bar Loading foo/1 ERROR: Module evaluation aborted $ module list Currently Loaded Modulefiles:

  1. bar/1 $ module purge $ module config abort_on_error +load $ module load foo bar Loading foo/1 ERROR: Module evaluation aborted $ module list No Modulefiles Currently Loaded.

When enabled for switch sub-command, whole action aborts if switched-on modulefile fails to load. By default, switch only aborts if switched-off modulefile fails to unload.

$ module load bar $ module switch bar foo Loading foo/1 ERROR: Module evaluation aborted

Switching from bar/1 to foo/1 WARNING: Load of switched-on foo/1 failed $ module list No Modulefiles Currently Loaded. $ module load bar $ module config abort_on_error +switch $ module switch bar foo Loading foo/1 ERROR: Module evaluation aborted

Switching from bar/1 to foo/1 WARNING: Load of switched-on foo/1 failed $ module list Currently Loaded Modulefiles:

  1. bar/1

abort_on_error configure option has no impact on modulecommands defined in modulefiles. When --force option is set, sub-commands apply the continue on error behavior.

When --enable-new-features installation option is set, the_abort on error_ behavior is also enabled on load andswitch sub-commands.

Improve error reporting

Module evaluation error reports have been tweaked to appear where these errors occur rather reporting them on the message block of the main action. Such change helps to better understand the module evaluation flow.

$ module load baz Loading baz/1 Loading requirement: foo/1 $ module switch foo bar Unloading baz/1 Module ERROR: invalid command name "bad_command" while executing "bad_command" (file "/path/to/modulefiles/baz/1" line 4) Please contact <root@localhost>

Unloading foo/1 ERROR: Unload of dependent baz/1 failed

Switching from foo/1 to bar ERROR: Unload of switched-off foo/1 failed

switch sub-command specific error messages are not reported whenmodule switch command is run from a modulefile (like unload of switched-off module failed, load of switched-on module failed messages or Switching block message). On switch sub-command, load failure of switched-on module is now reported as an error.

$ module load foo $ module switch foo qux Loading qux/1 Module ERROR: invalid command name "bad_command" while executing "bad_command" (file "/path/to/modulefiles/qux/1" line 2) Please contact <root@localhost>

Switching from foo/1 to qux/1 ERROR: Load of switched-on qux/1 failed

Conflict error messages now describe the loaded module name and version that conflicts rather the generic conflict module specification.

$ cat /path/to/modulefiles/bar/1 #%Module conflict foo $ module load foo $ module load bar Loading bar/1 ERROR: Module cannot be loaded due to a conflict. HINT: Might try "module unload foo/1" first.

New options for source-sh modulefile command

Support for new shell mode named bash-eval is added tosource-sh modulefile command. With this new mode, the generated output of the bash shell script is evaluated to get the environment changes instead of sourcing this script.

$ cat /path/to/modulefiles/foo/foo-output.sh #!/bin/bash cat <<EOF export FOO=value; alias foo_alias='echo foo_alias;' foo_function() { echo foo_function; } EOF $ cat /path/to/modulefiles/foo/1 #%Module set foo_script [file dirname $ModulesCurrentModulefile]/foo-output.sh source-sh bash-eval $foo_script

$ module show foo/1

/path/to/modulefiles/foo/1:

set-alias foo_alias {echo foo_alias;} set-function foo_function { echo foo_function} setenv FOO value

$ module load foo/1 $ foo_alias foo_alias

bash-eval shell support is also available on sh-to-modsub-command.

A new option is added to source-sh modulefile command: --ignore. It filters shell elements changed by script, not to get these changes applied when loading modulefile. This option accepts a list, separated by colon character, of shell elements: envvar, alias, function,complete or chdir.

$ module purge $ cat /path/to/modulefiles/foo/2 #%Module set foo_script [file dirname $ModulesCurrentModulefile]/foo-output.sh source-sh --ignore alias:function bash-eval $foo_script

$ module show foo/2

/path/to/modulefiles/foo/2:

setenv FOO value

$ module load foo/2 $ foo_alias bash: foo_alias: command not found

This --ignore option can be used with any shell supported bysource-sh.

v5.3

This new version is backward-compatible with previous version 5 release. It fixes bugs but also introduces new functionalities that are described in this section. See the 5.3 release notes for a complete list of the changes between Modules v5.2 and v5.3.

Module cache

A module cache file can be created under each modulepath directory with newcachebuild sub-command. Cache file is named .modulecache and contains in one file all modulefiles and modulercs found in modulepath directory.

When cache file is available, module search mechanism uses this file rather walking through the content of modulepath directory. I/O operations are saved this way which reduces search processing time.

When searching for available modules without cache, each file contained in enabled modulepaths is opened to check if it is a modulefile or not. Such checks lead to a large number of I/O operations on large module setup like in the below example where a total of 1051 modulefiles are available:

$ module -o "" avail -t | wc -l 1051 $ syscall_list=access,close,getdents64,newfstatat,openat,read $ strace -f -e syscalllist−csyscall_list -c syscalll​ist−cMODULES_CMD bash avail % time seconds usecs/call calls errors syscall

31.09 0.003776 2 1424 9 openat 28.56 0.003469 2 1649 3 newfstatat 14.08 0.001710 1 1421 close 11.85 0.001439 3 460 getdents64 10.88 0.001321 0 1505 read 3.54 0.000430 4 107 5 access

100.00 0.012145 1 6566 17 total

After building cache file for every enabled modulepaths in this example setup, a lot of I/O operations are saved when searching for available modules:

$ module cachebuild Creating /path/to/modulefiles/.modulecache Creating /path/to/modulefiles.2/.modulecache Creating /path/to/modulefiles.3/.modulecache $ module config cache_buffer_bytes 1000000 $ strace -f -e syscalllist−csyscall_list -c syscalll​ist−cMODULES_CMD bash avail % time seconds usecs/call calls errors syscall

70.19 0.000544 2 255 read 13.16 0.000102 2 38 9 openat 8.90 0.000069 1 35 close 6.06 0.000047 1 31 2 newfstatat 1.68 0.000013 1 11 2 access

100.00 0.000775 2 370 13 total

A significant execution time drop may be noticed, especially if modulepath directories are stored on heavily loaded network filesystem.

To further optimize I/O operation count, the cache_buffer_bytesconfiguration option can be set like in the above example to use fewer number of read operation to load cache file content.

To build cache file, user should be granted write access on modulepath directory. Modulefiles or directories that are not accessible for everyone are not recorded in cache. An indication is saved instead to test these limited access elements when cache is loaded to determine if they are available to currently running user.

Cache file can be ignored with --ignore-cache command line switch or more permanently with ignore_cache configuration option.

Cache file is valid indefinitely by default but cache_expiry_secsconfiguration option can be used to define the number of seconds a cache file is considered valid after being generated. Expired cache file is ignored.

Cache file of enabled modulepaths can be deleted all at once withcacheclear sub-command.

Querying available module variants

A new mechanism named Extra match search is introduced to evaluate modulefiles during a module search to find those matching an extra query on a variant value, a dependency or an environment variable definition.

During this specific evaluation, modulefiles are interpreted in scan mode to collect the different Tcl modulefile commands they use. Special care should be given when writing modulefiles to ensure they cope with such evaluation mode.

Extra match search mechanism is available on avail,whatis and paths sub-commands.

With this new mechanism, it is possible to list all available variant defined in modulefiles with their associated values:

$ module config avail_output modulepath:alias:dirwsym:sym:tag:key:variant $ module config variant_shortcut toolchain=% $ module avail --------------------- /path/to/modulefiles --------------------- bar/1.0{%a,b} foo/1.0{%a} qux/1.0{%a,b} bar/2.0{%b} foo/2.0{%a,b,c} qux/2.0{%b,c}

Key: modulepath {%value}={toolchain=value} default-version {variant=value}

You can also search for modules defining a specific variant value:

$ module avail %a --------------------- /path/to/modulefiles --------------------- bar/1.0{%a,b} foo/1.0{%a} foo/2.0{%a,b,c} qux/1.0{%a,b}

Key: modulepath {%value}={toolchain=value} default-version {variant=value}

Note

As extra match search implies additional modulefile evaluations, it is advised to build and use Module cache to improve search speed.

Two new elements, variant and variantifspec, are added to the allowed value list of avail_output and avail_terse_outputconfiguration options. The latter is set in the default value list of both options. When variant is set, variants and their possible values are reported along module they are associated to. When variantifspec is set, available variants are reported only if a variant is specified in search query.

$ module config --reset avail_output $ module config avail_output Modules Release 5.3.0 (2023-05-14)

Key: modulepath $ module avail foo %a --------------------- /path/to/modulefiles --------------------- foo/1.0{%a} foo/2.0{%a,b,c}

Key: modulepath {%value}={toolchain=value} default-version {variant=value}

Append or subtract elements to current option value

For command-line switches and configuration options whose value is a colon-separated list, it is now possible to append or subtract elements to the current value. With a + prefix elements are appended and with a -prefix elements are subtracted.

This new feature applies to --output/-o command-line switches and avail_output, avail_terse_output,list_output, list_terse_output, colors,protected_envvars, shells_with_ksh_fpath,tag_abbrev, tag_color_name, variant_shortcutconfiguration options.

For instance, to output available foo modules without modulepath and tag information:

$ module -o -modulepath:tag avail foo foo/1.0 foo/2.0

On following example, terse output mode of list sub-command is permanently updated to report variant information:

$ module config list_terse_output Modules Release 5.3.0 (2023-05-14)

v5.2

This new version is backward-compatible with previous version 5 release. It fixes bugs but also introduces new functionalities that are described in this section. See the 5.2 release notes for a complete list of the changes between Modules v5.1 and v5.2.

Optional requirements

The --optional option has been added to the prereq,prereq-all, depends-on and always-load modulefile commands to indicate that specified requirement is optional. An automatic load attempt is also performed for optional requirements. If requirement is not found or cannot be loaded, the dependency is yet considered satisfied as it is optional.

$ module show foo

/path/to/modulefiles/foo:

prereq --optional bar

$ module load foo Loading foo Loading requirement: bar

If the optional requirement is unloaded or loaded afterward, the dependent module is automatically reloaded thanks to the auto_handlingmechanism.

$ module unload bar Unloading bar Unloading dependent: foo Reloading dependent: foo $ module list Currently Loaded Modulefiles:

  1. foo $ module load bar Loading bar Unloading dependent: foo Reloading dependent: foo

Modules loaded by other modules with the module try-loadcommand are now considered optional requirements. Dependent module can be loaded without the try-load modules, but now it gets automatically reloaded if try-load module is loaded afterward, to take it into account.

Linting modulefiles

Static analysis of modulefile, modulerc and global/user rc is now possible with lint sub-command. It relies on an external program defined withtcl_linter configuration option. Modules or files specified on the command-line are resolved then passed to the Tcl linter program.

$ cat /path/to/modulefiles/foo/1.0 #%Module if {"str" eq} { else { } $ module lint foo/1.0 Linting /path/to/modulefiles/foo/1.0 ERROR line 2: Could not complete statement. One close brace would complete the first line One close brace would complete at end of line 4. One close brace would complete the script body at line 5. Assuming completeness for further processing. ERROR line 2: Bad expression: missing operand at @ in expression ""str" eq_@_" WARNING line 3: Unknown command "else" NOTICE line 4: Close brace not aligned with line 3 (1 0)

Nagelfar is the Tcl linter recommended for Modules and set by default. This default can be changed at installation time with --with-tcl-linterand --with-tcl-linter-opts options. It can also be configured later on through tcl_linter config option.

Specific syntax databases and plugins for Nagelfar are added by Modules to precisely lint modulefile commands syntax in addition to regular Tcl syntax. The installation of these specific files is controlled with--enable-nagelfar-addons option (enabled by default). Their location is controlled by the --nagelfardatadir option.

$ module lint bar@:1 /path/to/modulefiles/.modulerc Linting /path/to/modulefiles/.modulerc ERROR line 35: Wrong number of arguments (3) to "module-alias" ERROR line 41: Wrong number of arguments (3) to "module-virtual"

Linting /path/to/modulefiles/bar/1.2 WARNING line 19: Unknown command "unk" $ module lint ~/.modulerc Linting /home/user/.modulerc WARNING line 2: Command "setenv" should not be be used in global rc file

lint sub-command outputs messages returned by the Tcl linter program. Nagelfar produces NOTICE, WARNING and ERROR messages. If linter does not report a thing, lint sub-command will be silent, unless if the--verbose/-v is set.

$ module lint /path/to/modulefiles/bar/.version bar/1.4 $ module lint -v /path/to/modulefiles/bar/.version bar/1.4 Linting /path/to/modulefiles/bar/.version Linting /path/to/modulefiles/bar/1.4

When no file is specified to lint sub-command, all the global/user rc files and all the modulefiles and modulercs from enabled modulepaths are analyzed. If the --all/-a option is set, all hidden modulefiles are also linted.

$ module lint Linting /home/user/.modulerc WARNING line 2: Command "setenv" should not be be used in global rc file

Linting /path/to/modulefiles/.modulerc ERROR line 35: Wrong number of arguments (3) to "module-alias" ERROR line 41: Wrong number of arguments (3) to "module-virtual"

Linting /path/to/modulefiles/bar/1.2 WARNING line 19: Unknown command "unk" ...

To use Nagelfar as Tcl linter for Modules, this open source tool has to be installed on your system. When installing from tarball distribution, make surenagelfar.tcl command is found through a PATH lookup or that tcl_linter option is set to its full path location.Nagelfar is also made available as a RPM package in EPEL and Fedora repositories.

mod-to-sh sub-command

New sub-command is added to translate modulefile into shell code:mod-to-sh. It evaluates modulefiles passed as argument and produces code for specified shell.

$ cat /path/to/modulefiles/foo #%Module setenv FOO value set-function foo {echo foo} $ module mod-to-sh bash foo FOO=value; export FOO; foo () { echo foo; }; export -f foo;

Designated modulefiles are evaluated as if they were loading. But instead of producing shell code that is evaluated in current shell session,module command outputs this shell code.

$ cat /path/to/modulefiles/bar #%Module setenv BAR othervalue set-alias bar {echo bar} $ module mod-to-sh fish foo bar set -xg FOO value; set -xg BAR othervalue; alias bar echobar; function foo; echo foo; end; $ module list No Modulefiles Currently Loaded.

All shells supported by modulecmd.tcl script are supported bymod-to-sh.

$ module mod-to-sh python foo bar import os os.environ['FOO'] = 'value' os.environ['BAR'] = 'othervalue'

Initial environment

When Modules initializes, it evaluates the initrc andmodulespath configuration files to enable default modulepaths and load default modules. Initial environment corresponds to the environment state after this initialization.

Initial environment is now saved in an environment variable (__MODULES_LMINIT) in current shell session to remember what are the initial modulepaths and initial modules with their tags and variants if any.

reset sub-command is introduced, in a similar fashion than onLmod, to restore the initial environment. Here, reset relies on the collection mechanism based and the content of__MODULES_LMINIT. Currently enabled modulepaths and loaded modules are respectively unused and unloaded to use the modulepaths and load the modules with tags and variants as described by initial environment.

$ module list Currently Loaded Modulefiles:

  1. foo/1.0 2) bar/1.0 $ module switch bar/1.0 qux/1.0 $ module reset Unloading qux/1.0 Loading bar/1.0

restore sub-command has been adapted to reinitialize the environment to its initial state when no collection name is provided and no _default_collection exists or if __init__ virtual collection name is provided.

It is possible to view the content of the initial environment withsaveshow sub-command. It is displayed when no argument is provided and no collection exists or if __init__ name is provided.

$ module saveshow init

initial environment:

module use --append /path/to/modulefiles module load foo/1.0 module load bar/1.0

Users have the ability to define what is their initial environment state and thus adapt the behavior of reset sub-command withreset_target_state configuration option. Default value is__init__ and it corresponds to the behavior described above. When set to__purge__, a purge command is performed when resetting. Any other value corresponds to the name of a collection to restore.

$ module config reset_target_state purge $ module reset Unloading bar/1.0 Unloading foo/1.0

Stashing environment

The ability to stash current environment is added with the introduction of thestash sub-command. When called current environment is saved in a_stash_ collection then initial environment is restored.

$ module list Currently Loaded Modulefiles:

  1. bar/2.0 2) foo/2.0 $ module stash Unloading foo/2.0 Unloading bar/2.0 $ module list No Modulefiles Currently Loaded.

Sub-commands are added to specifically handle stash collections. Their names are prefixed with stash, like stashlist to list existing stash collections or stashshow to display their content.

$ module stashlist Stash collection list: 0) stash-1665377597432 1) stash-1664946764252 $ module stashshow

/home/user/.module/stash-1665377597432:

module use --append /path/to/modulefiles module load bar module load foo

Stash collections can be designated on sub-commands by their collection name or stash index. Most recent stash collection has index 0, the one after is designated with index 1, and so on. When no stash collection is specified, most recent one is assumed.

$ module stashshow 1

/home/user/.module/stash-1664946764252:

module use --append /path/to/modulefiles module load foobar

Stash collections are restored with the stashpop sub-command. It changes the user environment to match the stash definition, then it deletes the stash collection file.

$ module stashpop Loading bar/2.0 Loading foo/2.0 $ module stashlist Stash collection list: 0) stash-1664946764252

Stash collections can be deleted one by one with stashrm sub-command or all together with stashclear.

$ module stashrm $ module stashlist No stash collection.

Siteconfig hook variables

Several Tcl variables are introduced for Site-specific configurationscript to define specific commands and variables in the evaluation context of modulefiles and modulercs. These commands and variables setup insiteconfig.tcl can be used in modulefile or modulerc. Sites can easily extend modulefile and modulerc syntax with specific elements.

modulefile_extra_cmds variable defines a list of commands to expose in the modulefile evaluation context and the associated procedure to run when this command is called. This variable has to be defined insiteconfig.tcl located for instance at /etc/environment-modules/siteconfig.tcl.

In the following example modulefile_extra_cmds is used to define the sys command and bound it to the sys procedure that is also defined in siteconfig.tcl.

proc sys {mode} { switch -- $mode { name { return myhost-$::tcl_platform(machine) } default { error "Unknown mode '$mode'" } } } set modulefile_extra_cmds {sys sys}

Once siteconfig.tcl is setup, the sys command can be called by modulefiles. In the following example it is used to determine the application path.

$ cat /path/to/modulefiles/foo/1.2 #%Module append-path PATH /path/to/apps/foo-1.2/[sys name]/bin

$ module show foo/1.2

/path/to/modulefiles/foo/1.2:

append-path PATH /path/to/apps/foo-1.2/myhost-x86_64/bin

modulerc_extra_cmds follows the same approach thanmodulefile_extra_cmds and makes specific commands available during modulerc evaluation.

modulefile_extra_vars variable defines a list of variables to expose in the modulefile evaluation context and their associated value. This variable has to be defined in siteconfig.tcl.

In the following example modulefile_extra_vars is used to define the APP_ROOT variable with /path/to/apps as value.

set modulefile_extra_vars {APP_ROOT /path/to/apps}

Once siteconfig.tcl is setup, the APP_ROOT variable can be used in modulefiles.

$ cat /path/to/modulefiles/bar/2.1 #%Module append-path PATH $APP_ROOT/bar-2.1/[sys name]/bin

$ module show bar/2.1

/path/to/modulefiles/bar/2.1:

append-path PATH /path/to/apps/bar-2.1/myhost-x86_64/bin

modulerc_extra_vars follows the same approach thanmodulefile_extra_vars and makes specific variables available during modulerc evaluation.

v5.1

This new version is backward-compatible with previous version 5 release. It fixes bugs but also introduces new functionalities that are described in this section. See the 5.1 release notes for a complete list of the changes between Modules v5.0 and v5.1.

Control output redirection

Since version 4.0, the module function is initialized differently on sh, bash, ksh, zsh and fish shells when their session is found interactive. In such situation module redirects its output from_stderr_ to stdout. Once initialized the redirection behavior is inherited in sub-sessions.

The redirect_output configuration option is introduced in version 5.1, to supersede the default behavior set at initialization time.

$ module load unknown >/dev/null $ module config redirect_output 0 $ module load unknown >/dev/null ERROR: Unable to locate a modulefile for 'unknown'

The --redirect and --no-redirect command-line switches are also added to change the output redirection behavior for a single command:

$ module load unknown --redirect >/dev/null $ module load unknown --no-redirect >/dev/null ERROR: Unable to locate a modulefile for 'unknown'

Change modulefile command behavior

Depending on the evaluation mode of the modulefile (e.g. load, unload,display, etc) commands have different behavior. Most common example is thesetenv command that sets an environment variable when modulefile is loaded and unsets it when it is unloaded. A different behavior may be wished sometimes for commands. This is why options are introduced for some modulefile commands to control what happens on particular evaluation mode.

The --return-value option is added to the getenv andgetvariant modulefile commands to ensure that the value of the designated environment variable or variant is returned even if modulefile is evaluated in display mode:

$ cat /path/to/modulefiles/foo/1.0 #%Module5.1 if {[getenv --return-value VAR] eq {}} { setenv VAR value } $ module display foo/1.0

/path/to/modulefiles/foo/1.0:

setenv VAR value

--remove-on-unload, --append-on-unload, --prepend-on-unload and--noop-on-unload options are added to the remove-path andmodule unuse modulefile commands to control the behavior applied when modulefile is unloaded. With these options it is possible for instance to restore the paths unset at load time or to set other paths:

$ module display bar/1.0

/path/to/modulefiles/bar/1.0:

module unuse --prepend-on-unload /path/to/dir1 module use /path/to/dir2

$ module use Search path for module files (in search order): /path/to/dir1 /path/to/modulefiles $ module bar/1.0 $ module use Search path for module files (in search order): /path/to/dir2 /path/to/modulefiles $ module unload bar/1.0 $ module use Search path for module files (in search order): /path/to/dir1 /path/to/modulefiles

Following the same trend, the --unset-on-unload and --noop-on-unloadoptions are added to the unsetenv modulefile command to be able to choose between unsetting variable, setting a value or performing no operation when modulefile is unloaded.

Reducing number of I/O operations

A new configuration option named mcookie_check is introduced to control the verification made to files to determine if they are modulefiles. By default this configuration option is set to always and when searching for modulefiles within enabled modulepaths each file below these directories is opened to check if it starts with the Modules magic cookie (i.e.,#%Module file signature).

These historical checks lead to a large number of I/O operations on large module setup like in the below example where a total of 1098 modulefiles are available:

$ module -o "" avail -t | wc -l 1098 $ module config mcookie_check always $ strace -f -e open,openat,read,close -c $MODULES_CMD bash avail ... % time seconds usecs/call calls errors syscall

44.29 0.044603 25 1734 166 open 34.15 0.034389 16 2056 close 11.87 0.011949 24 483 5 openat 9.69 0.009761 4 2146 read

100.00 0.100702 6419 171 total

For each file, 3 I/O operations (open, read and close) are achieved to determine if it is a modulefile and include it in search results. When modulefiles are located in a shared filesystem concurrently accessed by hundreds of users, a module avail command may take some time to finish.

When setting the mcookie_check configuration option to the evalvalue, files are not checked anymore when searching for modulefiles, only when evaluating them. All files under modulepaths are considered modulefiles, so the content of these directories must be carefully checked to use thismcookie_check mode which lead to a significant reduction of I/O operations:

$ module config mcookie_check eval $ strace -f -e open,openat,read,close -c $MODULES_CMD bash avail ... % time seconds usecs/call calls errors syscall

30.56 0.013717 14 944 close 28.76 0.012911 21 612 156 open 26.41 0.011857 24 483 5 openat 14.26 0.006403 6 1034 read

100.00 0.044888 3073 161 total

A substantial reduction of execution time may be noticed depending on the storage setup used to host the modulepath directories. A special care should be given to the content of these directories to ensure they only contain modulefiles (see MODULES_MCOOKIE_CHECK).

See the Reduce I/O load cookbook recipe to learn additional features of Modules that could be leveraged to lower the number of I/O operations.

Shell command completion

New modulefile commands complete and uncomplete are added to get the ability to respectively define and unset command completion. bash,tcsh and fish shells are supported.

module display foo

/path/to/modulefiles/foo/1.0:

append-path PATH /path/to/foo-1.0/bin complete fish foo {-s V -l version --description 'Command version'} complete fish foo {-s h -l help --description 'Command help'}

module load foo foo - -h --help (Command help) -V --version (Command version)

sh-to-mod sub-command and source-sh modulefile command have also been updated to track shell completion changes.

$ module sh-to-mod bash /path/to/foo-1.0/share/setup-env.sh #%Module complete bash foo {-o default -F _foo} append-path PATH /path/to/foo-1.0/bin set-function _foo { ...bash completion code...}

Lmod Tcl modulefile support

With this new version, Modules now supports Tcl modulefiles written forLmod, the alternative module implementation developed in Lua. Such modulefiles can be safely evaluated by Modules without raising error.

Support has been added for the following modulefile commands introduced by Lmod: depends-on, prereq-any, always-load,module load-any, pushenv, require-fullnameand family.

The Compatibility with Lmod Tcl modulefile section in themodulefile man page describes the differences existing between the two implementations.

Note that when processing a family command, theLMOD_FAMILY_<NAME> environment variable is defined by Modules to be compatible with existing modulefiles or scripts relying on such variable.

More tagging capabilities

The new --tag option helps to define extra tags onto a loading module. These tags comes in addition to those inherited from the module state or those associated with the module-tag modulefile command.

The --tag option is available on load, load-any,switch and try-load sub-commands and onalways-load, depends-on, module, prereq,prereq-all and prereq-any modulefile commands.

Informational messages of module evaluation have been updated to mention in module denomination the tags applying to it, as it is done in listsub-command output:

$ module load -v --tag=sticky:bar foo/1.0 Loading foo/1.0

In case the designated module is already loaded, the additional tags are added to the list of tags already applied to this module.

$ module list Currently Loaded Modulefiles:

  1. foo/1.0 $ module load -v --tag=sticky:bar foo/1.0 Tagging foo/1.0

The keep-loaded tag is introduced in this version. It avoids an auto-loaded module to get automatically unloaded when its dependent modules are unloaded. This new tag can be set with the module-tag modulefile command or when module is loaded with the always-load modulefile command. Default tag_abbrev configuration option has been updated to add the kL abbreviation for keep-loaded tag. Default dark and light color palettes have been updated too.

$ module show bar/1.0

/path/to/modulefiles/bar/1.0:

always-load foo/1.0

$ module load bar/1.0 Loading bar/1.0 Loading requirement: foo/1.0 $ module unload bar/1.0 $ module list Currently Loaded Modulefiles:

  1. foo/1.0

Key: auto-loaded keep-loaded

When saving a collection, the tags defined with --tag option are recorded to set them again when collection is restored. Tags resulting from module load state, like auto-loaded and keep-loaded, are also recorded.

$ module load --tag=sticky bar/1.0 Loading bar/1.0 Loading requirement: foo/1.0 $ module save $ module saveshow

/home/user/.module/default:

module use --append /path/to/modulefiles module load --tag=auto-loaded:keep-loaded foo module load --tag=sticky bar

The collection_pin_tag configuration option is added to record in collection all tags set on loaded modules. This configuration option is disabled by default.

Note

Collection saved now starts with a #%Module5.1 file signature if--tag option is recorded in it. Such collection could only be handled by Modules version 5.1 and above.

v5.0

With this new major version the module experience has been updated to benefit by default from all the advanced behaviors and features developed over the Modules 4 versions. Modules 5.0 also introduces some breaking changes to improve the consistency of the whole solution. See the 5.0 release notes for a complete list of the changes between Modules v4.8 and v5.0. The changes document gives an in-depth view of the modified behaviors.

Upgraded default configuration

Release after release, new advanced features were added on Modules 4. They were set off by default to avoid breaking change during the version 4 cycle. With the move to a new major release, these features are set on to improve by default usages for everybody. These features enabled by default are:

Some other features that were enabled by default during the Modules 4 cycle have been turned off as they may not be useful for basic usages. Among other things setting off the following features makes the definition of themodule function simpler. Even if off by default, these features can now be enabled once Modules is installed through the initrcconfiguration file.

Removing compatibility version

The ability to co-install version 3.2 of Modules along newer version is discontinued. The installation option --enable-compat-version,switchml shell function and MODULES_USE_COMPAT_VERSIONenvironment variables are thus removed.

The interesting features of Modules 3.2 that were missing in the initial Modules 4 release in 2017 have been reintroduced progressively (likeclear sub-command or --icase search). With Modules 5.0, the refresh sub-command is even changed to the behavior it had on Modules 3.2. So it is a good time for the big jump.

If you are still using Modules 3.2, please refer to the Changes between versionsdocument that describes the differences of this version compared to the newer releases.

Improving Modules initialization

Modules initialization files are now installed by default in the _etc_directory designated by the --etcdir installation option. The initialization configuration file is named initrc in this directory, and the modulepath-specific configuration file is named modulespath. When both files exist, now they are both evaluated instead of just themodulespath file.

Modules magic cookie (i.e., #%Module file signature) is now required at the start of initrc. An error is produced if the magic cookie is missing or if the optional version number placed after the cookie string is higher than the version of the modulecmd.tcl script in use.

Note that initrc configuration file can host more thanmodule use and module load commands.module configuration can also be achieved with this file through the use of module config commands.

Modules initialization has been enhanced for situations where a module environment is found already defined. In this case the loaded modules are automatically refreshed which is useful to re-apply the non-persistent environment configuration (i.e., shell alias and function that are not exported to the sub-shell). For instance when starting a sub-shell session it ensures that the loaded environment is fully inherited from parent shell:

$ ml show foo/1.0

/path/to/modulefiles/foo/1.0:

set-alias foo {echo foo}

$ ml foo/1.0 $ alias foo alias foo='echo foo' $ bash $ ml Currently Loaded Modulefiles:

  1. foo/1.0 $ alias foo alias foo='echo foo'

Simplifying path-like variable reference counting

The reference counting mechanism used for path-like environment variable enables to determine if a path entry has been added several times (by several loaded modules for instance) to know whether or not this path entry should be unset when unloading a module. Entry is not removed if multiple loaded modules rely on it.

The mechanism is not applied anymore to the Modules-specific path variables (like LOADEDMODULES) as an element entry in these variables cannot be added multiple times without duplication. For instance, a given module name and version cannot be added twice in LOADEDMODULES as this module is only loaded once. With this change a thinner environment is produced bymodule. An exception is made for MODULEPATH environment variable where the mechanism still applies.

$ $MODULES_CMD bash load foo/2.0 LMFILES=/path/to/modulefiles/foo/2.0; export LMFILES; LOADEDMODULES=foo/2.0; export LOADEDMODULES; __MODULES_LMTAG=foo/2.0&mytag; export __MODULES_LMTAG; test 0;

Reference counting mechanism has also been simplified for entries in path-like variable that are only referred once. For such entries no entry is set in the reference counting variable (which are now called__MODULES_SHARE_). A reference count entry is set only if the entry in the path-like variable is referred more than one time.

$ ml foo/3.0 $ echo $PATHVAR /path/to/dir1 $ echo $__MODULES_SHARE_PATHVAR

$ ml bar/1.0 $ echo $PATHVAR /path/to/dir1 $ echo $__MODULES_SHARE_PATHVAR /path/to/dir1:2 $ ml -foo/3.0 $ echo $PATHVAR /path/to/dir1 $ echo $__MODULES_SHARE_PATHVAR

$

When the use and unuse module sub-commands are not called during a modulefile evaluation, the reference counter associated with each entry in MODULEPATH environment variable is ignored. In such context, a module use will not increase the reference counter of a path entry already defined and a module unuse will remove specified path whatever its reference counter value. Same change applies for append-path, prepend-path and remove-pathmodule sub-commands when called from the command-line.

$ echo $MODULEPATH /path/to/modulefiles $ echo $__MODULES_SHARE_MODULEPATH /path/to/modulefiles:2 $ ml use /path/to/modulefiles $ echo $__MODULES_SHARE_MODULEPATH /path/to/modulefiles:2 $ ml unuse /path/to/modulefiles $ echo $MODULEPATH

$ echo $__MODULES_SHARE_MODULEPATH

$

v4.8

This new version is backward-compatible with previous version 4 releases. It fixes bugs but also introduces new functionalities that are described in this section. See the 4.8 release notes for a complete list of the changes between Modules v4.7 and v4.8.

Editing modulefiles

edit sub-command is introduced to give the ability to open modulefiles in a text editor. Modulefiles can be specified like with any other sub-command: using regular, symbolic or aliased names or using advanced version specifiers.

$ ml edit foo

edit sub-command resolves the path toward the designated modulefile then call configured text editor to open this modulefile with it. Below, the modulefile is opened with the vi command:

#%Module module-whatis [module-info name] setenv PATH /path/to/foo-1.0/bin ~ ~ ~ "/path/to/modulefiles/foo/1.0" 3L, 42B 1,1 All

The editor configuration option controls the editor command to use. This option can be configured at installation time with the--with-editor installation option. If not set, editorconfiguration option is set by default to vi.

editor configuration option can be changed with theconfig sub-command. Which sets the MODULES_EDITORenvironment variable.

The VISUAL or the EDITOR environment variables override the default value of editor configuration option but are overridden by the MODULES_EDITOR environment variable.

Using version range in version list

The Advanced module version specifiers mechanism has been improved to allow the use of version range (@:version, @vers1:vers2 or @version:) within version list (@version1,version2,...).

It is now possible to write for instance mod@:1.2,1.4:1.6,1.8: to designate all versions of module mod, except versions 1.3 and 1.7.

This improvement is available where the advanced version specifier syntax is supported. Thus it can be either used from the command-line or when writing modulefiles, for instance to hide or tag modules or to declare requirements.

Try module load with no complain if not found

Add the try-load sub-command that tries to load the modulefile passed as argument, like the load sub-command, but does not raise an error if this modulefile cannot be found.

$ module load unknown ERROR: Unable to locate a modulefile for 'unknown' $ echo $? 1 $ module try-load unknown $ echo $? 0 $ module list No Modulefiles Currently Loaded.

This sub-command first introduced by the Lmod project is added to Modules to improve the compatibility between the two module implementations.

try-load is also available within modulefile context to continue the evaluation of a modulefile in case no module is found in its attempt to load another modulefile

$ module display foo/1.0

/path/to/modulefiles/foo/1.0:

module try-load unknown/1.0

$ module load foo/1.0 $ module list Currently Loaded Modulefiles:

  1. foo/1.0

Module variants

Module variants is a new mechanism that allows to pass arguments to evaluated modulefiles in order to achieve different environment variable or module requirement setup with a single modulefile.

Variant specification relies on the Advanced module version specifiersmechanism, which leverages the variant syntax of the Spack package manager:

$ module config advanced_version_spec 1 $ module load -v bar/1.2 toolchain=a -debug Loading bar/1.2{-debug:toolchain=a}

Variants are defined in modulefile with the variant command, which defines the variant type and its accepted values:

#%Module4.8 variant toolchain a b c variant --boolean --default off debug

select software build depending on variant values

set suffix -[getvariant toolchain] if {$ModuleVariant(debug)} { append suffix -dbg }

prepend-path PATH /path/to/bar-1.2$suffix/bin prepend-path LD_LIBRARY_PATH /path/to/bar-1.2$suffix/lib

The bar/1.2 modulefile defines a toolchain variant, which accepts thea, b and c values, and a debug Boolean variant, which is setoff by default. Once these two variants are declared, their value specified on module designation are instantiated in the ModuleVariantarray variable which could also be queried with the getvariantmodulefile command. Selected variant values enable to define a specific installation build path for the bar/1.2 software.

If a variant is not specified when designating module and if this variant is not declared with a default value, an error is obtained:

$ module purge $ module load bar@1.2 Loading bar/1.2 ERROR: No value specified for variant 'toolchain' Allowed values are: a b c

Once module is loaded, selected variants are reported on the listsub-command output:

$ module load bar@1.2 toolchain=b $ module list Currently Loaded Modulefiles:

  1. bar/1.2{-debug:toolchain=b}

Key: {-variant}={variant=off} {variant=value}

Note

The default value of the --with-list-output installation option has been updated to include variant information.

Variant specification could be used where the Advanced module version specifiers is supported. For instance a module may express a dependency over a specific module variant:

$ module show foo/2.1 toolchain=c

/path/to/modulefiles/foo/2.1:

variant toolchain a b c prereq bar@1.2 toolchain={toolchain} prepend-path PATH /path/to/foo-2.1-{toolchain}/bin prepend-path LD_LIBRARY_PATH /path/to/foo-2.1-{toolchain}/lib

In this example, foo/2.1 module depends on bar/1.2 and the same toolchain variant should be selected for both modules in order to load two software builds that are compatible between each other.

$ module purge $ module config auto_handling 1 $ module load foo/2.1 toolchain=a Loading foo/2.1{toolchain=a} Loading requirement: bar/1.2{-debug:toolchain=a}

Variant shortcuts

The variant_shortcut configuration option is added to define shortcut characters for easily specifying variants. Instead of writing the variant name to specify it in module designation (e.g., name=value), the shortcut associated to this variant could be used (i.e., value):

$ module purge $ module config variant_shortcut toolchain=% $ module load foo/2.1 %a Loading foo/2.1{%a} Loading requirement: bar/1.2{-debug:%a}

Configured shortcuts are also used to report the loaded variant onlist sub-command output (shortcuts are explained in key section):

$ module list Currently Loaded Modulefiles:

  1. bar/1.2{-debug:%a} 2) foo/2.1{%a}

Key: auto-loaded {-variant}={variant=off} {%value}={toolchain=value} {variant=value}

v4.7

This new version is backward-compatible with previous version 4 releases. It fixes bugs but also introduces new functionalities that are described in this section. See the 4.7 release notes for a complete list of the changes between Modules v4.6 and v4.7.

Determining module implementation and version

New Modules variables are introduced to determine during the evaluation of a modulefile or a modulerc what module implementation is currently in use. TheModuleTool variable corresponds to the name of the _module_implementation and is set to Modules for this project. TheModuleToolVersion variable corresponds to the version number of the implementation (e.g. 4.7.0).

With these new variables it is possible to precisely know what _module_command is in use then adapt modulefile code to handle a specific behavior or leverage a new feature.

The modulefile command versioncmp is also introduced to provide a simple way to compare two version strings and return if first version string is less than, equal to or greater than second one.

if {[info exists ModuleTool] && $ModuleTool eq {Modules} && [versioncmp $ModuleToolVersion 4.7] >= 0} { # here some code specific for Modules 4.7 and later versions }

The ModuleTool and ModuleToolVersion variables and theversioncmp modulefile command are supported by the Lmod project starting version 8.4.8.

Symbolic version to designate module loaded version

When the Advanced module version specifiers is enabled, the loadedsymbolic version may be used to designate the currently loaded version of specified module.

$ ml display foo@loaded

/path/to/modulefiles/foo/1.0:

module-whatis foo/1.0

If no version of specified module can be found loaded, an error is returned.

$ ml display foo@loaded ERROR: No loaded version found for 'foo' module

Module tags

Module tags are piece of information that can be associated to individual modulefiles. Tags could be purely informational or may lead to specific behaviors.

Module tags may be inherited from the module state set by a modulefile command or consequence of a module action. Tags may also be associated to modules by using the new module-tag modulefile command.

Module tags are reported along the module they are associated to onavail and list sub-command results. Tags could be reported either:

$ cat /path/to/modulefiles/foo/.modulerc #%Module module-tag mytag foo module-tag othertag foo/1.0 $ ml av --------------- /path/to/modulefiles --------------- foo/1.0 mytag:othertag foo/2.0 $ ml foo/1.0 $ ml Currently Loaded Modulefiles:

  1. foo/1.0 mytag:othertag

$ # set SGR code to report 'mytag' with blue background color $ ml config colors "hi=1:di=94:L=90;47:mytag=102" $ ml av --------------- /path/to/modulefiles --------------- foo
$ ml Currently Loaded Modulefiles:

  1. foo

The tag_abbrev configuration option is available to define abbreviated strings for module tags and then use these abbreviations instead of tag names when reporting tags on avail and list command results.

$ # add abbreviation for 'othertag' tag $ ml config tag_abbrev loaded=L:othertag=oT $ ml av --------------- /path/to/modulefiles --------------- foo
$ ml Currently Loaded Modulefiles:

  1. foo

When a SGR code is set for a tag in the color palette, this graphical rendition is applied by default over the module name and the tag name or its abbreviation is not displayed. If tag name or abbreviation is added to thetag_color_name configuration option, graphical rendering is applied to the tag name or abbreviation rather than over the module name they are attached to.

$ # add SGR code for 'oT' tag and set rendition over tag name $ ml config colors "hi=1:di=94:L=90;47:mytag=44:oT=41" $ ml config tag_color_name oT $ ml av --------------- /path/to/modulefiles --------------- foo
$ ml Currently Loaded Modulefiles:

  1. foo

Tags inherited from module state, consequence of a module action or set by using module-tag but that have a special meaning currently are:

Tag Description Set with Abbr. Color
auto-loaded Module has been loaded automatically Inherited aL mod/1.0
forbidden Module cannot be loaded Inherited frommodule-forbid F mod/1.0
hidden Module is not visible on avail Inherited frommodule-hide H mod/1.0
hidden-loaded See Hiding loaded modules Inherited frommodule-hide H mod/1.0
loaded Module is currently loaded Inherited L mod/1.0
nearly-forbidden Module will soon not be able to load anymore Inherited frommodule-forbid nL mod/1.0
sticky See Sticky modules module-tag S mod/1.0
super-sticky See Sticky modules module-tag sS

Hiding loaded modules

The --hidden-loaded option has been added to the module-hidemodulefile command and it indicates that designated hidden modules remain hidden after being loaded.

$ cat /path/to/modulefiles/foo/1.0 #%Module module load bar $ cat /path/to/modulefiles/bar/.modulerc #%Module4.7 module-hide --soft --hidden-loaded bar

In this example, foo depends on bar which is set soft hidden and hidden once loaded. As a consequence, automated load of bar module will not be reported and bar/1.0 will not appear in loaded module list by default:

$ ml foo $ ml Currently Loaded Modulefiles:

  1. foo/1.0

However bar/1.0 is loaded. Hidden loaded modules can be unveiled with the--all/-a option set on the list sub-command.hidden-loaded tag (abbreviated by default to H when colored output is disabled) applies to such modules.

$ ml -a Currently Loaded Modulefiles:

  1. bar/1.0 2) foo/1.0

To also get the informational messages about hidden loaded module automated load or unload, the new verbosity level verbose2 can be used (with-vv option for instance):

$ ml purge $ ml -vv foo Loading bar/1.0

Loading foo/1.0 Loading requirement: bar/1.0

Sticky modules

Module stickiness is introduced, in a similar fashion than on the Lmodproject, to allow to glue modules to the loaded environment. A sticky module cannot be unloaded, unless if the unload action is forced or if the module reloads after being unloaded.

A modulefile is declared sticky by applying it the sticky tag with themodule-tag modulefile command.

$ cat /path/to/modulefiles/foo/.modulerc #%Module4.7 module-tag sticky foo/1.0 $ ml Currently Loaded Modulefiles:

  1. foo/1.0 $ ml -foo Unloading foo/1.0 ERROR: Unload of sticky module 'foo/1.0' skipped $ ml Currently Loaded Modulefiles:
  2. foo/1.0 $ ml --force -foo Unloading foo/1.0 WARNING: Unload of sticky module 'foo/1.0' forced $ ml No Modulefiles Currently Loaded.

Modulefile can also be defined super-sticky by applying the corresponding module tag. Super-sticky module cannot be unloaded even if the unload action is forced. It can only be unloaded if the module reloads afterward.

$ cat /path/to/modulefiles/bar/.modulerc #%Module4.7 module-tag super-sticky bar/1.0 $ ml Currently Loaded Modulefiles:

1) $ ml purge Unloading bar/1.0 ERROR: Unload of super-sticky module 'bar/1.0' skipped $ ml purge -f Unloading bar/1.0 ERROR: Unload of super-sticky module 'bar/1.0' skipped $ ml Currently Loaded Modulefiles:

1)

Modulefiles targeted by a sticky or a super-sticky tag are colored onavail and list sub-command outputs to indicate such tag applies. If colored output is disabled a tag abbreviation is reported along module designation (respectively S and sS).

In case the stickiness applies to the generic module name (and does not target a specific module version or version-set), one version of the sticky or super-sticky module can be swapped by another version of this same module:

$ cat /path/to/modulefiles/baz/.modulerc #%Module4.7 module-tag sticky baz $ ml Currently Loaded Modulefiles:

  1. baz/2.0 $ ml switch baz/1.0 $ ml Currently Loaded Modulefiles:
  2. baz/1.0

Explaining avail/list output

A Key section is added at the end of the avail and listsub-commands output to give hints on the meaning of the graphical rendition applied to elements or what the elements set in parentheses or chevrons along module name stand for.

$ ml av ------------------ /path/to/modulefiles ------------------ foo/1.0 foo/2.0 foo/3.0

Key: loaded default-version sticky =othertag modulepath module-alias

Configuring avail/list output

New configuration options are introduced to control what content to output in addition to modules names on the regular and terse output modes of theavail and list sub-commands.

These new configuration options named avail_output,avail_terse_output, list_output andlist_terse_output can be updated using the configsub-command or set at installation time respectively with the--with-avail-output, --with-avail-terse-output,--with-list-output and --with-list-terse-outputconfigure options.

The four options accept a colon separated list of elements as value. Accepted elements for the avail-related options are: modulepath,alias, dirwsym, sym, tag and key. Accepted elements for the list-related options are: header, idx, sym, tagand key.

In the following example, default output configuration for the availsub-command is checked then module tags and key section are removed to get a simpler output:

$ ml config avail_output Modules Release 4.7.0 (2021-02-19)

Key: modulepath module-alias sticky default-version forbidden $ ml config avail_output modulepath:alias:dirwsym:sym $ ml av ------------------ /path/to/modulefiles ------------------ bar/1.0 bar/2.0 foo/1.0 foo/2.0 foo/2.2

The --output/-o switches are added to define a specific output configuration for the duration of the associated command line. The following example shows how to limit the content reported on a modulelist to the loaded index and the symbolic versions in addition to the module names:

$ ml Currently Loaded Modulefiles:

  1. bar/1.0 2) foo/2.0

Key: default-version sticky $ ml -o idx:sym

  1. bar/1.0 2) foo/2.0

When the new configuration options or command line switches are set to an empty value, the module names are the sole information reported:

$ ml -t -o "" bar/1.0 foo/2.0

In case the modulepath element is withdrawn from the availsub-command output configuration, the available modules from all enabled modulepaths are reported as a single list:

$ ml av --------------- /path/to/other/modulefiles --------------- baz/1.0 baz/2.0

------------------ /path/to/modulefiles ------------------ bar/1.0 bar/2.0 foo/1.0 foo/2.0 foo/2.2

Key: modulepath module-alias sticky default-version forbidden $ ml av --output=alias:tag bar/1.0 baz/1.0 foo/1.0 foo/2.2 bar/2.0 baz/2.0 foo/2.0

Note

The avail_report_dir_sym and avail_report_mfile_sym locked configuration options have been removed. Their behaviors can now be obtained by respectively adding the dirwsym and sym elements to theavail_output or avail_terse_output configuration options.

v4.6

This new version is backward-compatible with previous version 4 releases. It fixes bugs but also introduces new functionalities that are described in this section. See the 4.6 release notes for a complete list of the changes between Modules v4.5 and v4.6.

sh-to-mod sub-command

The sh-to-mod sub-command is added to output as a modulefile content the environment changes done by the evaluation of a shell script passed as argument. sh-to-mod is especially useful for software providing a shell script for their enablement in shell session: it can convert these scripts into modulefiles.

Say for instance, a foo software has been installed and it provides afoo-setup.sh script to activate foo software in user environment:

$ cat /path/to/foo-1.2/foo-setup.sh #!/bin/sh export FOOENV="$1" export PATH=/path/to/foo-1.2/bin:$PATH alias foo='foobin -q -l'

Calling module sh-to-mod on this shell script outputs the environment changes it performs as a modulefile content:

$ module sh-to-mod sh /path/to/foo-1.2/foo-setup.sh arg1 #%Module prepend-path PATH /path/to/foo-1.2/bin set-alias foo {foobin -q -l} setenv FOOENV arg1

Changes on environment variables, shell aliases, shell functions and current working directory are tracked. The following shells are supported: sh,dash, csh, tcsh, bash, ksh, ksh93, zsh and fish.

sh-to-mod acts as a full replacement for the standalonecreatemodule.sh and createmodule.py scripts. However those two scripts are currently still provided for compatibility purpose.

source-sh modulefile command

The source-sh modulefile command is introduced to source environment changes done by the evaluation of a shell script passed as argument. With newly introduced sh-to-mod sub-command resulting environment changes done by script are output as modulefile commands. source-sh applies those modulefile commands as if they were directly written in loading modulefile.

source-sh is useful for software providing a shell script for their enablement. If you want to enable such software with module yet using shell script provided by software for this task, just write a modulefile using source-sh command to call the shell script.

Keeping the same example used to describe sh-to-mod sub-command:foo software provides a foo-setup.sh script for its activation. Create a modulefile foo/1.2 that calls this script:

$ cat /path/to/modulefiles/foo/1.2 #%Module4.6 source-sh sh /path/to/foo-1.2/foo-setup.sh arg1

Displaying this modulefile indicates the environment changes done by script:

$ module display foo/1.2

/path/to/modulefiles/foo/1.2:

prepend-path PATH /path/to/foo-1.2/bin set-alias foo {foobin -q -l} setenv FOOENV arg1

Loading the modulefile applies the environment changes seen above:

$ module load -v foo/1.2 Loading foo/1.2 $ echo $FOOENV arg1 $ alias foo alias foo='foobin -q -l'

Track of these changes is kept in user environment to be able to undo them when modulefile is unloaded:

$ module unload -v foo/1.2 Unloading foo/1.2 $ echo $FOOENV

$ alias foo bash: alias: foo: not found

Changes on environment variables, shell aliases, shell functions and current working directory are tracked. The following shells are supported: sh,dash, csh, tcsh, bash, ksh, ksh93, zsh and fish.

Querying user's name and groups membership

Two new sub-commands are introduced for the module-info modulefile command: username and usergroups. They respectively fetch the name of the user currently running modulecmd.tcl or the name of all the groups this user is member of.

These two new modulefile commands can help to adapt code to specific users or groups. Like for instance to instantiate a modulefile for each group the user is member of:

$ cat /path/to/modulefiles/foo/.modulerc #%Module4.6 foreach grp [module-info usergroups] { module-virtual foo/$grp .common } $ id -G -n grp1 grp2 grp3 $ module avail --------------- /path/to/modulefiles --------------- foo/grp1 foo/grp2 foo/grp3

username and usergroups sub-commands of module-infomodulefile command are only supported on Unix platform.

Hiding modules

The newly introduced module-hide modulefile command enables to dynamically hide modulefiles, module aliases or symbolic versions specified to it:

$ cat /path/to/modulefiles/bar/.modulerc #%Module4.6 module-version bar/1.0 old

hide 'old' symbolic version

module-hide bar/old

hide all version 2 and above

module-hide bar@2: $ cat /path/to/modulefiles/.modulerc #%Module4.6

hide all versions of foo module

module-hide foo

module-hide commands should be placed in module rc files and can leverage the Advanced module version specifiers syntax as shown in the above example.

Hidden modules are excluded from available module search or module selection unless query refers to hidden module by its exact name:

$ ml av --------------- /path/to/modulefiles --------------- bar/1.0 bar/2.0 $ module load -v foo ERROR: Unable to locate a modulefile for 'foo' $ module load -v foo/1.0 Loading foo/1.0 $ module avail bar/old --------------- /path/to/modulefiles --------------- bar/1.0(old)

module-hide command accepts a --soft option to apply a lighter of hiding to modules:

$ cat /path/to/modulefiles/qux/.modulerc #%Module4.6

softly hide all qux modules

module-hide --soft qux

The soft hiding mode enables to hide modules from full availability listing yet keeping the ability to select such module for load without having to use module exact name:

$ ml av --------------- /path/to/modulefiles --------------- bar/1.0 bar/2.0 $ ml av qux --------------- /path/to/modulefiles --------------- qux/1.0 qux/2.0 $ module load -v qux Loading qux/2.0

Alternatively, a --hard option can be set on module-hide command to ensure designated modules do not unveil even if referred by their exact name:

$ cat /path/to/modulefiles/qux/.modulerc #%Module4.6

softly hide all qux modules

module-hide --soft qux

set highest version of qux hard hidden

module-hide --hard qux/3.0 $ ml av qux/3.0 $ ml qux/3.0 ERROR: Unable to locate a modulefile for 'qux/3.0'

Some users or groups can be set unaffected by hiding mechanism with the --not-user or --not-group options:

$ cat /path/to/modulefiles/quuz/.modulerc #%Module4.6

hiding does not apply to grp1 and grp2 groups

module-hide --not-group {grp1 grp2} quuz

$ id --groups --name grp1 grp7 $ ml av quuz --------------- /path/to/modulefiles --------------- quuz/1.0 quuz/2.0 $ ml -v quuz Loading quuz/2.0

Hiding mechanism can also be set effective only before or after a given date time with the --before and --after options. Accepted date time format is YYYY-MM-DD[THH:MM].

$ cat /path/to/modulefiles/fum/.modulerc #%Module4.6

hide only before a given date

module-hide --hard --before 2020-09-01T12:00 fum/1.0

hide only after a given date

module-hide --hard --after 2020-09-01 fum/2.0

$ date Fri 04 Sep 2020 06:21:48 AM CEST $ ml av fum --------------- /path/to/modulefiles --------------- fum/1.0

Hidden modules can be included in available module searches if option--all/-a is set on avail, aliases,whatis or search sub-commands. Hard hidden modules are unaffected by this option and stay hidden.

$ ml av -a --------------- /path/to/modulefiles --------------- bar/1.0(old) foo/1.0 fum/1.0 quuz/2.0 qux/2.0 bar/2.0 foo/2.0 quuz/1.0 qux/1.0

Forbidding use of modules

The module-forbid modulefile command is added to dynamically forbid the evaluation of modulefiles it specifies. When forbidden, a module cannot be loaded and an access error is returned when an attempt is made to evaluate it.

$ cat /path/to/modulefiles/foo/.modulerc #%Module4.6 module-forbid foo@1: $ ml foo/1.0 ERROR: Access to module 'foo/1.0' is denied $ ml No Modulefiles Currently Loaded.

module-forbid statements can be coupled with module-hidestatements to hide modules in addition to forbid their use.module-forbid supports the --not-user, --not-group,--before and --after options to still allow some users or forbid modules before or after a given date time.

An additional error message can be defined with the --message option to guide for instance users when they try to evaluate a forbidden module:

$ cat /path/to/modulefiles/bar/.modulerc #%Module4.6 module-forbid --message {Software bar/1.0 is decommissioned, please now use
bar/2.0} --after 2020-09-01 bar/1.0 $ ml bar/1.0 ERROR: Access to module 'bar/1.0' is denied Software bar/1.0 is decommissioned, please now use bar/2.0

When an evaluated module will soon be forbidden, a message is returned to the user to warn him/her of the near limit. An additional warning message can also be defined here with the --nearly-message option to guide users.

$ cat /path/to/modulefiles/qux/.modulerc #%Module4.6 module-forbid --nearly-message {Version 1.0 will soon expire, please now use
version 2.0} --after 2020-09-15 qux/1.0 $ date Tue 08 Sep 2020 06:49:43 AM CEST $ ml qux/1.0 Loading qux/1.0 WARNING: Access to module will be denied starting '2020-09-15' Version 1.0 will soon expire, please now use version 2.0

The range of time the nearly forbidden warning appears can be controlled with the nearly_forbidden_days configuration option, whose value equals to the number of days prior the module starts to be forbidden. This configuration is set to 14 (days) by default and this value can be controlled at configure time with--with-nearly-forbidden-days option. When thenearly_forbidden_days configuration is set through theconfig sub-command, the MODULES_NEARLY_FORBIDDEN_DAYSenvironment variable is set.

Tracing module execution

The trace verbosity is introduced between the verbose and debuglevels to report details on module searches, resolutions, selections and evaluations. Trace mode can be enabled by setting the verbosityconfig to the trace value or by using the -T/--tracecommand-line switches.

To specifically render trace messages, the tr key is added to the color palette with a default value of 2 (decreased intensity).

$ ml -T foo Evaluate modulerc: '/path/to/modulefiles/.modulerc' Get modules: {foo} matching 'foo' in '/path/to/modulefiles' Resolve: 'foo' into 'bar' Get modules: {bar bar/1.0} matching 'bar' in '/path/to/modulefiles' Select module: 'bar/1.0' (/path/to/modulefiles/bar/1.0) matching 'bar/1.0'

Loading bar/1.0 Evaluate modulefile: '/path/to/modulefiles/bar/1.0' as 'bar/1.0'

v4.5

This new version is backward-compatible with previous version 4 releases. It fixes bugs but also introduces new functionalities that are described in this section. See the 4.5 release notes for a complete list of the changes between Modules v4.4 and v4.5.

ml command

The ml command is added to Modules. ml is a frontend to the modulecommand that reduces the number of characters to type to trigger module actions.

With no argument provided ml is equivalent to module list, ml foocorresponds to module load foo and ml -foo means module unload foo:

$ ml foo $ ml Currently Loaded Modulefiles:

  1. foo/2 $ ml -foo $ ml No Modulefiles Currently Loaded.

Multiple modules to either load or unload can be combined on a single command. The unloads are first processed then the loads.

ml accepts all command-line switches and sub-commands accepted bymodule command:

$ ml avail -t foo foo/1 foo/2

This handy interface has been originally developed by the Lmod project. Having this command line interface also supported on Modules helps to provide a similar user experience whatever the module implementation used.

JSON format output

The -j and --json command line switches are added for the avail,list, savelist, whatis and search module sub-commands. When set, the output result of these sub-commands is rendered in JSON format:

$ module avail --json bar | python -mjson.tool { "/path/to/modulefiles": { "bar/2.3": { "name": "bar/2.3", "pathname": "/path/to/modulefiles/bar/2.3", "symbols": [ "default" ], "type": "modulefile" }, "bar/3.4": { "name": "bar/3.4", "pathname": "/path/to/modulefiles/bar/3.4", "symbols": [], "type": "modulefile" } } } $ ml whatis -j foo/1.2.3 | python -mjson.tool { "/path/to/modulefiles": { "foo/1.2.3": { "name": "foo/1.2.3", "whatis": [ "The foo/1.2.3 modulefile" ] } } }

Improved Windows support

A new option to the ./configure script named--enable-windows-support is introduced to install additional files relative to the enablement of Modules on the Windows platform. When set, this option installs module.cmd, ml.cmd and envml.cmd scripts inbindir and initialization script cmd.cmd in initdir. With these four files the Modules installation may be used from either a Unix or a Windows platform.

module.cmd, ml.cmd and envml.cmd scripts respectively provide themodule, ml and envml commands for Windows cmd terminal shell, relying on modulecmd.tcl script which was already able to produce shell code for this Windows shell. Initialization script cmd.cmd adds the directory of module.cmd, ml.cmd and envml.cmd to PATH.

These Windows-specific files are relocatable: module.cmd, ml.cmd andenvml.cmd scripts expect to find initialization script cmd.cmd in theinit directory next to them (to setup Modules-specific variables in current environment) and cmd.cmd expects modulecmd.tcl to be found inlibexec directory and the 3 commands in bin directory next to it.

Starting from this 4.5 release a distribution zipball is published to install Modules on Windows. This zip archive ships an install and an uninstall scripts (INSTALL.bat and UNINSTALL.bat). The zipball can be built locally from Modules sources by running make dist-win.

The Installing Modules on Windows document describes how to install Modules on Windows from the distribution zipball.

Error stack trace

Error messages will now embed a stack trace for unknown errors to help localize the root cause of issues. This change applies to modulefile evaluation:

Loading foo/1.2 Module ERROR: add-path cannot handle path equals to separator string while executing "append-path PATH :" (file "/path/to/modulefiles/foo/1.2" line 24) Please contact <root@localhost>

A stack trace is also returned when an unknown error occurs inmodulecmd.tcl script, which facilitates issue report and analysis:

$ module load bar ERROR: invalid command name "badcommand" while executing "badcommand" (procedure "module" line 14) invoked from within "module load bar" ("eval" body line 1) invoked from within "eval $execcmdlist" Please report this issue at https://github.com/cea-hpc/modules/issues

Automatic default and latest symbolic versions

When the implicit default mechanism and the Advanced module version specifiers are both enabled, a default and a latest symbolic versions are automatically defined for each module name.

This new feature gives the ability to select the highest version available for a module, without knowing beforehand this version name:

$ module load -v foo@latest Loading foo/1.10

The symbolic versions are automatically defined unless a symbolic version, an alias or a regular module version already exists for these default orlatest version names.

v4.4

This new version is backward-compatible with previous version 4 releases. It fixes bugs but also introduces new functionalities that are described in this section. See the 4.4 release notes for a complete list of the changes between Modules v4.3 and v4.4.

Warning

Modules configuration option handling has been reworked internally to provide a unified way for all options to get initialized, retrieved or set. Existing site-specific configuration script should be reviewed to make use of the new getConf, setConf, unsetConfand lappendConf procedures to manipulate configuration options.

Specify modules in a case insensitive manner

The ability to match module name in a case insensitive manner has been added. This feature can be enabled at different level with the following values set to the icase configuration option:

It can help for instance to load a module without knowing the case used to name its relative modulefile:

$ module config icase always $ module load -v mysoftware Loading MySoftware/1.0

Insensitive case match activation can be controlled at configure time with the --with-icase option, which could be passed any of the above activation levels. This option could be superseded with the MODULES_ICASE environment variable, which could be set through the config sub-command with theicase option. Command-line switch --icase supersedes in turns any other icase configurations. When this command-line switch is passed, icasemode equals always.

Extended default

The extended default mechanism has been introduced to help selecting a module when only the first numbers in its version are specified. Starting portion of the version, part separated from the rest of the version string by a .character, could be used to refer to a more precise version number.

This mechanism is activated through the new configuration optionextended_default. It enables to refer to a module named foo/1.2.3 asfoo/1.2 or foo/1:

$ module config extended_default 1 $ module load -v foo/1 Loading foo/1.2.3

When multiple versions match partial version specified and only one module should be selected, the default version (whether implicitly or explicitly defined) among matches is returned. The following example shows thatfoo/1.1.1, the foo module default version, is selected when it matches query. Elsewhere the highest version (also called the latest version or the implicit default) among matching modules is returned:

$ module av foo --------------- /path/to/modulefiles --------------- foo/1.1.1(default) foo/1.2.1 foo/1.10 foo/1.1.10 foo/1.2.3 $ module load -v foo/1.1 Loading foo/1.1.1 $ module purge $ module load -v foo/1.2 Loading foo/1.2.3 $ module purge $ module load -v foo/1 Loading foo/1.1.1

In case implicit_default option is disabled and no explicit default is found among matches, an error is returned:

$ module config implicit_default 0 $ module load -v foo/1.2 ERROR: No default version defined for 'foo/1.2'

When it is enabled, extended default applies everywhere a module could be specified, which means it could be used with any module sub-command or any modulefile Tcl command receiving a module specification as argument. It may help for instance to declare dependencies between modules:

$ module show bar/3

/path/to/modulefiles/bar/3.4:

prereq foo/1.2

$ module load --auto bar/3 Loading bar/3.4 Loading requirement: foo/1.2.3

Extended default activation can be controlled at configure time with the--enable-extended-default option. This option could be superseded with theMODULES_EXTENDED_DEFAULT environment variable, which could be set through the config sub-command with the extended_default option.

Advanced module version specifiers

The ability to specify finer constraints on module version has been added to Modules. It enables to filter the module selection to a given version list or range by specifying after the module name a version constraint prefixed by the@ character.

This new feature leverages the version specifier syntax of the Spackpackage manager as this syntax covers all the needs for a fine-grained selection of module versions. It copes very well with command-line typing, by avoiding characters having a special meaning on shells. Moreover the users of Spack that also are users of Modules may already be familiar with this syntax.

The mechanism introduced here is called _advanced module version specifier_and it can be activated through the new configuration optionadvanced_version_spec. Constraints can be expressed to refine the selection of module version to:

This new feature enables for instance to list available versions of modulefoo higher or equal to 1.2:

$ module config advanced_version_spec 1 $ module av foo --------------- /path/to/modulefiles --------------- foo/1.1.1(default) foo/1.2.1 foo/1.10 foo/1.1.10 foo/1.2.3 $ module av foo@1.2: --------------- /path/to/modulefiles --------------- foo/1.2.1 foo/1.2.3 foo/1.10

Then choose to load for instance a version higher than or equal to 1.2 and less than or equal to 1.3. Default version is selected if it corresponds to a version included in the range, elsewhere the highest version (also called latest version or implicit default) is selected:

$ module load -v foo@1.2:1.3 Loading foo/1.2.3

In case implicit_default option is disabled and no explicit default is found among version specifier matches, an error is returned:

$ module config implicit_default 0 $ module load -v foo@1.2:1.3 ERROR: No default version defined for 'foo@1.2:1.3'

When advanced module version specifier is enabled, it applies everywhere a module could be specified, which means it could be used with any module sub-command or any modulefile Tcl command receiving a module specification as argument. It may help for instance to declare smoother dependencies between modules:

$ module show bar@:2

/path/to/modulefiles/bar/2.3:

prereq foo@1.1.10,1.2.1

$ module load --auto bar@:2 Loading bar/2.3 Loading requirement: foo/1.2.1

Advanced specification of single version or list of versions may benefit from the activation of the Extended default mechanism (range of versions natively handles abbreviated versions):

$ module config extended_default 1 $ module load -v foo@1.2 Loading foo/1.2.3 $ module unload -v foo @1.2,1.5 Unloading foo/1.2.3

Advanced module version specifier activation can be controlled at configure time with the --enable-advanced-version-spec option. This option could be superseded with the MODULES_ADVANCED_VERSION_SPEC environment variable, which could be set through the config sub-command with theadvanced_version_spec option.

v4.3

This new version is backward-compatible with previous version 4 releases. It fixes bugs but also introduces new functionalities that are described in this section. See the 4.3 release notes for a complete list of the changes between Modules v4.2 and v4.3.

Modulepath rc file

A .modulerc file found at the root of an enabled modulepath directory is now evaluated when modulepath is walked through to locate modulefiles. This modulepath rc file gives for instance the ability to define module alias whose name does not correspond to any module directory in this modulepath. Thus this kind of module alias would not be found unless if it is defined at the modulepath global scope.

Further I/O operations optimization

Additional work has been performed to save a significant number of filesystem I/O operations made to search and evaluate modulefiles.

When fully read, the content of a modulefile is now cached in memory to avoid new I/O operations in case this modulefile should be read one more time during the same module command evaluation.

Except for path, paths, list, avail and aliases module commands always fully read a modulefile whether its full content is needed or just its header to verify its validity. This way modulefiles are only read once on commands that first check modulefile validity then read again valid files to get their full content.

Last but not least, Modules Tcl extension library is introduced to extend the Tcl language in order to provide more optimized I/O commands to read a file or a directory content than native Tcl commands do. This library is built and enabled in modulecmd.tcl script with --enable-libtclenvmodulesconfigure argument (it is enabled by default). As this library is written in C, it must be compiled and --with-tcl or --with-tclinclude configure arguments may be used to indicate where to find Tcl development files.

Modules Tcl extension library greatly reduces the number of filesystem I/O operations by removing unneeded ioctl, fcntl and lstat system calls done (by Tcl open command) to read each modulefile. Directory content read is also improved by fetching hidden and regular files in one pass. Moreover .modulerc and .version read access is tested only if these files are found in the directory.

Colored output

The ability to graphically enhance some part of the produced output has been added to improve readability. Among others, error, warning and info message prefixes can be colored as well as modulepath, module alias and symbolic version.

Color mode can be set to never, auto or always. When color mode is set to auto, output is colored only if the standard error output channel is attached to a terminal.

Default color mode could be controlled at configure time with the--enable-color and the --disable-color option, which respectively correspond to the auto and never color mode. This default mode could be superseded with the CLICOLOR, CLICOLOR_FORCE and MODULES_COLORenvironment variables and the --color command-line switch.

Color to apply to each element can be controlled with the MODULES_COLORSenvironment variable or the --with-dark-background-colors and--with-light-background-colors configure options. These variable and options take as value a colon-separated list in the same fashion LS_COLORSdoes. In this list, output item that should be highlighted is designated by a key which is associated to a Select Graphic Rendition (SGR) code.

The MODULES_TERM_BACKGROUND environment variable and the--with-terminal-background configure option help Modules to determine if the color set for dark background or the color set for light background should be used to color output in case no specific color set is defined with theMODULES_COLORS variable.

Output items able to be colorized and their relative key are: highlighted element (hi), debug information (db), tag separator (se); Error (er), warning (wa), module error (me) and info (in) message prefixes; Modulepath (mp), directory (di), module alias (al), module symbolic version (sy), module default version (de) and modulefile command (cm).

For instance the default color set for a terminal with dark background is defined to:

hi=1:db=2:se=2:er=91:wa=93:me=95:in=94:mp=1;94:di=94:al=96:sy=95:de=4:cm=92

When colored output is enabled and a specific graphical rendition is defined for module default version, the default symbol is omitted and instead the defined graphical rendition is applied to the relative modulefile. When colored output is enabled and a specific graphical rendition is defined for module alias, the @ symbol is omitted.

CLICOLOR and CLICOLOR_FORCE environment variables are also honored to define color mode. The never mode is set if CLICOLOR equals to 0. If CLICOLOR is set to another value, it corresponds to the auto mode. The always mode is set if CLICOLOR_FORCE is set to a value different than 0. Color mode set with these two variables is superseded by mode set with MODULES_COLOR environment variable.

Configure modulecmd with config sub-command

The config sub-command has been added to module to help getting or setting the modulecmd.tcl options. With no additional command-line argument, this sub-command reports the current value of all existing options with a mention to indicate if this value has been overridden from a command-line switch or from an environment variable.

See the description of this sub-command in the module man page for a complete reference on existing configuration options.

Most of the options can be altered by passing the option name and a value to the sub-command. Setting an option by this mean overrides its default value, set at installation time in modulecmd.tcl script, by defining the environment variable which supersedes this default.:

$ module config auto_handling 1 $ module config auto_handling Modules Release 4.3.0 (2019-07-26)

Setting options with module config could be done in the Modules initialization RC file to change default value of options when modulecommand is initialized.

When command-line switch --reset and an option name is passed to theconfig sub-command, it restores default value for configuration option by unsetting related environment variable.

With command-line switch --dump-state, the config sub-command reports, in addition to currently set options, the current state of modulecmd.tclscript and Modules-related environment variables. Providing the output of themodule config --dump-state command when submitting an issue to the Modules project will help to analyze the situation.

Control module command verbosity

The ability to control message verbosity has been added so module command can be configured whether it should display more or less information. Available verbosity levels from the least to the most verbose are:

Default verbosity level can be controlled at configure time with the--with-verbosity option, which could be passed any of the above level names. This default verbosity level could be superseded with theMODULES_VERBOSITY environment variable, which could be set through theconfig sub-command with the verbosity option. Command-line switches--silent, --verbose and --debug supersede in turns any other verbosity configuration to respectively set module command silent, verbose or in debug mode.

Other new sub-commands, command-line switches and environment variables

v4.2

This new version is backward-compatible with previous version 4 releases. It fixes bugs but also introduces new functionalities that are described in this section. See the 4.2 release notes for a complete list of the changes between Modules v4.1 and v4.2.

Modulefile conflict constraints consistency

With the conflict modulefile command, a given modulefile can list the other modulefiles it conflicts with. To load this modulefile, the modulefiles it conflicts with cannot be loaded.

This constraint was until now satisfied when loading the modulefile declaring the conflict but it vanished as soon as this modulefile was loaded. In the following example a modulefile declares a conflict with b:

$ module load b a WARNING: a cannot be loaded due to a conflict. HINT: Might try "module unload b" first. $ module list Currently Loaded Modulefiles:

  1. b $ module purge $ module load a b $ module list Currently Loaded Modulefiles:
  2. a 2) b

Consistency of the declared conflict is now ensured to satisfy this constraint even after the load of the modulefile declaring it. This is achieved by keeping track of the conflict constraints of the loaded modulefiles in an environment variable called MODULES_LMCONFLICT:

$ module load a b ERROR: WARNING: b cannot be loaded due to a conflict. HINT: Might try "module unload a" first. $ module list Currently Loaded Modulefiles:

  1. a

An environment variable is used to keep track of this conflict information to proceed the same way than used to keep track of the loaded modulefiles with the LOADEDMODULES environment variable.

In case a conflict constraint toward a modulefile is set by an already loaded modulefile, loading the conflicting modulefile will lead to a load evaluation attempt in order for this modulefile to get the chance to solve the constraint violation. If at the end of the load evaluation, the conflict has not been solved, modulefile load will be discarded.

Warning

On versions 4.2.0 and 4.2.1, a conflict constraint set by an already loaded modulefile forbade the load of the conflicting modulefile. This has been changed starting version 4.2.2 to better cope with behaviors of previous Modules version: an evaluation attempt of the conflicting modulefile is made to give it the opportunity to solve this conflict by using module unload modulefile command.

Modulefile prereq constraints consistency

With the prereq modulefile command, a given modulefile can list the other modulefiles it pre-requires. To load this modulefile, the modulefiles it pre-requires must be loaded prior its own load.

This constraint was until now satisfied when loading the modulefile declaring the prereq but, as for the declared conflict, it vanished as soon as this modulefile was loaded. In the following example c modulefile declares a prereq on a:

$ module load c WARNING: c cannot be loaded due to missing prereq. HINT: the following module must be loaded first: a $ module list No Modulefiles Currently Loaded. $ module load a c $ module list Currently Loaded Modulefiles:

  1. a 2) c $ module unload a $ module list Currently Loaded Modulefiles:
  2. c

Consistency of the declared prereq is now ensured to satisfy this constraint even after the load of the modulefile declaring it. This is achieved, like for the conflict consistency, by keeping track of the prereq constraints of the loaded modulefiles in an environment variable calledMODULES_LMPREREQ:

$ module load a c $ module list Currently Loaded Modulefiles:

  1. a 2) c $ module unload a ERROR: WARNING: a cannot be unloaded due to a prereq. HINT: Might try "module unload c" first. $ module list Currently Loaded Modulefiles:
  2. a 2) c

By-passing module defined constraints

The ability to by-pass a conflict or a prereq constraint defined by modulefiles is introduced with the --force command line switch (-f for short notation) for the load, unload and switch sub-commands.

With this new command line switch, a given modulefile is loaded even if it conflicts with other loaded modulefiles or even if the modulefiles it pre-requires are not loaded. Some example reusing the same modulefiles a,b and c than above:

$ module load b $ module load --force a WARNING: a conflicts with b $ module list Currently Loaded Modulefiles:

  1. b 2) a $ module purge $ module load --force c WARNING: c requires a loaded $ module list Currently Loaded Modulefiles:
  2. c

--force also enables to unload a modulefile required by another loaded modulefiles:

$ module load a c $ module list Currently Loaded Modulefiles:

  1. a 2) c $ module unload --force a WARNING: a is required by c $ module list Currently Loaded Modulefiles:
  2. c

In a situation where some of the loaded modulefiles have unsatisfied constraints corresponding to the prereq and conflict they declare, thesave and reload sub-commands do not perform and return an error.

Automated module handling mode

An automatic management of the dependencies between modulefiles has been added and it is called automated module handling mode. This new mode consists in additional actions triggered when loading or unloading a modulefile to satisfy the constraints it declares.

When loading a modulefile, following actions are triggered:

When unloading a modulefile, following actions are triggered:

In case a loaded modulefile has some of its declared constraints unsatisfied (pre-required modulefile not loaded or conflicting modulefile loaded for instance), this loaded modulefile is excluded from the automatic reload actions described above.

For the specific case of the switch sub-command, where a modulefile is unloaded to then load another modulefile. Dependent modulefiles to Unload are merged into the Dependent modulefiles to Reload that are reloaded after the load of the switched-to modulefile.

This automated module handling mode integrates concepts (like the Dependent Reload mechanism) of the Flavours extension, which was designed for Modules compatibility version. As a whole, automated module handling mode can be seen as a generalization and as an expansion of the Flavours concepts.

This new feature can be controlled at build time with the--enable-auto-handling configure option. This default configuration can be superseded at run-time with the MODULES_AUTO_HANDLING environment variable or the command line switches --auto and --no-auto.

By default, automated module handling mode is disabled and will stay so until the next major release version (5.0) where it will be enabled by default. This new feature is currently considered experimental and the set of triggered actions will be refined over the next feature releases.

Consistency of module load/unload commands in modulefile

With the module load modulefile command, a given modulefile can automatically load a modulefile it pre-requires. Similarly with the module unload modulefile command, a given modulefile can automatically unload a modulefile it conflicts with.

Both commands imply additional actions on the loaded environment (loading or unloading extra modulefiles) that should cope with the constraints defined by the loaded environment.

Additionally module load and module unload modulefile commands express themselves constraints on loaded environment that should stay satisfied to ensure consistency.

To ensure the consistency of module load modulefile command once the modulefile defining it has been loaded, this command is assimilated to aprereq command. Thus the defined constraint is recorded in theMODULES_LMPREREQ environment variable. Same approach is used for module unload modulefile command which is assimilated to a conflict command. Thus the defined constraint is recorded in the MODULES_LMCONFLICTenvironment variable.

To ensure the consistency of the loaded environment, the additional actions of the module load and module unload modulefile commands have been adapted in particular situations:

Please note that loading and unloading results may differ than from previous Modules version now that consistency is checked:

Modulefile alias and symbolic modulefile name consistency

With the module-alias and module-version modulefile commands, alternative names can be given to a modulefile. When these names are used to load for instance a modulefile, they are resolved to the modulefile they target which is then processed for the load action.

Until now, the alias and symbolic version names were correctly resolved for the load and unload actions and also for the querying sub-commands (like avail or whatis). However this alternative name information vanishes once the modulefile it resolves to is loaded. As a consequence there was no consistency over these alternative designations. In the following example f modulefile declares a conflict on e alias which resolves tod modulefile:

$ module load e $ module list Currently Loaded Modulefiles:

  1. d $ module info-loaded e $ module load f $ module list Currently Loaded Modulefiles:
  2. d 2) f

Consistency of the alternative names set on a modulefile with module-aliasand module-version commands is now ensured to enable modulefile commandsprereq, conflict, is-loaded and module-info loaded using these alternative designations as argument. This consistency is achieved, like for the conflict and prereq consistencies, by keeping track of the alternative names of the loaded modulefiles in an environment variable calledMODULES_LMALTNAME:

$ module load e $ module list Currently Loaded Modulefiles:

  1. d $ module info-loaded e d $ module load f WARNING: f cannot be loaded due to a conflict. HINT: Might try "module unload e" first. $ module list Currently Loaded Modulefiles:
  2. d

Environment variable change through modulefile evaluation context

All environment variable edition commands (setenv, unsetenv,append-path, prepend-path and remove-path) have been updated to:

Exception is made for the whatis evaluation mode: environment variables targeted by variable edition commands are not set to the defined value in the evaluation context during this whatis evaluation. These variables are only initialized to an empty value if undefined. This exception is made to save performances on this global evaluation mode.

Express Modules compatibility of modulefile with versioned magic cookie

Any modulefile should start with the #%Module file signature (also called the Modules magic cookie) and sometimes a version number may be placed right after this string. Until now this version number corresponded to a modulefile format version but it was never checked.

Starting with this new Modules release, this version number reflects the minimum version of Modules required to interpret the modulefile. If the version number is set along the magic cookie string it is now checked and the modulefile is interpreted only if Modules version is greater or equal to this version number. For instance, if a modulefile begins with the #%Module4.3string, it can only be evaluated by Modules version 4.3 and above. Elsewhere the modulefile is ignored like files without the #%Module magic cookie set.

Improved module message report

Module sub-commands like load, unload or switch, may perform multiple load or unload modulefile evaluations in a row. Also these kind of evaluation modes may sometimes trigger additional load or unload evaluations, when for instance a modulefile contains a module load command.

To improve the readability of the module messages produced relatively to a load or an unload evaluation, these messages are now stacked under a_Loading_ or an Unloading message block that gathers all the messages produced for a given modulefile evaluation:

$ module load --no-auto foo Loading foo/1.2 ERROR: foo/1.2 cannot be loaded due to missing prereq. HINT: the following module must be loaded first: bar/4.5

In addition, foreground load, unload, switch and restoreactions (i.e., asked on the command-line) now report a summary of the additional load and unload evaluations that were eventually triggered in the process:

$ module load --auto foo Loading foo/1.2 Loading requirement: bar/4.5

New modulefile commands

2 new modulefile Tcl commands have been introduced:

v4.1

This new version is backward-compatible with previous version 4 releases. It fixes bugs but also introduces new functionalities that are described in this section. See the 4.1 release notes for a complete list of the changes between Modules v4.0 and v4.1.

Virtual modules

A virtual module stands for a module name associated to a modulefile. The modulefile is the script interpreted when loading or unloading the virtual module which appears or can be found with its virtual name.

The module-virtual modulefile command is introduced to give the ability to define these virtual modules. This new command takes a module name as first argument and a modulefile location as second argument:

module-virtual app/1.2.3 /path/to/virtualmod/app

With this feature it is now possible to dynamically define modulefiles depending on the context.

Extend module command with site-specific Tcl code

module command can now be extended with site-specific Tcl code. modulecmd.tcl now looks at a siteconfig.tcl file in anetcdir defined at configure time (by default $prefix/etc). If it finds this Tcl script file, it is sourced within modulecmd.tcl at the beginning of the main procedure code.

siteconfig.tcl enables to supersede any global variable or procedure definitions made in modulecmd.tcl with site-specific code. A module sub-command can for instance be redefined to make it fit local needs without having to touch the main modulecmd.tcl.

Quarantine mechanism to protect module execution

To protect the module command run-time environment from side effect coming from the current environment definition a quarantine mechanism is introduced. This mechanism, sets within module function definition and shell initialization script, modifies the modulecmd.tcl run-time environment to sanitize it.

The mechanism is piloted by environment variables. First of allMODULES_RUN_QUARANTINE, a space-separated list of environment variable names. Every variable found in MODULES_RUN_QUARANTINE will be set in quarantine during the modulecmd.tcl run-time. Their value will be set empty or set to the value of the corresponding MODULES_RUNENV_<VAR>environment variable if defined. Once modulecmd.tcl is started it restores quarantine variables to their original values.

MODULES_RUN_QUARANTINE and MODULES_RUNENV_<VAR> environment variables can be defined at build time by using the following configure option:

--with-quarantine-vars='VARNAME[=VALUE] ...'

Quarantine mechanism is available for all supported shells except cshand tcsh.

Module function to return value in scripting languages

On Tcl, Perl, Python, Ruby, CMake and R scripting shells, module function was not returning value and until now an occurred error led to raising a fatal exception.

To make module function more friendly to use on these scripting shells it now returns a value. False in case of error, true if everything goes well.

As a consequence, returned value of a module sub-command can be checked. For instance in Python:

if module('load', 'foo'):

success

else:

failure

New modulefile commands

4 new modulefile Tcl commands have been introduced:

Multiple collections, paths or modulefiles can be passed respectively tois-saved, is-used and is-avail in which case true is returned if at least one argument matches condition (acts as a OR boolean operation). No argument may be passed to is-loaded, is-saved and is-usedcommands to return if anything is respectively loaded, saved or used.

If no loaded modulefile matches the module-info loaded query, an empty string is returned.

New module sub-commands

Modulefile-specific commands are sometimes wished to be used outside of a modulefile context. Especially for the commands managing path variables or commands querying current environment context. So the following modulefile-specific commands have been made reachable as module sub-commands with same arguments and properties as if called from within a modulefile:

The is-loaded sub-command returns a boolean value. Small Python example:

if module('is-loaded', 'app'): print 'app is loaded' else: print 'app not loaded'

info-loaded returns a string value and is the sub-command counterpart of the module-info loaded modulefile command:

$ module load app/0.8 $ module info-loaded app app/0.8

v4.0

Major evolution occurs with this v4.0 release as the traditional _module_command implemented in C is replaced by the native Tcl version. This full Tcl rewrite of the Modules package was started in 2002 and has now reached maturity to take over the binary version. This flavor change enables to refine and push forward the module concept.

This document provides an outlook of what is changing when migrating from v3.2 to v4.0 by first describing the introduced new features. Both v3.2 and v4.0 are quite similar and transition to the new major version should be smooth. Slights differences may be noticed in a few use-cases. So the second part of the document will help to learn about them by listing the features that have been discontinued in this new major release or the features where a behavior change can be noticed.

New features

On its overall this major release brings a lot more robustness to the_module_ command with now more than 4000 non-regression tests crafted to ensure correct operations over the time. This version 4.0 also comes with fair amount of improved functionalities. The major new features are described in this section.

Additional shells supported

Modules v4 introduces support for fish, lisp, tcl and Rcode output.

Non-zero exit code in case of error

All module sub-commands will now return a non-zero exit code in case of error whereas Modules v3.2 always returned zero exit code even if issue occurred.

Output redirect

Traditionally the module command output text that should be seen by the user on stderr since shell commands are output to stdout to change shell's environment. Now on sh, bash, ksh, zsh and fish shells, output text is redirected to stdout after shell command evaluation if shell is in interactive mode.

Filtering avail output

Results obtained from the avail sub-command can now be filtered to only get the default version of each module name with use of the --defaultor -d command line switch. Default version is either the explicitly set default version or the highest numerically sorted modulefile or module alias if no default version set.

It is also possible to filter results to only get the highest numerically sorted version of each module name with use of the --latest or -Lcommand line switch.

Extended support for module alias and symbolic version

Module aliases are now included in the result of the avail, whatisand apropos sub-commands. They are displayed in the module path section where they are defined or in a global/user modulerc section for aliases set in user's or global modulerc file. A @ symbol is added in parenthesis next to their name to distinguish them from modulefiles.

Search may be performed with an alias or a symbolic version-name passed as argument on avail, whatis and apropos sub-commands.

Modules v4 resolves module alias or symbolic version passed to unloadcommand to then remove the loaded modulefile pointed by the mentioned alias or symbolic version.

A symbolic version sets on a module alias is now propagated toward the resolution path to also apply to the relative modulefile if it still correspond to the same module name.

Hiding modulefiles

Visibility of modulefiles can be adapted by use of file mode bits or file ownership. If a modulefile should only be used by a given subset of persons, its mode an ownership can be tailored to provide read rights to this group of people only. In this situation, module only reports the modulefile, during anavail command for instance, if this modulefile can be read by the current user.

These hidden modulefiles are simply ignored when walking through the modulepath content. Access issues (permission denied) occur only when trying to access directly a hidden modulefile or when accessing a symbol or an alias targeting a hidden modulefile.

Improved modulefiles location

When looking for an implicit default in a modulefile directory, aliases are now taken into account in addition to modulefiles and directories to determine the highest numerically sorted element.

Modules v4 resolves module alias or symbolic version when it points to a modulefile located in another modulepath.

Access issues (permission denied) are now distinguished from find issues (cannot locate) when trying to access directly a directory or a modulefile as done on load, display or whatis commands. In addition, on this kind of access not readable .modulerc or .version files are ignored rather producing a missing magic cookie error.

Module collection

Modules v4 introduces support for module collections. Collections describe a sequence of module use then module load commands that are interpreted by Modules to set the user environment as described by this sequence. When a collection is activated, with the restore sub-command, modulepaths and loaded modules are unused or unloaded if they are not part or if they are not ordered the same way as in the collection.

Collections are generated by the save sub-command that dumps the current user environment state in terms of modulepaths and loaded modules. By default collections are saved under the $HOME/.module directory. Collections can be listed with savelist sub-command, displayed with saveshowand removed with saverm.

Collections may be valid for a given target if they are suffixed. In this case these collections can only be restored if their suffix correspond to the current value of the MODULES_COLLECTION_TARGET environment variable. Saving collection registers the target footprint by suffixing the collection filename with .$MODULES_COLLECTION_TARGET.

Path variable element counter

Modules 4 provides path element counting feature which increases a reference counter each time a given path entry is added to a given path-like environment variable. As consequence a path entry element is removed from a path-like variable only if the related element counter is equal to 1. If this counter is greater than 1, path element is kept in variable and reference counter is decreased by 1.

This feature allows shared usage of particular path elements. For instance, modulefiles can append /usr/local/bin to PATH, which is not unloaded until all the modulefiles that loaded it unload too.

Optimized I/O operations

Substantial work has been done to reduce the number of I/O operations done during global modulefile analysis commands like avail orwhatis. stat, open, read and close I/O operations have been cut down to the minimum required when walking through the modulepath directories to check if files are modulefiles or to resolve module aliases.

Interpretation of modulefiles and modulerc are handled by the minimum required Tcl interpreters. Which means a configured Tcl interpreter is reused as much as possible between each modulefile interpretation or between each modulerc interpretation.

Sourcing modulefiles

Modules 4 introduces the possibility to source a modulefile rather loading it. When it is sourced, a modulefile is interpreted into the shell environment but then it is not marked loaded in shell environment which differ from load sub-command.

This functionality is used in shell initialization scripts once modulefunction is defined. There the etc/modulerc modulefile is sourced to setup the initial state of the environment, composed of _module use_and module load commands.

Removed features and substantial behavior changes

Following sections provide list of Modules v3.2 features that are discontinued on Modules v4 or features with a substantial behavior change that should be taken in consideration when migrating to v4.

Package initialization

MODULESBEGINENV environment snapshot functionality is not supported anymore on Modules v4. Modules collection mechanism should be used instead tosave and restore sets of enabled modulepaths and loaded modulefiles.

Command line switches

Some command line switches are not supported anymore on v4.0. When still using them, a warning message is displayed and the command is ran with these unsupported switches ignored. Following command line switches are concerned:

Module sub-commands

During an help sub-command, Modules v4 does not redirect output made on stdout in ModulesHelp Tcl procedure to stderr. Moreover when runninghelp, version 4 interprets all the content of the modulefile, then call the ModulesHelp procedure if it exists, whereas Modules 3.2 only interprets the ModulesHelp procedure and not the rest of the modulefile content.

When load is asked on an already loaded modulefiles, Modules v4 ignores this new load order whereas v3.2 refreshed shell alias definitions found in this modulefile.

When switching on version 4 an old modulefile by a new one, no error is raised if old modulefile is not currently loaded. In this situation v3.2 threw an error and abort switch action. Additionally onswitch sub-command, new modulefile does not keep the position held by old modulefile in loaded modules list on Modules v4 as it was the case on v3.2. Same goes for path-like environment variables: replaced path component is appended to the end or prepended to the beginning of the relative path-like variable, not appended or prepended relatively to the position hold by the swapped path component.

During a switch command, version 4 interprets the swapped-out modulefile in unload mode, so the sub-modulefiles loaded, with module loadorder in the swapped-out modulefile are also unloaded during the switch.

Modules 4 provides path element counting feature which increases a reference counter each time a given path entry is added to a given environment variable. This feature also applies to the MODULEPATH environment variable. As consequence a modulepath entry element is removed from the modulepath enabled list only if the related element counter is equal to 1. When unusing a modulepath if its reference counter is greater than 1, modulepath is kept enabled and reference counter is decreased by 1.

On Modules 3.2 paths composing the MODULEPATH environment variable may contain reference to environment variable. These variable references are resolved dynamically when MODULEPATH is looked at during module sub-command action. This feature has been discontinued on Modules v4.

Following Modules sub-commands are not supported anymore on v4.0:

Modules specific Tcl commands

Modules v4 provides path element counting feature which increases a reference counter each time a given path entry is added to a given environment variable. As a consequence a path entry element is not always removed from a path-like variable when calling to remove-path or calling toappend-path or append-path at unloading time. The path element is removed only if its related element counter is equal to 1. If this counter is greater than 1, path element is kept in variable and reference counter is decreased by 1.

On Modules v4, module-info mode returns during an unload sub-command the unload value instead of remove on Modules v3.2. However if mode_is tested against remove value, true will be returned. During a switchsub-command on Modules v4, unload then load is returned instead ofswitch1 then switch2 then switch3 on Modules v3.2. However if_mode is tested against switch value, true will be returned.

When using set-alias, Modules v3.2 defines a shell function when variables are in use in alias value on Bourne shell derivatives, Modules 4 always defines a shell alias never a shell function.

Some Modules specific Tcl commands are not supported anymore on v4.0. When still using them, a warning message is displayed and these unsupported Tcl commands are ignored. Following Modules specific Tcl commands are concerned:

Further reading

To get a complete list of the differences between Modules v3.2 and v4, please read the Changes between versions document.

A significant number of issues reported for v3.2 have been closed on v4. List of these closed issues can be found at:

https://github.com/cea-hpc/modules/milestone/1?closed=1