Skip to main content

Software

Mods and rockers

29 September 2016
RAF Roundel
RAF Roundel

Modules are used on many supercomputers since it allows multiple versions of software to be easily chosen to be loaded into a users environment. We would now like to show how modules work on our supercomputer since we apply some tricks to try and make maintaining them easier and make them “rock” even more.

What are modules?

First lets just confirm what we are talking about. The software for modules can be found at http://modules.sourceforge.net/ but their are other similar packages such as Lmod. Modules are controlled by their modulefiles and are a series of instructions on what to do when a user wants to apply an action to a module (usually to load the module). For example we could have a module called “helloworld” which loads an executable called “hello.exe” into my environment by setting the PATH. So the module “helloworld” (maybe called “1.0” to signal the version) would contain:

#%Module1.0
prepend-path PATH /usr/local/helloworld/bin

A user could then type:

$ module load helloworld

And then the user could run “hello.exe“.

A user could then unload the module and it would know to remove the entry from PATH specified in the modulefile.

$ module unload helloworld

That is a very quick introduction to modules.

What design choices do we have?

After a number of modules being created (and modules can have different versions) the modules we use were modified to contain some special instructions, namely:

#%Module1.0
source /software/modules/modulefile_deps/initialise.tcl
prepend-path PATH /usr/local/helloworld/bin

We source a file which every file uses called initialise.tcl. This file may contain code to log the use of the module, or to perform help functionality.

Improving information about a module

The module system has a command called “whatis” and that describes the module. However we may have 10 versions of the same package and adding information about each version would get painful.

Therefore in the initialise.tcl file we may source another file which performs the following:

regexp {.*/modules/([^/]*/[^/]*).*} $ModulesCurrentModulefile matched topname

if [ file exists "/software/modules/modulefile_deps/$topname.tcl" ] {
  source /software/modules/modulefile_deps/$topname.tcl
} else {
  source /software/modules/modulefile_deps/missing-whatis.tcl
}

# Lets break out of the module at this point since whatis information
# has been set.
if { [ module-info mode whatis ] } {
  break
}

By using the location of the modulefile stored in “$ModulesCurrentModulefile“, this then looks for a common file based on the package name (assuming a certain layout of the modules to give “[theme/package/version]“, extract the “[theme/package]” and store in “$topname“, to acquire information from a file stored in “[theme/package].tcl” such as:

module-whatis "
Helloworld is a package to introduce the concept of hello world program.

- Website : http://www.hello.com
- Licence : Unknown."

So if we have different versions of the same module we can use a common file to print information about the module so a user can type:

$ module whatis helloworld/1.0
helloworld/1.0                :
Helloworld is a package to introduce the concept of hello world program.

- Website : http://www.hello.com
- Licence : Unknown.

Or

$ module whatis helloworld/2.0
helloworld/2.0                :
Helloworld is a package to introduce the concept of hello world program.

- Website : http://www.hello.com
- Licence : Unknown.

What other things can we do?

We have integrated the module-whatis information with our current documentation system based on ReadTheDocs. This allows the module information to be easily updated. We can create a heirachy of files to allow us to extract module versions and information – this will make maintaining the information we have about modules by reducing duplication.

Other aspects we may want a common function for is to provide help. So when a user types “module help helloworld” it provides information to the user (maybe a document on the web) which can be followed to help the user run the software.

It is hoped that by providing the user with more information, the software will be easier to use and efficiently run.