EasyBuild¶
EasyBuild is a software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way.
Configuration¶
The default configuration is following:
Default configuration
eb --show-config
#
# Current EasyBuild configuration
# (C: command line argument, D: default value, E: environment variable, F: configuration file)
#
buildpath (D) = /home/username/.local/easybuild/build
containerpath (D) = /home/username/.local/easybuild/containers
installpath (D) = /home/username/.local/easybuild
repositorypath (D) = /home/username/.local/easybuild/ebfiles_repo
robot-paths (D) = /storage-apps/easybuild/software/EasyBuild/4.7.2/easybuild/easyconfigs
sourcepath (D) = /home/username/.local/easybuild/sources
Editing configuration¶
Configuring EasyBuild can be done by:
- using eb with command line arguments
- setting environment variables ($EASYBUILD_...)
- providing one or more configuration files
Recommendation
You can find all information about EasyBuild configuration here. We suggest to use eb command line arguments for configuration.
Shared EasyBuild installations¶
All software installations realized with EasyBuild available for all users are stored in /storage-apps/easybuild/. Following directories are important:
/storage-apps/easybuild/software- contains all original software files installed with EasyBuild (binaries, configs, etc.)/storage-apps/easybuild/modules- contains all modulefiles generated by EasyBuild/storage-apps/easybuild/software/EasyBuild/4.7.2/easybuild/easyconfigs- build recipes of software for EasyBuild (.eb files)
Typical workflow¶
Compilation with EasyBuild
Software compilation with EasyBuild is possible only on login nodes!
Let's install gnuplot package with EasyBuild. Start with loading of EasyBuild module first and then continue with checking of all available easyconfig (consist version and toolchain) files for gnuplot:
Check all easyconfig files for gnuplot
module load EasyBuild
eb -S gnuplot
== found valid index for /storage-apps/easybuild/software/EasyBuild/4.7.2/easybuild/easyconfigs, so using it...
CFGS1=/storage-apps/easybuild/software/EasyBuild/4.7.2/easybuild/easyconfigs
* $CFGS1/g/gnuplot/gnuplot-5.0.3-foss-2016a.eb
* $CFGS1/g/gnuplot/gnuplot-5.0.3-intel-2016a.eb
* $CFGS1/g/gnuplot/gnuplot-5.0.5-foss-2016b.eb
* $CFGS1/g/gnuplot/gnuplot-5.0.5-intel-2016b.eb
* $CFGS1/g/gnuplot/gnuplot-5.0.6-intel-2017a.eb
* $CFGS1/g/gnuplot/gnuplot-5.2.2-foss-2017b.eb
* $CFGS1/g/gnuplot/gnuplot-5.2.2-foss-2018a.eb
* $CFGS1/g/gnuplot/gnuplot-5.2.2-intel-2017b.eb
* $CFGS1/g/gnuplot/gnuplot-5.2.2-intel-2018a.eb
* $CFGS1/g/gnuplot/gnuplot-5.2.5-foss-2018b.eb
* $CFGS1/g/gnuplot/gnuplot-5.2.5_fix-libcerf.patch
* $CFGS1/g/gnuplot/gnuplot-5.2.6-GCCcore-8.2.0.eb
* $CFGS1/g/gnuplot/gnuplot-5.2.6-foss-2018b.eb
* $CFGS1/g/gnuplot/gnuplot-5.2.6-fosscuda-2018b.eb
* $CFGS1/g/gnuplot/gnuplot-5.2.6_fix_missing_dep_for_gobject_to_cairopdf.patch
* $CFGS1/g/gnuplot/gnuplot-5.2.8-GCCcore-8.3.0.eb
* $CFGS1/g/gnuplot/gnuplot-5.2.8-GCCcore-9.3.0.eb
* $CFGS1/g/gnuplot/gnuplot-5.4.1-GCCcore-10.2.0.eb
* $CFGS1/g/gnuplot/gnuplot-5.4.2-GCCcore-10.3.0.eb
* $CFGS1/g/gnuplot/gnuplot-5.4.2-GCCcore-11.2.0.eb
* $CFGS1/g/gnuplot/gnuplot-5.4.4-GCCcore-11.3.0.eb
* $CFGS1/g/gnuplot/gnuplot-5.4.6-GCCcore-12.2.0.eb
Note: 6 matching archived easyconfig(s) found, use --consider-archived-easyconfigs to see them
Consider installation of gnuplot-5.4.6 with GCCcore-12.2.0 toolchain. Check dependencies first with -Dr option. Already installed dependencies are marked with [X].
Check all dependencies for installation
eb -Dr gnuplot-5.4.6-GCCcore-12.2.0.eb
== Temporary log file in case of crash /tmp/eb-esp0xabd/easybuild-1gbj5uyi.log
== found valid index for /storage-apps/easybuild/software/EasyBuild/4.7.2/easybuild/easyconfigs, so using it...
== found valid index for /storage-apps/easybuild/software/EasyBuild/4.7.2/easybuild/easyconfigs, so using it...
Dry run: printing build status of easyconfigs and dependencies
CFGS=/storage-apps/easybuild/software/EasyBuild/4.7.2/easybuild/easyconfigs
* [x] $CFGS/m/M4/M4-1.4.19.eb (module: M4/1.4.19)
* [x] $CFGS/b/Bison/Bison-3.8.2.eb (module: Bison/3.8.2)
* [x] $CFGS/f/flex/flex-2.6.4.eb (module: flex/2.6.4)
* [x] $CFGS/z/zlib/zlib-1.2.12.eb (module: zlib/1.2.12)
* [x] $CFGS/b/binutils/binutils-2.39.eb (module: binutils/2.39)
* [x] $CFGS/g/GCCcore/GCCcore-12.2.0.eb (module: GCCcore/12.2.0)
* [x] $CFGS/h/help2man/help2man-1.49.2-GCCcore-12.2.0.eb (module: help2man/1.49.2-GCCcore-12.2.0)
* [x] $CFGS/m/M4/M4-1.4.19-GCCcore-12.2.0.eb (module: M4/1.4.19-GCCcore-12.2.0)
* [x] $CFGS/z/zlib/zlib-1.2.12-GCCcore-12.2.0.eb (module: zlib/1.2.12-GCCcore-12.2.0)
* [x] $CFGS/b/Bison/Bison-3.8.2-GCCcore-12.2.0.eb (module: Bison/3.8.2-GCCcore-12.2.0)
* [x] $CFGS/f/flex/flex-2.6.4-GCCcore-12.2.0.eb (module: flex/2.6.4-GCCcore-12.2.0)
* [x] $CFGS/b/binutils/binutils-2.39-GCCcore-12.2.0.eb (module: binutils/2.39-GCCcore-12.2.0)
* [x] $CFGS/p/pkgconf/pkgconf-1.9.3-GCCcore-12.2.0.eb (module: pkgconf/1.9.3-GCCcore-12.2.0)
* [x] $CFGS/n/ncurses/ncurses-6.3-GCCcore-12.2.0.eb (module: ncurses/6.3-GCCcore-12.2.0)
* [x] $CFGS/l/libtool/libtool-2.4.7-GCCcore-12.2.0.eb (module: libtool/2.4.7-GCCcore-12.2.0)
* [x] $CFGS/b/bzip2/bzip2-1.0.8-GCCcore-12.2.0.eb (module: bzip2/1.0.8-GCCcore-12.2.0)
* [x] $CFGS/g/groff/groff-1.22.4-GCCcore-12.2.0.eb (module: groff/1.22.4-GCCcore-12.2.0)
* [ ] $CFGS/l/libpng/libpng-1.6.38-GCCcore-12.2.0.eb (module: libpng/1.6.38-GCCcore-12.2.0)
* [x] $CFGS/e/expat/expat-2.4.9-GCCcore-12.2.0.eb (module: expat/2.4.9-GCCcore-12.2.0)
* [x] $CFGS/l/libreadline/libreadline-8.2-GCCcore-12.2.0.eb (module: libreadline/8.2-GCCcore-12.2.0)
* [ ] $CFGS/n/NASM/NASM-2.15.05-GCCcore-12.2.0.eb (module: NASM/2.15.05-GCCcore-12.2.0)
* [ ] $CFGS/p/pixman/pixman-0.42.2-GCCcore-12.2.0.eb (module: pixman/0.42.2-GCCcore-12.2.0)
* [ ] $CFGS/g/gperf/gperf-3.1-GCCcore-12.2.0.eb (module: gperf/3.1-GCCcore-12.2.0)
* [x] $CFGS/p/pkgconf/pkgconf-1.8.0.eb (module: pkgconf/1.8.0)
* [x] $CFGS/o/OpenSSL/OpenSSL-1.1.eb (module: OpenSSL/1.1)
* [x] $CFGS/c/cURL/cURL-7.86.0-GCCcore-12.2.0.eb (module: cURL/7.86.0-GCCcore-12.2.0)
* [x] $CFGS/d/DB/DB-18.1.40-GCCcore-12.2.0.eb (module: DB/18.1.40-GCCcore-12.2.0)
* [x] $CFGS/p/Perl/Perl-5.36.0-GCCcore-12.2.0.eb (module: Perl/5.36.0-GCCcore-12.2.0)
* [x] $CFGS/a/Autoconf/Autoconf-2.71-GCCcore-12.2.0.eb (module: Autoconf/2.71-GCCcore-12.2.0)
* [x] $CFGS/a/Automake/Automake-1.16.5-GCCcore-12.2.0.eb (module: Automake/1.16.5-GCCcore-12.2.0)
* [x] $CFGS/a/Autotools/Autotools-20220317-GCCcore-12.2.0.eb (module: Autotools/20220317-GCCcore-12.2.0)
* [ ] $CFGS/p/Perl/Perl-5.36.0-GCCcore-12.2.0-minimal.eb (module: Perl/5.36.0-GCCcore-12.2.0-minimal)
* [ ] $CFGS/u/util-linux/util-linux-2.38.1-GCCcore-12.2.0.eb (module: util-linux/2.38.1-GCCcore-12.2.0)
* [x] $CFGS/l/libffi/libffi-3.4.4-GCCcore-12.2.0.eb (module: libffi/3.4.4-GCCcore-12.2.0)
* [x] $CFGS/u/UnZip/UnZip-6.0-GCCcore-12.2.0.eb (module: UnZip/6.0-GCCcore-12.2.0)
* [x] $CFGS/n/ncurses/ncurses-6.3.eb (module: ncurses/6.3)
* [x] $CFGS/g/gettext/gettext-0.21.1.eb (module: gettext/0.21.1)
* [x] $CFGS/x/XZ/XZ-5.2.7-GCCcore-12.2.0.eb (module: XZ/5.2.7-GCCcore-12.2.0)
* [x] $CFGS/l/libarchive/libarchive-3.6.1-GCCcore-12.2.0.eb (module: libarchive/3.6.1-GCCcore-12.2.0)
* [x] $CFGS/c/CMake/CMake-3.24.3-GCCcore-12.2.0.eb (module: CMake/3.24.3-GCCcore-12.2.0)
* [ ] $CFGS/g/gnuplot/gnuplot-5.4.6-GCCcore-12.2.0.eb (module: gnuplot/5.4.6-GCCcore-12.2.0)
== Temporary log file(s) /tmp/eb-esp0xabd/easybuild-1gbj5uyi.log* have been removed.
== Temporary directory /tmp/eb-esp0xabd has been removed.
To get information about configuration of software and how the installation is done, inspect the .eb file:
Check .eb file
eb --show-ec gnuplot-5.4.6-GCCcore-12.2.0.eb
== Temporary log file in case of crash /tmp/eb-5lb8lvwf/easybuild-vqyv_apk.log
== found valid index for /storage-apps/easybuild/software/EasyBuild/4.7.2/easybuild/easyconfigs, so using it...
== Contents of /storage-apps/easybuild/software/EasyBuild/4.7.2/easybuild/easyconfigs/g/gnuplot/gnuplot-5.4.6-GCCcore-12.2.0.eb:
easyblock = 'ConfigureMake'
name = 'gnuplot'
version = '5.4.6'
homepage = 'http://gnuplot.sourceforge.net'
description = """Portable interactive, function plotting utility"""
toolchain = {'name': 'GCCcore', 'version': '12.2.0'}
source_urls = [('https://sourceforge.net/projects/gnuplot/files/gnuplot/%(version)s', 'download')]
sources = [SOURCE_TAR_GZ]
checksums = ['02fc27918200ed64d8f0c3b84fe81b95b59cd47ad99f270939ae497c19f27419']
builddependencies = [
('binutils', '2.39'),
('pkgconf', '1.9.3'),
('Autotools', '20220317'),
]
dependencies = [
('ncurses', '6.3'),
('cairo', '1.17.4'),
('libjpeg-turbo', '2.1.4'),
('libpng', '1.6.38'),
('libgd', '2.3.3'),
('Pango', '1.50.12'),
('libcerf', '2.3'),
('X11', '20221110'),
('Qt5', '5.15.7'),
('Lua', '5.4.4'),
('wxWidgets', '3.2.2.1'),
]
preconfigopts = 'autoreconf && '
configopts = '--with-qt=qt5 --without-latex '
sanity_check_paths = {
'files': ['bin/gnuplot'],
'dirs': []
}
# make sure that pdf terminal type is available
sanity_check_commands = ["gnuplot -e 'set terminal pdf'"]
moduleclass = 'vis'
== Temporary log file(s) /tmp/eb-5lb8lvwf/easybuild-vqyv_apk.log* have been removed.
== Temporary directory /tmp/eb-5lb8lvwf has been removed.
You can now install software and all missing dependencies (-r option) with the following command:
Install software with all missing dependencies
eb -r gnuplot-5.4.6-GCCcore-12.2.0.eb --installpath=/home/projects/<project_id>/yourfolder
Tip
We encourage users to use shared project folders for personal EasyBuild installations to ensure the sharing of software installations between all project participants.
Finally, you can use your personal modules and load your software module for usage.
Use your modules and software
module use /home/projects/<project_id>/yourfolder/modules/all
module load gnuplot-5.4.6-GCCcore-12.2.0
Created by: