Conference call notes 20150812

(back to Conference calls)

Notes on the 34th EasyBuild conference call, Wednesday August 12th 2015 (5.00pm - 5.40pm CET)

Attendees

Alphabetical list of attendees (7):

• Petar Forai (IMP/IMBA, Austria)
• Fotis Georgatos (freelancer)
• Ryan Goggin (IQC, University of Waterloo, Canada)
• Kenneth Hoste (HPC-UGent, Belgium)
• Riccardo Murri (University of Zurich, Switzerland)
• Alan O'Cais (JSC, Germany)
• Robert Schmidt (OHRI, Canada)

Agenda

• outlook to EasyBuild v2.2.1 (Kenneth)
• Q&A

outlook to EasyBuild v2.2.1

• bug fix release
• support for --package never worked because it tried to include the temporary 'fake' module file in the package, rather than the final one
• the unit tests for --package missed this due to heavy mocking of fpm :(
• fixed in
• issues with --job
• hidden dependencies end up not being installed as hidden, because --hidden doesn't get added to the eb cmdline of the job being submitted
• --job and --try don't work well together
• tweaked easyconfigs are stored in --tmpdir by default, and thus typically not available anymore in the submitted job
  • workaround is to use a shared directory as --tmpdir when using --job
  • actual solution would be to only apply --try after submitting the job
    • can this actually work (w.r.t. dep resolution, etc.)?
• --try options are still being passed down, despite a tweaked easyconfig being used in the job (??)
• also take a look at reported issue https://github.com/hpcugent/easybuild/issues/135 ?
• enhancements to EasyConfig.dump: formatting according to 'code' style for easyconfigs + retaining comments
• include easyconfigs for 2015b common toolchains + software built with either foss/2015b or intel/2015b

Other topics

• more flexibility should be supported w.r.t. what goes into module files generates by EasyBuild, and how they are composed
  • required in order to get TACC to pick up EasyBuild
  • this could be tackled by introducing the notion of a module file 'template'
  • either an actual file that can be provided, or support for providing a 'module generator plugin' similar to the support for custom module naming schemes
  • implementing this should be fairly straightforward, it basically boils down to letting module_generator construct a dict with values that can be used to complete a supplied template
  • this would most likely be accompanied by additional easyconfig parameters (e.g., usage, helptxt, ...)
    • see also whatis PR by Markus: https://github.com/hpcugent/easybuild-framework/pull/1271
  • also missing: support for Lmod specific features like prereq (vs load), module families, module properties, etc.
• EasyConfig.dump() doesn't include the list of hiddendependencies in dependencies, even though that's required => https://github.com/hpcugent/easybuild-framework/issues/1350
  • relates to support for making dependency resolution aware of subtoolchains (WIP by Alan, see
• Fotis: ping on support for easily overriding easyconfig parameters
  • see https://github.com/hpcugent/easybuild-framework/pull/712
  • Kenneth: needs to be looked at again
  • some concerns w.r.t. combo with --try, reproducibility of builds (stored easyconfig files don't include modifications), ability to only override via environment variable(s) (no cmdline or cfgfile support), etc.