SimplyHPC and PETSc

PETSc

[SubmitAzureJob.ps1] (https://github.com/vbaros/SimplyHPC/blob/master/Powershell.scripts/SubmitAzureJob.ps1) is an example script that automatizes deployment and running PETSc.

The Portable Extensible Toolkit for Scientific Computing (PETSc) from Argonne National Laboratory is a well known tool for parallel linear algebra. It contains classes for distributed vectors and matrices and a large number of algorithms for eigenvalues, singular value decomposition, and solving linear systems of equations. The PETSc library has been used in numerous research and software projects.

We have produced a fast version of PETSc on Windows, using the Intel Compilers (C and Fortran) and the INTEL MKL BLAS library. In addition, we have written a easy-to-use Windows installer for the library. You can get the license at [[email protected]](mailto:[email protected]?Subject=PETSc for Windows).

In this example we run petsc-solver.exe with MPI to solve a system of equation Ax=b with an iterative solver until a defined convergence criteria are met. In order PETSc requires Microsoft MPI and SMPD and MPID services. You can configure all prerequsitives for target applications in two configuration files.

The ServiceDefinition.csdef (see HPCWorkerRole Project\bin) contains a list of tasks that must be executed during the startup phase. In this example we run an installation file in a silent, elevated mode to install MS MPI, run a batch file to install SMPD and MPID services, and a command line script to disable Firewall.

Please note that disabling Firewall can pose a serious threat to your security policies, we do it here only for demonstration purposes to simplify the tutorial.

<Startup>
  <!--ONLY FOR TESTING PURPOSES ... needs to be removed for production-->
  <Task commandLine="Startup\DisableFirewall.cmd" taskType="simple" executionContext="elevated"></Task>
  <Task commandLine="Startup\MSMPISetup.exe -unattend -force -minimal" taskType="simple" executionContext="elevated"></Task>
  <Task commandLine="Startup\installservices.bat" taskType="simple" executionContext="elevated"></Task>
</Startup>
<LocalResources>
  <LocalStorage name="jobdata" sizeInMB="10000" />
</LocalResources>
  ...
</WorkerRole>
</ServiceDefinition>

Next, PETSc should be declared. A SubmitAzureJob.ps1 script contains an example of such a declaration.

$XMLConfigurationPath = 'AzureConfigurationActive.xml'
$pathToDeploymentTemplate = 'd:\installationpath\SimplyHPC\HpcWorkerRole\bin\Debug\'

$clusterName = 'petsc2' # needs to be unique across all azure clusters.

$sizeOfNodes = 'Large' # A2, A8, Small, Large...
$numberOfNodes = 3
$jobDataDirectory = 'd:\MicrosoftHPC\Tools\PETScRun'
$jobExecutable = 'petsc-solver.exe' $jobParameters = '-fin nordborg.sbaij -mat_type mpisbaij -ksp_max_it 200 -log_summary summary.txt -ksp_final_residual -ksp_converged_reason'

Here, we execute petsc-solver.exe on three Large instances with a set of $jobParameters. The script creates a new Azure Service, executes PETSc with MPI in the petsc2 cluster and download the results after the job has finished. The script can be executed from PowerShell.