Skip to content

Support multiple platforms #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Support multiple platforms #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f7aca09
update component URLs, outline impl
wpbonelli Jan 22, 2023
7e5059a
detect platform, support configurable install locn (WIP)
wpbonelli Jan 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,26 @@ branding:
icon: 'download-cloud'
color: 'blue'
inputs:
components:
description: 'Components to install'
default: ''
cache:
description: 'Cache the install'
default: true
components:
description: 'Components to install'
default: ''
path:
description: 'Location to install oneAPI'
required: false
default: /opt/intel/oneapi
prune:
description: 'Speed up save/restore by removing infrequently used files'
default: true
setvars:
description: 'Whether to configure the oneAPI environment'
required: false
default: true
list:
description: 'List the available components'
required: false
default: false
runs:
using: 'node16'
Expand Down
294 changes: 203 additions & 91 deletions src/main.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,131 +1,243 @@
const os = require('os');
const path = require('path')
const process = require('process')

const cache = require('@actions/cache')
const core = require('@actions/core')
const exec = require('@actions/exec')
const github = require('@actions/github')
const io = require('@actions/io')
const tc = require('@actions/tool-cache')
const {isFeatureAvailable, ValidationError} = require("@actions/cache");


let key = 'v0'

const componentUrls = {
ccl: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19029/l_oneapi_ccl_p_2021.7.1.16948_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19029/l_oneapi_ccl_p_2021.7.1.16948_offline.sh',
linux: {
ccl: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19029/l_oneapi_ccl_p_2021.7.1.16948_offline.sh',
"[email protected]": 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19029/l_oneapi_ccl_p_2021.7.1.16948_offline.sh',

dal: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19032/l_daal_oneapi_p_2021.7.1.16996_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19032/l_daal_oneapi_p_2021.7.1.16996_offline.sh',

dnn: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19035/l_onednn_p_2022.2.1.16994_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19035/l_onednn_p_2022.2.1.16994_offline.sh',

dpl: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19046/l_oneDPL_p_2021.7.2.15007_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19046/l_oneDPL_p_2021.7.2.15007_offline.sh',

icx: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19030/l_dpcpp-cpp-compiler_p_2022.2.1.16991_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19030/l_dpcpp-cpp-compiler_p_2022.2.1.16991_offline.sh',

ifx: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18998/l_fortran-compiler_p_2022.2.1.16992_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18998/l_fortran-compiler_p_2022.2.1.16992_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18909/l_fortran-compiler_p_2022.2.0.8773_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18703/l_fortran-compiler_p_2022.1.0.134_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18481/l_fortran-compiler_p_2022.0.2.83_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18436/l_fortran-compiler_p_2022.0.1.70_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18210/l_fortran-compiler_p_2021.4.0.3224_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/17959/l_fortran-compiler_p_2021.3.0.3168_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/17756/l_fortran-compiler_p_2021.2.0.136_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/17508/l_fortran-compiler_p_2021.1.2.62_offline.sh',

impi: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19010/l_mpi_oneapi_p_2021.7.1.16815_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19010/l_mpi_oneapi_p_2021.7.1.16815_offline.sh',

ipp: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19007/l_ipp_oneapi_p_2021.6.2.16995_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19007/l_ipp_oneapi_p_2021.6.2.16995_offline.sh',

ippcp: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18999/l_ippcp_oneapi_p_2021.6.2.15006_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19049/l_dpcpp-cpp-compiler_p_2022.2.1.16991_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18849/l_dpcpp-cpp-compiler_p_2022.2.0.8772_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18717/l_dpcpp-cpp-compiler_p_2022.1.0.137_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18478/l_dpcpp-cpp-compiler_p_2022.0.2.84_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18435/l_dpcpp-cpp-compiler_p_2022.0.1.71_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18209/l_dpcpp-cpp-compiler_p_2021.4.0.3201_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/17928/l_dpcpp-cpp-compiler_p_2021.3.0.3168_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18999/l_ippcp_oneapi_p_2021.6.2.15006_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/17749/l_dpcpp-cpp-compiler_p_2021.2.0.118_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/17513/l_dpcpp-cpp-compiler_p_2021.1.2.63_offline.sh',

mkl: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19035/l_onednn_p_2022.2.1.16994_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19035/l_onednn_p_2022.2.1.16994_offline.sh',

tbb: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19041/l_tbb_oneapi_p_2021.7.1.15005_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19041/l_tbb_oneapi_p_2021.7.1.15005_offline.sh',

vpl: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19044/l_oneVPL_p_2022.2.5.17121_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19044/l_oneVPL_p_2022.2.5.17121_offline.sh'
},
mac: {
ifx: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18997/m_fortran-compiler-classic_p_2022.2.1.16314.dmg',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18997/m_fortran-compiler-classic_p_2022.2.1.16314.dmg',

ippcp: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18988/m_cpp-compiler-classic_p_2022.2.1.16313_offline.dmg',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18988/m_cpp-compiler-classic_p_2022.2.1.16313_offline.dmg'
},
windows: {
ifx: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18996/w_fortran-compiler_p_2022.2.1.19749.exe',
'[email protected]': "https://registrationcenter-download.intel.com/akdlm/irc_nas/18996/w_fortran-compiler_p_2022.2.1.19749.exe",

ippcp: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19031/w_dpcpp-cpp-compiler_p_2022.2.1.19748.exe',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19031/w_dpcpp-cpp-compiler_p_2022.2.1.19748.exe'
}
}

dal: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19032/l_daal_oneapi_p_2021.7.1.16996_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19032/l_daal_oneapi_p_2021.7.1.16996_offline.sh',
function getPlatform() {
platform = os.platform()
if (platform === 'linux')
return 'linux'
else if (platform === 'darwin')
return 'mac'
else
return 'windows'
}

dnn: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19035/l_onednn_p_2022.2.1.16994_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19035/l_onednn_p_2022.2.1.16994_offline.sh',
async function restoreCache(installPath, components) {
for (const component of components) {
const url = componentUrls[component.component]
if (!url) {
core.setFailed(`Unknown component: ${component.component}`)
return false
}
key = key + ':' + path.parse(url).base
}

dpl: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19046/l_oneDPL_p_2021.7.2.15007_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19046/l_oneDPL_p_2021.7.2.15007_offline.sh',
console.log(`Restoring from cache (key: ${key}, path: ${installPath})`)

icx: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19030/l_dpcpp-cpp-compiler_p_2022.2.1.16991_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19030/l_dpcpp-cpp-compiler_p_2022.2.1.16991_offline.sh',
// todo still need to hide GNU tar on Windows?
const restoreKey = await cache.restoreCache([installPath], key)
// todo restore GNU tar if needed

ifx: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18998/l_fortran-compiler_p_2022.2.1.16992_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18998/l_fortran-compiler_p_2022.2.1.16992_offline.sh',
if (restoreKey) {
console.log(`Restore succeeded: ${restoreKey}`)
// clear key so we will not try to save
key = ''
return true
} else {
console.log(`Restore failed: ${restoreKey}`)
return false
}
}

impi: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19010/l_mpi_oneapi_p_2021.7.1.16815_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19010/l_mpi_oneapi_p_2021.7.1.16815_offline.sh',
async function prune(installPath, components) {
const platform = getPlatform()

ipp: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19007/l_ipp_oneapi_p_2021.6.2.16995_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19007/l_ipp_oneapi_p_2021.6.2.16995_offline.sh',
// todo support macOS/Windows and custom install location
// how to support custom install lcn on Windows?
// may be hidden dependency on a path somewhere?
// https://github.com/modflowpy/install-intelfortran-action#install-location

ippcp: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18999/l_ippcp_oneapi_p_2021.6.2.15006_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/18999/l_ippcp_oneapi_p_2021.6.2.15006_offline.sh',
let dirs = []
if (platform === 'linux')
dirs.concat([
`/opt/intel/oneapi/compiler/latest/linux/compiler/lib/ia32_lin`,
`/opt/intel/oneapi/compiler/latest/linux/bin/ia32`,
`/opt/intel/oneapi/compiler/latest/linux/lib/emu`,
`/opt/intel/oneapi/compiler/latest/linux/lib/oclfpga`
])

mkl: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19035/l_onednn_p_2022.2.1.16994_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19035/l_onednn_p_2022.2.1.16994_offline.sh',
// todo prune locations for other platforms and components?

tbb: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19041/l_tbb_oneapi_p_2021.7.1.15005_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19041/l_tbb_oneapi_p_2021.7.1.15005_offline.sh',
console.log('Pruning oneapi install')

vpl: 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19044/l_oneVPL_p_2022.2.5.17121_offline.sh',
'[email protected]': 'https://registrationcenter-download.intel.com/akdlm/irc_nas/19044/l_oneVPL_p_2022.2.5.17121_offline.sh'
await exec.exec('du', ['-sh', installPath])
await exec.exec('sudo', ['rm', '-rf'].concat(dirs)) // need root permission
await exec.exec('du', ['-sh', installPath])
}

async function restoreCache (components) {
const useCache = core.getBooleanInput('cache')
if (!useCache) return false
async function install(installPath, component) {
// todo get current OS and select from componentUrls
const url = componentUrls['Linux'][component.component]
console.log(`Installing ${component.component} from ${url}`)
const installerPath = await tc.downloadTool(url)
// todo set install path
await exec.exec('sudo', ['bash', installerPath, '-s', '-a', '-s', '--action', 'install', '--eula', 'accept'])
// todo need to use pwsh or cmd to support windows install?
await io.rmRF(installerPath)
await prune()
// todo missing libimf.so workaround https://github.com/modflowpy/install-intelfortran-action/blob/main/action.yml#L140
// todo set environment variables and run setvars scripts, if enabled
// todo hide GNU linker on Windows so MSVC is found
}

for (const component of components) {
const url = componentUrls[component]
if (!url) {
core.error(`Unknown oneapi component: ${component}`)
process.exit(1)
async function installList(installPath, components) {
for (const component of components) {
console.log(`Installing component: ${component}`)
await install(installPath, component)
}
key = key + ':' + path.parse(url).base
}
console.log(`Key ${key}`)

console.log('Restoring from cache')
const restoreKey = await cache.restoreCache(['/opt/intel/oneapi'], key)

if (restoreKey) {
console.log(`Restore succeeded: ${restoreKey}`)
// clear key so we will not try to save
key = ''
return true
} else {
console.log(`Restore failed: ${restoreKey}`)
return false
}
}

async function prune () {
if (!core.getBooleanInput('prune')) { return }

const dirs = ['/opt/intel/oneapi/compiler/latest/linux/compiler/lib/ia32_lin',
'/opt/intel/oneapi/compiler/latest/linux/bin/ia32',
'/opt/intel/oneapi/compiler/latest/linux/lib/emu',
'/opt/intel/oneapi/compiler/latest/linux/lib/oclfpga']

console.log('Pruning oneapi install')
await exec.exec('du', ['-sh', '/opt/intel/oneapi'])
// rmRF does not have root permission
await exec.exec('sudo', ['rm', '-rf'].concat(dirs))
await exec.exec('du', ['-sh', '/opt/intel/oneapi'])
}
function list(verbose) {
let components = [];
if (verbose)
console.log('Available components:')

for (let platform in componentUrls) {
for (let component in platform) {
if (verbose)
console.log(` ${platform} ${component}: ${platform[component]}`)
components.push({
platform: platform,
component: component,
version: platform[component]
})
}
}

async function install (component) {
const url = componentUrls[component]
console.log(`Installing ${component} from ${url}`)
const installerPath = await tc.downloadTool(url)
await exec.exec('sudo', ['bash', installerPath, '-s', '-a', '-s', '--action', 'install', '--eula', 'accept'])
await io.rmRF(installerPath)
await prune()
return components
}

async function installList (components) {
for (const component of components) {
await install(component)
}
function setVars(installPath, components) {
const platform = getPlatform()
// is it safe to assume top-level setvars scripts will always exist?
const script = platform === 'windows'
? path.join(installPath, 'setvars-vcvarsall.bat')
: path.join(installPath, 'setvars.sh')
console.log(`Running environment init script: ${script}`)
exec.exec(script).then(exitCode => {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think exec'ing a script will change the environment for future steps. It looks like you need to do exportVariable
from this package https://github.com/actions/toolkit/tree/main/packages/core, similar to the addPath below. It will be hard to do this because there are many variables and many packages. Other tools implemented something that sources setvars.sh and then diffs the previous environment to collect a set of variables, then sets the variables. The compiler should be usable with path and LD_LIBRARY_PATH. The libraries would need CPATH, LIBRARY_PATH. I would save this task for later.

if (exitCode !== 0)
core.setFailed(`Failed to run environment init script: ${script}`)
})

for (let component in components) {
// todo component-specific setvars scripts?
}
}

function list () {
if (!core.getBooleanInput('list')) { return }
console.log('Available components:')
for (const component in componentUrls) {
console.log(` ${component}: ${componentUrls[component]}`)
}
function setPath(installPath, components) {
const platform = getPlatform()
for (const component in components) {
if (component.component.includes('compiler')) {
const compilerBin = path.join(installPath, 'latest', 'latest', platform, 'bin', 'intel64')
console.log(`Adding compiler bin dir to PATH: ${compilerBin}`)
core.addPath(compilerBin)
}
// todo other component-specific paths?
}
}

async function run () {
try {
list()
const components = core.getMultilineInput('components')
if (await restoreCache(components)) {
return
async function run() {
try {
const cache = core.getBooleanInput('cache')
const components = core.getMultilineInput('components')
const installPath = getPlatform() === 'windows'
? core.toWin32Path(core.getInput('path'))
: core.toPosixPath(core.getInput('path'))
const prune = core.getBooleanInput('prune')
const setvars = core.getBooleanInput('setvars')
const listComponents = core.getBooleanInput('list')

if (await restoreCache(installPath, components))
return

await installList(installPath, components)
} catch (error) {
key = ''
core.setFailed(error.message)
}
await installList(components)
} catch (error) {
key = ''
core.setFailed(error.message)
}
core.saveState('key', key)
core.saveState('key', key)
}

run()