Skip to content

Commit 697d7f2

Browse files
DomInvivoDomInvivo
authored
Merge pull request #435 from datamol-io/simple-ipu-install
Simple ipu install
2 parents 2415fd8 + 0e5aeed commit 697d7f2

File tree

3 files changed

+119
-56
lines changed

3 files changed

+119
-56
lines changed

‎README.md‎

Lines changed: 6 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -54,30 +54,16 @@ pip install --no-deps -e .
5454
```
5555

5656
### For IPU developers
57-
5857
```bash
59-
mkdir ~/.venv # Create the folder for the environment
60-
python3 -m venv ~/.venv/graphium_ipu # Create the environment
61-
source ~/.venv/graphium_ipu/bin/activate # Activate the environment
62-
63-
# Update pip to the latest version
64-
python3 -m pip install --upgrade pip
65-
66-
# Install the PopTorch wheel
67-
# Make sure this is the 3.3 SDK
68-
# Change the link according to your operating system and the `PATH_TO_SDK`
69-
pip install PATH_TO_SDK/poptorch-3.3.0+113432_960e9c294b_ubuntu_20_04-cp38-cp38-linux_x86_64.whl
70-
71-
# Enable Poplar SDK (including Poplar and PopART)
72-
source PATH_TO_SDK/enable
58+
# Install Graphcore's SDK and Graphium dependencies in a new environment called `.graphium_ipu`
59+
./install_ipu.sh .graphium_ipu
60+
```
7361

74-
# Install the IPU specific and graphium requirements
75-
pip install -r requirements_ipu.txt
62+
The above step needs to be done once. After that, enable the SDK and the environment as follows:
7663

77-
# Install Graphium in dev mode
78-
pip install --no-deps -e .
64+
```bash
65+
source enable_ipu.sh .graphium_ipu
7966
```
80-
If you are new to Graphcore IPUs, you can find more details in the section below: `First Time Running On IPUs`.
8167

8268
## Training a model
8369

@@ -112,42 +98,6 @@ graphium-train --config-path [PATH] --config-name [CONFIG]
11298
Thanks to the modular nature of `hydra` you can reuse many of our config settings for your own experiments with Graphium.
11399

114100

115-
## First Time Running on IPUs
116-
For new IPU developers this section helps provide some more explanation on how to set up an environment to use Graphcore IPUs with Graphium.
117-
118-
```bash
119-
# Set up a virtual environment as normal
120-
mkdir ~/.venv # Create the folder for the environment
121-
python3 -m venv ~/.venv/graphium_ipu # Create the environment
122-
source ~/.venv/graphium_ipu/bin/activate # Activate the environment
123-
124-
python3 -m pip install --upgrade pip
125-
# We can download the Poplar SDK directly using `wget` - more details on the various Graphcore downloads can be found here `https://www.graphcore.ai/downloads`
126-
127-
# NOTE: For simplicity this will download the SDK directly where you run this command, we recommend doing this outside the Graphium directory.
128-
# Make sure to download the right file according to your operating system
129-
wget -q -O 'poplar_sdk-ubuntu_20_04-3.3.0-208993bbb7.tar.gz' 'https://downloads.graphcore.ai/direct?package=poplar-poplar_sdk_ubuntu_20_04_3.3.0_208993bbb7-3.3.0&file=poplar_sdk-ubuntu_20_04-3.3.0-208993bbb7.tar.gz'
130-
131-
# Unzip the SDK file
132-
tar -xzf poplar_sdk-ubuntu_20_04-3.3.0-208993bbb7.tar.gz
133-
# Then use pip to install the wheel
134-
python3 -m pip install poplar_sdk-ubuntu_20_04-3.3.0+1403-208993bbb7/poptorch-3.3.0+113432_960e9c294b_ubuntu_20_04-cp38-cp38-linux_x86_64.whl
135-
# Enable Poplar SDK (including Poplar and PopART)
136-
source poplar_sdk-ubuntu_20_04-3.3.0+1403-208993bbb7/enable
137-
138-
# Then as a quick test make sure poptorch is correctly installed
139-
# If it is, this will not execute properly.
140-
python3 -c "import poptorch;print('poptorch installed correctly')"
141-
142-
# Install the IPU specific and graphium requirements
143-
pip install -r requirements_ipu.txt
144-
# Install Graphium in dev mode
145-
python -m pip install --no-deps -e .
146-
147-
```
148-
149-
150-
151101
## License
152102

153103
Under the Apache-2.0 license. See [LICENSE](LICENSE).

‎enable_ipu.sh‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
#!/bin/bash
2+
3+
# Default location for the virtual environment
4+
default_venv_name=".graphium_ipu"
5+
6+
# Allow the user to specify the location of their virtual environment
7+
# If not specified, use the default location
8+
venv_name=${1:-$default_venv_name}
9+
10+
# Constants
11+
sdk_path="${venv_name}/poplar_sdk-ubuntu_20_04-3.3.0+1403-208993bbb7"
12+
13+
# Source the virtual environment
14+
source ${venv_name}/bin/activate
15+
source ${sdk_path}/enable

‎install_ipu.sh‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
#!/bin/bash
2+
3+
# Default location for the virtual environment
4+
default_venv_name=".graphium_ipu"
5+
6+
# Allow the user to specify the location of their virtual environment
7+
# If not specified, use the default location
8+
venv_name=${1:-$default_venv_name}
9+
10+
# Constants
11+
sdk_compressed_file="poplar_sdk-ubuntu_20_04-3.3.0-208993bbb7.tar.gz"
12+
sdk_wheel_file="poptorch-3.3.0+113432_960e9c294b_ubuntu_20_04-cp38-cp38-linux_x86_64.whl"
13+
sdk_url="https://downloads.graphcore.ai/direct?package=poplar-poplar_sdk_ubuntu_20_04_3.3.0_208993bbb7-3.3.0&file=${sdk_compressed_file}"
14+
sdk_path="${venv_name}/poplar_sdk-ubuntu_20_04-3.3.0+1403-208993bbb7"
15+
16+
# Check for Python3 and pip
17+
if ! command -v python3 &>/dev/null; then
18+
echo "Python3 is required but it's not installed. Exiting."
19+
exit 1
20+
fi
21+
22+
if ! command -v pip3 &>/dev/null; then
23+
echo "pip3 is required but it's not installed. Exiting."
24+
exit 1
25+
fi
26+
27+
# Remove existing venv directory if it exists
28+
if [[ -d $venv_name ]]; then
29+
echo "Removing existing virtual environment directory..."
30+
rm -rf $venv_name
31+
fi
32+
33+
# Create the virtual environment
34+
echo "Creating virtual environment..."
35+
mkdir -p $venv_name
36+
python3 -m venv $venv_name
37+
source $venv_name/bin/activate
38+
39+
# Update pip to the latest version
40+
echo "Upgrading pip..."
41+
python3 -m pip install --upgrade pip
42+
43+
# Download the Poplar SDK
44+
echo "Downloading Poplar SDK..."
45+
wget -q -O "${venv_name}/${sdk_compressed_file}" "$sdk_url"
46+
47+
# Check the wget exit status
48+
if [ $? -ne 0 ]; then
49+
echo "Failed to download Poplar SDK. Exiting."
50+
exit 1
51+
fi
52+
53+
# Unzip the SDK file
54+
echo "Extracting Poplar SDK..."
55+
tar -xzf "$venv_name/$sdk_compressed_file" -C $venv_name
56+
57+
# Install the PopTorch wheel
58+
echo "Installing PopTorch..."
59+
python3 -m pip install "${sdk_path}/${sdk_wheel_file}"
60+
61+
# Enable Poplar SDK (including Poplar and PopART)
62+
echo "Enabling Poplar SDK..."
63+
source ${sdk_path}/enable
64+
65+
# Install the IPU specific and Graphium requirements
66+
echo "Installing IPU specific and Graphium requirements..."
67+
python3 -m pip install -r requirements_ipu.txt
68+
69+
# Install Graphium in dev mode
70+
echo "Installing Graphium in dev mode..."
71+
python3 -m pip install --no-deps -e .
72+
73+
# This is a quick test make sure poptorch is correctly installed
74+
if python3 -c "import poptorch;print('poptorch installed correctly')" &> /dev/null; then
75+
echo "Installation completed successfully."
76+
else
77+
echo "Installation was not successful. Please check the logs and try again."
78+
exit 1 # Exit with status code 1 to indicate failure
79+
fi
80+
81+
# Download the datafiles (Total ~ 10Mb - nothing compared to the libraries)
82+
echo "Downloading the sub-datasets consisting on the ToyMix dataset"
83+
toymix_dir=expts/data/neurips2023/small-dataset/
84+
mkdir -p $toymix_dir
85+
86+
base_url="https://storage.googleapis.com/graphium-public/datasets/neurips_2023/Small-dataset/"
87+
files=("ZINC12k.csv.gz" "Tox21-7k-12-labels.csv.gz" "qm9.csv.gz" "qm9_random_splits.pt" "Tox21_random_splits.pt" "ZINC12k_random_splits.pt")
88+
89+
for file in "${files[@]}"; do
90+
if [ ! -f "${toymix_dir}${file}" ]; then
91+
echo "Downloading ${file}..."
92+
wget -P "${toymix_dir}" "${base_url}${file}"
93+
else
94+
echo "${file} already exists. Skipping..."
95+
fi
96+
done
97+
98+
echo "Data has been successfully downloaded."

0 commit comments

Comments
 (0)