Skip to content

Commit 22c8c3c

Browse files
docs: explain CUDA 11 compilation [no ci] (#16824)
1 parent 6db3d1f commit 22c8c3c

File tree

1 file changed

+42
-0
lines changed

1 file changed

+42
-0
lines changed

docs/build.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,48 @@ GeForce RTX 3070 8.6
178178
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES="86;89"
179179
```
180180

181+
### Overriding the CUDA Version
182+
183+
If you have multiple CUDA installations on your system and want to compile llama.cpp for a specific one, e.g. for CUDA 11.7 installed under `/opt/cuda-11.7`:
184+
185+
```bash
186+
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_COMPILER=/opt/cuda-11.7/bin/nvcc -DCMAKE_INSTALL_RPATH="/opt/cuda-11.7/lib64;\$ORIGIN" -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON
187+
```
188+
189+
#### Fixing Compatibility Issues with Old CUDA and New glibc
190+
191+
If you try to use an old CUDA version (e.g. v11.7) with a new glibc version you can get errors like this:
192+
193+
```
194+
/usr/include/bits/mathcalls.h(83): error: exception specification is
195+
incompatible with that of previous function "cospi"
196+
197+
198+
/opt/cuda-11.7/bin/../targets/x86_64-linux/include/crt/math_functions.h(5545):
199+
here
200+
```
201+
202+
It seems the least bad solution is to patch the CUDA installation to declare the correct signatures.
203+
Replace the following lines in `/path/to/your/cuda/installation/targets/x86_64-linux/include/crt/math_functions.h`:
204+
205+
```C++
206+
// original lines
207+
extern __DEVICE_FUNCTIONS_DECL__ __device_builtin__ double cospi(double x);
208+
extern __DEVICE_FUNCTIONS_DECL__ __device_builtin__ float cospif(float x);
209+
extern __DEVICE_FUNCTIONS_DECL__ __device_builtin__ double sinpi(double x);
210+
extern __DEVICE_FUNCTIONS_DECL__ __device_builtin__ float sinpif(float x);
211+
extern __DEVICE_FUNCTIONS_DECL__ __device_builtin__ double rsqrt(double x);
212+
extern __DEVICE_FUNCTIONS_DECL__ __device_builtin__ float rsqrtf(float x);
213+
214+
// edited lines
215+
extern __DEVICE_FUNCTIONS_DECL__ __device_builtin__ double cospi(double x) noexcept (true);
216+
extern __DEVICE_FUNCTIONS_DECL__ __device_builtin__ float cospif(float x) noexcept (true);
217+
extern __DEVICE_FUNCTIONS_DECL__ __device_builtin__ double sinpi(double x) noexcept (true);
218+
extern __DEVICE_FUNCTIONS_DECL__ __device_builtin__ float sinpif(float x) noexcept (true);
219+
extern __DEVICE_FUNCTIONS_DECL__ __device_builtin__ double rsqrt(double x) noexcept (true);
220+
extern __DEVICE_FUNCTIONS_DECL__ __device_builtin__ float rsqrtf(float x) noexcept (true);
221+
```
222+
181223
### Runtime CUDA environmental variables
182224
183225
You may set the [cuda environmental variables](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars) at runtime.

0 commit comments

Comments
 (0)