Skip to content

Commit 1e4a838

Browse files
committed
Fix cargo install commands for candle with CUDA
The `cargo install --path router ...` commands for the features `candle-cuda` and `candle-cuda-turing` were using the flag `--no-default-features` that was ignoring the default features in the `text-embeddings-router` being `candle,http,dynamic-linking`, hence the command was failing as `cudarc` requires to have any linking out of `static-linking`, `dynamic-linking` or `dynamic-loading` for the CUDA Libraries, hence when ignoring the `default` features, and just adding `-F http` either the e.g. `-F dynamic-loading` feature was missing or just to only keep the e.g. `-F candle-cuda`, as `dynamic-linking` and `http` are default features already.
1 parent d7af1fc commit 1e4a838

File tree

2 files changed

+6
-6
lines changed

2 files changed

+6
-6
lines changed

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -530,13 +530,13 @@ export PATH=$PATH:/usr/local/cuda/bin
530530
Then run:
531531

532532
```shell
533-
# This can take a while as we need to compile a lot of cuda kernels
533+
# This can take a while as we need to compile a lot of CUDA kernels
534534

535535
# On Turing GPUs (T4, RTX 2000 series ... )
536-
cargo install --path router -F candle-cuda-turing -F http --no-default-features
536+
cargo install --path router -F candle-cuda-turing
537537

538538
# On Ampere and Hopper
539-
cargo install --path router -F candle-cuda -F http --no-default-features
539+
cargo install --path router -F candle-cuda
540540
```
541541

542542
You can now launch Text Embeddings Inference on GPU with:

docs/source/en/local_gpu.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -39,18 +39,18 @@ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
3939

4040
## Step 3: Install necessary packages
4141

42-
This step can take a while as we need to compile a lot of cuda kernels.
42+
This step can take a while as we need to compile a lot of CUDA kernels.
4343

4444
### For Turing GPUs (T4, RTX 2000 series ... )
4545

4646
```shell
47-
cargo install --path router -F candle-cuda-turing -F http --no-default-features
47+
cargo install --path router -F candle-cuda-turing
4848
```
4949

5050
### For Ampere and Hopper
5151

5252
```shell
53-
cargo install --path router -F candle-cuda -F http --no-default-features
53+
cargo install --path router -F candle-cuda
5454
```
5555

5656
## Step 4: Launch Text Embeddings Inference

0 commit comments

Comments
 (0)