Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -530,13 +530,13 @@ export PATH=$PATH:/usr/local/cuda/bin
Then run:

```shell
# This can take a while as we need to compile a lot of cuda kernels
# This can take a while as we need to compile a lot of CUDA kernels

# On Turing GPUs (T4, RTX 2000 series ... )
cargo install --path router -F candle-cuda-turing -F http --no-default-features
cargo install --path router -F candle-cuda-turing

# On Ampere and Hopper
cargo install --path router -F candle-cuda -F http --no-default-features
cargo install --path router -F candle-cuda
```

You can now launch Text Embeddings Inference on GPU with:
Expand Down
6 changes: 3 additions & 3 deletions docs/source/en/local_gpu.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,18 +39,18 @@ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

## Step 3: Install necessary packages

This step can take a while as we need to compile a lot of cuda kernels.
This step can take a while as we need to compile a lot of CUDA kernels.

### For Turing GPUs (T4, RTX 2000 series ... )

```shell
cargo install --path router -F candle-cuda-turing -F http --no-default-features
cargo install --path router -F candle-cuda-turing
```

### For Ampere and Hopper

```shell
cargo install --path router -F candle-cuda -F http --no-default-features
cargo install --path router -F candle-cuda
```

## Step 4: Launch Text Embeddings Inference
Expand Down
Loading