Skip to content

llm-d/llm-d-pd-utils

Repository files navigation

PD utils

Build NIXL Image

docker build -t nixl:latest .
docker tag nixl:latest ghcr.io/<>/nixl:latest
docker push ghcr.io/<>/nixl:latest

Deploy & Test

cd benchmarks
./benchmark_deployment.sh nixl -rdma

Use -rdma for the deployment in cluster with RoCE enabled for performance. This will deploy nixl-client-roce and nixl-server-roce, and run a benchmarking script to measure the transfer throughput (GB/s). Refer to benchmarking for more details.