File tree Expand file tree Collapse file tree 1 file changed +8
-3
lines changed
docs/architecture/Components Expand file tree Collapse file tree 1 file changed +8
-3
lines changed Original file line number Diff line number Diff line change @@ -90,16 +90,21 @@ For more details see the [vLLM documentation](https://docs.vllm.ai/en/stable/get
90
90
91
91
## Working with docker image
92
92
93
+ ### Clone the llm-d-inference-sim repository
94
+ ``` bash
95
+ git clone https://github.com/llm-d/llm-d-inference-sim.git
96
+ ```
97
+
93
98
### Building
94
99
To build a Docker image of the vLLM Simulator, run:
95
100
``` bash
96
- make build-llm-d-inference-sim- image
101
+ make image-build
97
102
```
98
103
99
104
### Running
100
105
To run the vLLM Simulator image under Docker, run:
101
106
``` bash
102
- docker run --rm --publish 8000:8000 ai-aware-router /llm-d-inference-sim:0.0.1 /ai-aware-router/ llm-d-inference-sim --port 8000 --model " Qwen/Qwen2.5-1.5B-Instruct" --lora " tweet-summary-0,tweet-summary-1"
107
+ docker run --rm --publish 8000:8000 ghcr.io /llm-d/ llm-d-inference-sim:dev --port 8000 --model " Qwen/Qwen2.5-1.5B-Instruct" --lora " tweet-summary-0,tweet-summary-1"
103
108
```
104
109
** Note:** The above command exposes the simulator on port 8000, and serves the Qwen/Qwen2.5-1.5B-Instruct model.
105
110
@@ -108,7 +113,7 @@ docker run --rm --publish 8000:8000 ai-aware-router/llm-d-inference-sim:0.0.1 /a
108
113
### Building
109
114
To build the vLLM simulator, run:
110
115
``` bash
111
- make build-llm-d-inference-sim
116
+ make build
112
117
```
113
118
114
119
### Running
You can’t perform that action at this time.
0 commit comments