Skip to content

Commit 88d9d6e

Browse files
[HUD] SGLang vs VLLM comparison dashboard (#7101)
This PR consists of the changes of adding a new dashboard which would be used for comparing the results between SGLang and vLLM for all the different metrics. **Changes**: 1. Added a new property `repos` which is used for handling the implementation when comparing two or more repositories in a single dashboard. 2. Implemented new methods in the reports, benchmarks, graphs, and summary typescript files to handle new logic for comparing repos. 3. Refactored existing code by replacing commonly used strings with constants, and added common functions in the llmUtils file. 4. In the table section of the dashboard, added a new column "sourceRepository", which helps in identifying the repo name when results are rendered together for multiple repos. 5. In the Graphs, added a new prefix in the fields (eg: `vllm`) to help distinguish and compare the results better. **TODOs (Future scope)**: 1. Add a `QPS` dropdown filter to allow the user to compare two benchmarks with same qps value easily. Currently, it can be done by selecting the options in the graph, but is a nice-to-have feature. **Testing** Verified through local development that the dashboard is correctly rendering the results for both the repositories, including the table as well. Dashboard [link](https://torchci-60fwxh8nt-fbopensource.vercel.app/benchmark/llms?repos=vllm-project%2Fvllm%2Csgl-project%2Fsglang) `SGLang vs vLLM Comparison Dashboard` <img width="1709" height="948" alt="Screenshot 2025-09-08 at 2 25 46 PM" src="https://github.com/user-attachments/assets/83f9f5ce-fef9-4e38-8cf8-8014a3dbd3d9" /> <img width="1709" height="948" alt="Screenshot 2025-09-08 at 1 54 12 PM" src="https://github.com/user-attachments/assets/0b0a78fa-a60f-4673-9a6c-7be58e0f17a6" />
1 parent d26ec18 commit 88d9d6e

File tree

16 files changed

+995
-79
lines changed

16 files changed

+995
-79
lines changed

torchci/components/benchmark/llms/LLMsBenchmarkPage.tsx

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ import {
2424
HELION_BENCHMARK_NAME,
2525
REPO_TO_BENCHMARKS,
2626
} from "lib/benchmark/llms/common";
27+
import { LLMsBenchmarkMode } from "lib/benchmark/llms/types/benchmarkMode";
2728
import { LLMsBenchmarkProps } from "lib/benchmark/llms/types/dashboardProps";
2829
import { getBenchmarkDropdownFeatures } from "lib/benchmark/llms/utils/dashboardPickerUtils";
2930
import {
@@ -32,7 +33,7 @@ import {
3233
} from "lib/benchmark/llms/utils/llmUtils";
3334
import { LLMsDashboardPicker } from "./components/dashboardPicker/LLMsDashboardPicker";
3435
import { LLMsTimeRangePicker } from "./components/dashboardPicker/LLMsTimeRangePicker";
35-
import LLMsReport from "./components/LLMsReport";
36+
import LLMsReport from "./components/report/LLMsReport";
3637

3738
export default function LLMsBenchmarkPage() {
3839
const router = useRouter();
@@ -44,6 +45,7 @@ export default function LLMsBenchmarkPage() {
4445
const initialPropsState: LLMsBenchmarkProps = {
4546
repoName: DEFAULT_REPO_NAME,
4647
benchmarkName: "",
48+
mode: LLMsBenchmarkMode.General,
4749
modelName: DEFAULT_MODEL_NAME,
4850
backendName: DEFAULT_BACKEND_NAME,
4951
modeName: DEFAULT_MODE_NAME,
@@ -58,6 +60,7 @@ export default function LLMsBenchmarkPage() {
5860
rCommit: "",
5961
lBranch: MAIN_BRANCH,
6062
rBranch: MAIN_BRANCH,
63+
repos: [],
6164
};
6265

6366
const [props, dispatch] = useReducer(propsReducer, initialPropsState);

0 commit comments

Comments
 (0)