Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@ With `llm-d`, users can operationalize GenAI deployments with a modular solution

Built by leaders in the Kubernetes and vLLM projects, `llm-d` is a community-driven, Apache-2 licensed project with an open development model.

## Video Demonstration

import VideoEmbed from '@site/src/components/VideoEmbed';

<VideoEmbed videoId="32MqYC3OydE" />

## Architecture

`llm-d` adopts a layered architecture on top of industry-standard open technologies: vLLM, Kubernetes, and Inference Gateway.
Expand All @@ -17,7 +23,6 @@ Built by leaders in the Kubernetes and vLLM projects, `llm-d` is a community-dri
![llm-d Architecture](../assets/images/llm-d-arch-simplified.svg)



Key features of `llm-d` include:

- **vLLM-Optimized Inference Scheduler:** `llm-d` builds on IGW's pattern for customizable “smart” load-balancing via the Endpoint Picker Protocol (EPP) to define vLLM-optimized scheduling. Leveraging operational telemetry, the Inference Scheduler implements the filtering and scoring algorithms to make decisions with P/D-, KV-cache-, SLA-, and load-awareness. Advanced teams can implement their own scorers to further customize, while benefiting from other features in IGW, like flow control and latency-aware balancing. [See our Northstar design](https://docs.google.com/document/d/1kE1LY8OVjiOgKVD9-9Po96HODbTIbgHp4qgvw06BCOc/edit?tab=t.0#heading=h.4rgkvvo5gnle)
Expand Down
61 changes: 61 additions & 0 deletions src/components/VideoEmbed.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
import React from 'react';

const VideoEmbed = ({
videoId,
width = '100%',
height = '360',
responsive = true,
autoplay = false,
modestbranding = true,
rel = false
}) => {
// Construct the URL with optional parameters
const params = new URLSearchParams({
autoplay: autoplay ? 1 : 0,
modestbranding: modestbranding ? 1 : 0,
rel: rel ? 1 : 0
});

const src = `https://www.youtube.com/embed/${videoId}?${params.toString()}`;

const iframe = (
<iframe
src={src}
style={responsive ? {
width: '100%',
height: '100%'
} : {
width,
height
}}
frameBorder="0"
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
allowFullScreen
/>
);

if (responsive) {
return (
<div style={{
position: 'relative',
paddingBottom: '56.25%',
height: 0,
overflow: 'hidden'
}}>
<div style={{
position: 'absolute',
top: 0,
left: 0,
width: '100%',
height: '100%'
}}>
{iframe}
</div>
</div>
);
}

return iframe;
};

export default VideoEmbed;