Skip to content

[GPU] XAttention as a preview feature #32551

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 95 commits into from
Oct 26, 2025
Merged

[GPU] XAttention as a preview feature #32551

merged 95 commits into from
Oct 26, 2025

Conversation

@peterchen-intel
Copy link
Contributor

@peterchen-intel peterchen-intel commented Oct 25, 2025 •
edited
Loading

Details:

  • XAttention for FP16 KVCache as a preview feature
  • to add unit tests
  • to disable XAttention for legacy platforms (XAttention kernels are implemented for Xe2/Xe3 with CM)
  • to streamline the process of xattention. Currently kvcache shape is used to determine it. Maybe there is a better approach.
  • to add warning message for unsupported cases: multiple subsequences, typo error of kvcache precision, etc.
  • to remove the trivial converter nodes from xattention_threshold Parameter to PageAttention input.
  • to refactor xattention kernel impls by reusing RT parameters, instead of recomputing them.
  • to enable path of U8 KVCache (stretch goal)
  • WWB with long prompts

This PR should work along with openvinotoolkit/openvino.genai#2764.

Tickets:

Reviews and approvals are in #32064

Same as PR32064 14e57f9
$ git fetch origin pull/32064/head:pr/32064
$ git fetch origin pull/32551/head:pr/32551
$ git diff pr/32551 pr/32064
(empty)

To resolve PR32064 commits checks issue. one commit is signed by unknown author. 5201cdf
https://docs.github.com/en/github/authenticating-to-github/managing-commit-signature-verification/about-commit-signature-verification

riverlijunjie and others added 30 commits October 25, 2025 22:54
@riverlijunjie @peterchen-intel
Init PA CM Impl(1st/2nd token and kvcache update)
38642b4
@riverlijunjie @peterchen-intel
enabled simple pa unit tests pass
f876505
@riverlijunjie @peterchen-intel
Fix 2nd_token issue
4696ac9
@riverlijunjie @peterchen-intel
Fixed pipeline output corruption issue
5feb0c9 
   1. kvcache update's k/v offset issue
   2. 2nd token lse data overflow issue
@riverlijunjie @peterchen-intel
Fix 2nd non-16 alignment accuracy issue
b47dcf2
@riverlijunjie @peterchen-intel
Set best partition size for 2nd
cc443ad
@ceciliapeng2011 @peterchen-intel
update KV_BLOCK_SIZE to 256
a7507d7
@ceciliapeng2011 @peterchen-intel
initiate xattention integration
96fd507
@ceciliapeng2011 @peterchen-intel
qwen2.5-1.5b 4k trunk works with xatten.
be45411
@ceciliapeng2011 @peterchen-intel
4k aligned works.
78fc74e
@ceciliapeng2011 @peterchen-intel
fix block_mask not fully initialized issue.
3d6a3b2
@ceciliapeng2011 @peterchen-intel
fix of find_block
04e0520
@ceciliapeng2011 @peterchen-intel
xatten: fix accuacy problem caused by debug
31aeba7
@luo-cheng2021 @peterchen-intel
use int32 to store float INV_S to align python version accuracy
d4b0167
@ceciliapeng2011 @peterchen-intel
OV_GPU_XATTN_BLOCK_SIZE and OV_GPU_XATTN_THRESH
448ea69
@usstq @peterchen-intel
fix building error on windows.
80e06b8
@ceciliapeng2011 @peterchen-intel
process tail in find_block
3e9d1b3
@riverlijunjie @peterchen-intel
Fix f16 accuracy issue and optimize 2nd token to improve 5%
04eb7d6
@ceciliapeng2011 @peterchen-intel
fix waring_as_error on CI Windows.
5712942
@ceciliapeng2011 @peterchen-intel
dump block mask with DUMP_XATTN_BLOCK_MASK for debug
2a982b9
@riverlijunjie @peterchen-intel
Support kv cache u8 precision
6a6ba51
@ceciliapeng2011 @peterchen-intel
refactor: split into pa_common and sdpa_common, which include attenti…
7282350 
…on_common.
@ceciliapeng2011 @peterchen-intel
integrate xattn_post_proc kernel and FP16 kernel works. TODOto verify…
ba43757 
… u8 kvcache.
@riverlijunjie @peterchen-intel
update partition size
0a348ac
@ceciliapeng2011 @peterchen-intel
enable int8 kvcache for xatten, but accuracy fails.
94a9018
@ceciliapeng2011 @peterchen-intel
fix xattn kvcache u8 accuracy issue.
2f26586
@riverlijunjie @peterchen-intel
Fix 2nd accuracy issue
bdaa41e
@ceciliapeng2011 @peterchen-intel
Fix 2nd accuracy issue
d3505ac
@ceciliapeng2011 @peterchen-intel
fix xattn tailing issue: Q_blocks < K_blocks, as K_blocks is aligned …
730867d 
…to WGS
@riverlijunjie @peterchen-intel
Optimize single_token_finalization kernel with fixed unroll
d5bff20
@peterchen-intel peterchen-intel requested review from a team as code owners October 25, 2025 16:37
@peterchen-intel peterchen-intel requested review from CuriousPanCake and removed request for a team October 25, 2025 16:37
@github-actions github-actions bot added category: GPU OpenVINO GPU plugin category: transformations OpenVINO Runtime library - Transformations labels Oct 25, 2025
@peterchen-intel peterchen-intel removed the request for review from CuriousPanCake October 26, 2025 01:50
@peterchen-intel peterchen-intel added this pull request to the merge queue Oct 26, 2025
@peterchen-intel
Copy link
Contributor Author

Reviews and approvals are in #32064

@peterchen-intel peterchen-intel mentioned this pull request Oct 26, 2025
Closed
8 tasks
Merged via the queue into openvinotoolkit:master with commit 6d052b6 Oct 26, 2025
235 of 237 checks passed
@peterchen-intel peterchen-intel deleted the xattn/merge branch October 26, 2025 10:37
@peterchen-intel peterchen-intel mentioned this pull request Oct 26, 2025
Merged
@rnwang04 rnwang04 mentioned this pull request Oct 28, 2025
Closed
github-merge-queue bot pushed a commit to openvinotoolkit/openvino.genai that referenced this pull request Oct 28, 2025
@rnwang04 @ceciliapeng2011
@peterchen-intel @wgzintel
update gpu block size based on xattn (#2764)
77cb1f5 
- block size for XAttention
- work with
-- openvinotoolkit/openvino#32064
-- openvinotoolkit/openvino#32551

Tickets: CVS-173845

Related PRs
- #2924
- #2927

---------

Co-authored-by: cecilia peng <[email protected]>
Co-authored-by: Chen Peter <[email protected]>
Co-authored-by: wgzintel <[email protected]>
Co-authored-by: Copilot <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@riverlijunjie riverlijunjie riverlijunjie approved these changes

Assignees

No one assigned

Labels

category: GPU OpenVINO GPU plugin category: transformations OpenVINO Runtime library - Transformations Code Freeze

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@peterchen-intel @riverlijunjie @ceciliapeng2011 @luo-cheng2021 @usstq @rnwang04 @zhaixuejun1993 @luweizhou2016 @WeldonWangwang