update gpu block size based on xattn

[rnwang04]

• block size for XAttention
• benchmark.py supports XAttention configuration
• work with [GPU] XAttention as a preview feature openvino#32064

Tickets: CVS-173857

[Wovchena]

I'll leave C++ review to @vshampor

[Wovchena]

Waiting for Vasily

[vshampor]

The test at

openvino.genai/tests/python_tests/test_kv_cache_eviction.py

Line 105 in 2650995

def test_cache_optimized_generation_is_similar_to_unoptimized(test_struct, apply_rotation, use_sparse_attention):

must be extended with the XAttention case, and/or additional tests must be added to demonstrate just what behaviour you had in mind for the case where the user just tries to switch from SparseAttentionMode::TRISHAPE to SparseAttentionMode::XATTENTION without changing his expectations about the block size of 16 that TRISHAPE used to work perfectly with.

Also, I guess I'll have to ask again - why is it that only enabling xattention changes the GPU block size implicitly to 128? If it's such a performant block size anyway, why do you not change the GPU block size to 128 everywhere and avoid implicit block size changes of which the user is unaware?

[vshampor]

See my previous comment

[Copilot]

Pull Request Overview

This PR updates the GPU block size configuration to support XAttention, which uses a larger block size (256) compared to the standard GPU block size (16). The changes enable detection of XAttention at runtime and configure the appropriate block size accordingly.

• Adds XAttention detection logic based on cache dimensions
• Introduces sparse attention configuration support in benchmarking tools
• Refactors sparse attention setup into a reusable function

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
src/cpp/src/continuous_batching/cache_manager.hpp	Adds XAttention detection and sets GPU block size to 256 when XAttention is enabled
tools/who_what_benchmark/whowhatbench/model_loaders.py	Extracts sparse attention configuration into a separate function and adds validation logic
tools/llm_bench/llm_bench_utils/ov_utils.py	Adds validation to prevent conflicting sparse attention configuration
tools/llm_bench/task/text_generation.py	Moves GenerationConfig import outside conditional block for broader scope

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

Extra space in error message prefix: '==Failure ==' should be '==Failure=='.

Suggested change

	raise RuntimeError("==Failure ==: sparse_attention_config value can't be used with use_sparse_attention=False")
	raise RuntimeError("==Failure==: sparse_attention_config value can't be used with use_sparse_attention=False")

[Copilot]

Trailing whitespace at end of line should be removed.

Suggested change

	configure_sparse_attention(scheduler_params, scheduler_config)
	configure_sparse_attention(scheduler_params, scheduler_config)

[Copilot]

The variable gpu_block_size_xattn is defined but m_block_size assignment was moved to line 89. This leaves gpu_block_size_xattn defined far from its only usage point. Consider moving this constant definition closer to where it's used (before line 89) or defining it as a class constant for better maintainability.

[Copilot]

[nitpick] The nested ternary operator makes this line difficult to read. Consider using an if-else statement for better clarity, especially since this is a critical configuration decision.

Suggested change

	m_block_size = all_gpu_device ? ( has_xattention ? gpu_block_size_xattn : gpu_block_size ) : cpu_block_size;
	if (all_gpu_device) {
	if (has_xattention) {
	m_block_size = gpu_block_size_xattn;
	} else {
	m_block_size = gpu_block_size;
	}
	} else {
	m_block_size = cpu_block_size;
	}