A100 and H100 GPU Performance

[lxy-alexander]

I’m so sorry to bother you. I encountered an issue with the HGEMM kernel. On A100 and H100 GPUs, it only achieves about 60% of the expected performance.
Could you please advise which parts of the code should be adjusted or optimized to better adapt to these architectures?

[lxy-alexander]

I noticed that MMA instructions used in cutlass(hgemm_mma_stage_tn_cute.cu) for A100 GPU is same as the MMA instructions used in hgemm_mma_stage_tn_swizzle_x4.cu .
However, its performance on A100 is significantly lower than the cutlass version .
I’ve profiled both implementations but haven’t found any obvious way to further improve the performance on A100 so far.
Could you please tell me which factors might cause such a performance gap? What should I do to improve performance?