Skip to content

Commit 1735bfd

Browse files
XilunWuXilunWu
committed
Update on "[RFC][WIP][CP] Enable FlexAttention CP for llama3"
This PR uses the latest CP APIs to enable FlexAttention + CP for llama3. This PR removes the usage of context_paralle() context manager and use `_context_parallel_shard()` to shard the input data. Pull-Request: #1857 [ghstack-poisoned]
2 parents a5f9029 + 941ab61 commit 1735bfd

File tree

0 file changed

+0
-0
lines changed

    0 file changed

    +0
    -0
    lines changed

    0 commit comments

    Comments
     (0)