Skip to content

[WIP] async_tp shape mismatch in rs+mm repro #1705

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: autoparallel
Choose a base branch
from
Open

[WIP] async_tp shape mismatch in rs+mm repro #1705

wants to merge 1 commit into from

Conversation

IvanKobzarev
Copy link

@IvanKobzarev IvanKobzarev commented Sep 12, 2025 •
edited
Loading 

torchrun --nproc_per_node=8 --rdzv_backend c10d --rdzv_endpoint=localhost:0 --local-ranks-filter 0 --role rank --tee 3 -m torchtitan.train --job.config_file ./torchtitan/models/llama3/train_configs/llama3_8b.toml --model.name llama3_auto_parallel --training.steps 50 --training.local_batch_size 4 --training.dataset c4 --parallelism.data_parallel_replicate_degree 1 --parallelism.data_parallel_shard_degree 2 --parallelism.tensor_parallel_degree 4 --parallelism.context_parallel_degree 1 --experimental.bucket_all_gathers_fx none --experimental.bucket_reduce_scatters_fx none --metrics.log_freq 1 --profiling.profile_freq 20 --compile.enable

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Sep 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tianyu-l tianyu-l Awaiting requested review from tianyu-l tianyu-l is a code owner

@fegin fegin Awaiting requested review from fegin fegin is a code owner

@wwwjn wwwjn Awaiting requested review from wwwjn wwwjn is a code owner

@wconstab wconstab Awaiting requested review from wconstab wconstab is a code owner

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Meta Open Source bot.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@IvanKobzarev