[compiler toolkit] compiler toolkit without TP

[yiming0416]

Currently we hardcode "tp" dimension when parallelizing inputs (converting from plain tensor to DTensor). This won't work when there is no TP is our parallelisms (e.g., SimpleFSDP only or SimpleFSDP + EP)

The fundamental reason is that currently SimpleFSDP only accepts plain tensor inputs. Making it accept DTensor inputs would require eager rewrite of SimpleFSDP's frontend.

So as a workaround, we only DTensorize the inputs when there is a "tp" dimension in the world_mesh.

SimpleFSDP Only on llama3

NGPU=8 CONFIG_FILE=./torchtitan/models/llama3/train_configs/debug_model.toml ./run_train.sh --model.name compiler_toolkit.llama3 --parallelism.data_parallel_shard_degree=8

SimpleFSDP + EP on dsv3

NGPU=4 CONFIG_FILE=./torchtitan/models/deepseek_v3/train_configs/debug_model.toml ./run_train.sh --model.name compiler_toolkit.deepseek_v3 --parallelism.data_parallel_shard_degree=4 --parallelism.expert_parallel_degree=2 --activation_checkpoint.mode none