[Compiler Toolkit] Assert compile.enable=False

[SherlockNoMad]

As title, compile.enable should be False in the compiler toolkit style workflow

[xmfan]

seems weird, if anything shouldn't we assert it to always be True?

[ruisizhang123]

curious: currently, loss is compiled separately from model. This parallelize.py only controls model compilation part.

Do we want model's loss to be not compiled at all, or we still want users to control loss compilation?

Maybe it's better to check if "model" is in compile.components here? https://github.com/pytorch/torchtitan/blob/main/torchtitan/models/llama3/train_configs/debug_model.toml#L69

[fegin]

Suggested change

	not job_config.compile.enable
	"model" not in job_config.compile.components

[fegin]

Echo to @xmfan's comment. The compile flag must be set to false to enable the compiler toolkit, which seems counterintuitive.

[yiming0416]

Asserting compile.enable=False indeed seems counterintuitive under the compiler_toolkit experiment. WDYT about the following:
Refactor parallelize_deepseekv3() in experiments/simple_fsdp/deepseek_v3/parallelize.py into a parallelize_base_fn which contains everything until the line of if job_config.compile.enable

Then we import this parallelize_base_fn in the compiler_toolkit experiments folder. Then compile.enable=True, we apply the CompiledModule wrapper. In this way, we always assert compile.enable=True

[ruisizhang123]

ehhh, I think it would be better to keep simplefsdp's parallelize function close to model/deepseek_v3/parallelize.py, which would be easier to integrate new features from main/deepseek_v3.

Why don't we add a context manager to temporary override job_config.compile to False when calling simple_fsdp_parallelize_llama.... I don't think it would be so counterintuitive as long as you leave a comment saying you are doing region compile later.