Qwen 25vl training pipeline #830
Conversation
Signed-off-by: yaoyu-33 <[email protected]>
Signed-off-by: yaoyu-33 <[email protected]>
# Conflicts: # src/megatron/bridge/training/config.py
Signed-off-by: yaoyu-33 <[email protected]>
Signed-off-by: yaoyu-33 <[email protected]>
Signed-off-by: yaoyu-33 <[email protected]>
Signed-off-by: yaoyu-33 <[email protected]>
model Signed-off-by: yaoyu-33 <[email protected]>
Signed-off-by: yaoyu-33 <[email protected]>
Signed-off-by: yaoyu-33 <[email protected]>
| # Handle Hugging Face GenerationConfig / PretrainedConfig by converting to a callable dict | ||
| # compatible with our YAML representer logic | ||
| try: | ||
| from transformers import GenerationConfig, PretrainedConfig # type: ignore | ||
|
|
||
| if isinstance(val_to_convert, (GenerationConfig, PretrainedConfig)): | ||
| cfg_class = val_to_convert.__class__ | ||
| target = f"{inspect.getmodule(cfg_class).__name__}.{cfg_class.__qualname__}.from_dict" | ||
| logger.debug(f"Converting {cfg_class.__qualname__} at {current_path} to callable dict") | ||
| return { | ||
| "_target_": target, | ||
| "_call_": True, | ||
| "config_dict": val_to_convert.to_dict(), | ||
| } | ||
| except ModuleNotFoundError: | ||
| # transformers is optional; if unavailable, fall through to other handlers | ||
| pass |
There was a problem hiding this comment.
this should be handled here:
Megatron-Bridge/src/megatron/bridge/utils/yaml_utils.py
Lines 62 to 77 in f9aad1f
were you seeing serialization issues outside of this?
There was a problem hiding this comment.
it's because we are using transformers' model config in the config, that's causing some issues here.
There was a problem hiding this comment.
i dont think the yaml would work, here it's not for saving just for omega conf creation
| @@ -0,0 +1,233 @@ | |||
| # Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved. | |||
There was a problem hiding this comment.
should we offer higher-level functions like pretrain_gpt / finetune_gpt / pretrain_qwen25_vl, finetune_qwen25_vl that automatically select the right step for users?
or how else should we associate the custom forward step func with the recipe? otherwise it might not be obvious to users that an implementation exists
previously this was bound to the model config, but we want to keep the model providers independent of the training side
There was a problem hiding this comment.
good point, what do you think let recipe returns cfg and the step function?
There was a problem hiding this comment.
returning the config and the step function will be a breaking change for existing users
| from megatron.bridge.training.gpt_step import ( | ||
| _create_loss_function, | ||
| get_packed_seq_params, | ||
| ) |
There was a problem hiding this comment.
we can spilt these functions out of gpt_step to indicate they are more generic
There was a problem hiding this comment.
i feel gpt_step is the general one lol
There was a problem hiding this comment.
moved them out to utils
Signed-off-by: yaoyu-33 <[email protected]>
Signed-off-by: yaoyu-33 <[email protected]>
Signed-off-by: yaoyu-33 <[email protected]>
Signed-off-by: yaoyu-33 <[email protected]>
Signed-off-by: yaoyu-33 <[email protected]>
Signed-off-by: yaoyu-33 <[email protected]>
Signed-off-by: yaoyu-33 <[email protected]>
| @@ -0,0 +1,233 @@ | |||
| # Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved. | |||
There was a problem hiding this comment.
returning the config and the step function will be a breaking change for existing users
|
|
||
|
|
||
| def forward_step( | ||
| state: GlobalState, data_iterator: Iterable, model: GPTModel, return_schedule_plan: bool = False |
Signed-off-by: yaoyu-33 <[email protected]>
No description provided.