Search results

[QUESTION] Is there a way for Megatron to recompute the whole transformer layer except for the flash-attn part?

module: transformer engine

questionFurther information is requestedFurther information is requested

Status: Open.

#1732 In NVIDIA/Megatron-LM;

· xUhEngwAng opened on Aug 6, 2025

[QUESTION] why is optimizer state all in CPU memory in my runs?

questionFurther information is requestedFurther information is requested

Status: Open.

#1730 In NVIDIA/Megatron-LM;

· husam-e opened on Aug 4, 2025

[ROADMAP] Megatron Core MoE Q3-Q4 2025 Roadmap

call for contribution

module: moe

Status: Open.

#1729 In NVIDIA/Megatron-LM;

· yanring opened on Aug 4, 2025

[QUESTION]：moe a2a overlap

questionFurther information is requestedFurther information is requested

Status: Open.

#1728 In NVIDIA/Megatron-LM;

· qiunvlalala opened on Aug 3, 2025

[BUG] Inaccurate FLOPs Calculation for Models with Specialized Attention

bugSomething isn't workingSomething isn't working

Status: Open.

#1725 In NVIDIA/Megatron-LM;

· NuojCheng opened on Aug 1, 2025

[QUESTION]Why are inputs to ColumnParallelLinear different across ranks in register_forward_hook and register_forward_pre_hook

questionFurther information is requestedFurther information is requested

Status: Open.

#1722 In NVIDIA/Megatron-LM;

· UnbearableFate opened on Jul 31, 2025

Benchmarking DeepEP Guide

module: documentation

module: moe

Status: Open.

#1721 In NVIDIA/Megatron-LM;

· sbhavani opened on Jul 31, 2025

[QUESTION]May I allocate and register NCCL userbuffer for activations/gradients transmitted between PP stages?

community-request

module: training

questionFurther information is requestedFurther information is requested

Status: Open.

#1720 In NVIDIA/Megatron-LM;

· yanminjia opened on Jul 31, 2025

[QUESTION] <code>save_checkpoint</code> with <code>expert_tensor_parallel_size</code>

community-request

module: moe

questionFurther information is requestedFurther information is requested

Status: Open.

#1719 In NVIDIA/Megatron-LM;

· jeromeku opened on Jul 30, 2025

v0.12.2 pretrain failed, 0.14.0rc2 model conversion failed

community-request

module: training

questionFurther information is requestedFurther information is requested

Status: Open.

#1718 In NVIDIA/Megatron-LM;

· justalittlenoob opened on Jul 30, 2025

[QUESTION] Purpose of all2all communication for probs in MoEAlltoAllTokenDispatcher

community-request

module: moe

questionFurther information is requestedFurther information is requested

Status: Open.

#1716 In NVIDIA/Megatron-LM;

· snowpeakz opened on Jul 29, 2025

[BUG] OOB layer acces when using 1F1B scheduler

bugSomething isn't workingSomething isn't working

community-request

module: training

Status: Open.

#1715 In NVIDIA/Megatron-LM;

· ColorfulCookie opened on Jul 29, 2025