Remove FSDP1 support: use FSDP2 exclusively #4260
Open
+9
−185
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
With FSDP2 now stable, this commit removes all FSDP1-specific code to reduce maintenance burden. Changes: - Removed _sync_fsdp1_params_to_vllm() method from RLOO, GRPO, and Online DPO trainers - Removed fsdp_version detection and conditional logic in all trainers - Replaced FSDP1 version checks with direct calls to _sync_fsdp2_params_to_vllm() - Deleted fsdp1.yaml config files from trl/accelerate_configs and examples/accelerate_configs - Updated docs/source/clis.md to remove fsdp1 from predefined config profiles table Technical Details: - FSDP1 used recursive post-order traversal with FSDP.summon_full_params() - FSDP2 uses module.state_dict() which covers all parameters automatically - All vLLM weight synchronization now uses FSDP2 methods exclusively
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
It seems like QLoRA isn't supported for FSDP2, let's make this on hold |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
With FSDP2 now stable, this PR removes all FSDP1-specific code to reduce maintenance burden.
Changes
Code Changes
_sync_fsdp1_params_to_vllm()method from RLOO, GRPO, and Online DPO trainers_sync_fsdp2_params_to_vllm()Configuration Changes
trl/accelerate_configs/fsdp1.yamlexamples/accelerate_configs/fsdp1.yamlDocumentation Changes
docs/source/clis.mdto removefsdp1from predefined config profiles tableTechnical Details
FSDP1 Implementation (Removed):
FSDP.summon_full_params(module, recurse=False, writeback=False)for each moduleFSDP2 Implementation (Now Standard):
module.state_dict()which automatically handles all parametersFiles Changed
trl/trainer/rloo_trainer.pytrl/trainer/grpo_trainer.pytrl/trainer/online_dpo_trainer.pydocs/source/clis.mdtrl/accelerate_configs/fsdp1.yaml(deleted)examples/accelerate_configs/fsdp1.yaml(deleted)