Overview
pytorch-fsdp2 is an open-source data skill for Claude Code and compatible agents, published by Orchestra-Research. Its author describes it as: “Adds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and distributed checkpointing. Use when models exceed single-GPU memory or when you need DTensor-based…”. The project has 12k stars on GitHub and is available under the MIT license. Add it to your setup with `/plugin marketplace add Orchestra-Research/AI-Research-SKILLs`.
What pytorch-fsdp2 does
This skill teaches a coding agent how to **add PyTorch FSDP2** to a training loop with correct initialization, sharding, mixed precision/offload configuration, and checkpointing.
What’s inside
The SKILL.md for pytorch-fsdp2 is organised into these sections:
- When to use this skill
- Alternatives (when FSDP2 is not the best fit)
- Contract the agent must follow
- Step-by-step procedure
- 0) Version & environment sanity
- 1) Initialize distributed and set device
- 2) Build model on meta device (recommended for very large models)
- 3) Apply fullyshard() bottom-up (wrapping policy = “apply where needed”)
- 4) Configure reshardafterforward for memory/perf trade-offs
- 5) Mixed precision & offload (optional but common)
- 6) Optimizer, gradient clipping, accumulation
- 7) Checkpointing: prefer DCP or distributed state dict helpers
Installation
Add pytorch-fsdp2 to your agent with:
/plugin marketplace add Orchestra-Research/AI-Research-SKILLsReview a skill’s source before installing it — skills can instruct your agent to run commands. This command comes from the skill’s public repository, which is the source of truth for setup steps. New to skills? Read how to install Claude Code skills.
Supported agents
- Claude Code — ships a claude code plugin marketplace
- Codex — declared in the repository topics
License
pytorch-fsdp2 is published under the MIT license.
Strengths
- Clear MIT license — safe to read and adapt within its terms
- Ships in Orchestra-Research/AI-Research-SKILLs, an established project with 11,807 GitHub stars
- Works with Claude Code, Codex
Author
Topics
Frequently asked questions
- What does pytorch-fsdp2 do?
- Adds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and distributed checkpointing. Use when models exceed single-GPU memory or when you need DTensor-based sharding with DeviceMesh.
- How do I install pytorch-fsdp2?
- Run /plugin marketplace add Orchestra-Research/AI-Research-SKILLs in your agent, then reload your skills. Review the source at https://github.com/Orchestra-Research/AI-Research-SKILLs before installing.
- Is pytorch-fsdp2 free to use?
- Yes. pytorch-fsdp2 is free and open source under the MIT license, so you can read, run, and adapt it within that license's terms.
- Which agents does pytorch-fsdp2 work with?
- pytorch-fsdp2 lists Claude Code and Codex — named in its SKILL.md, its plugin marketplace or its repository topics. Skills use the portable SKILL.md format, so other agents that read that format may load it too.
- Where does pytorch-fsdp2 come from?
- pytorch-fsdp2 ships inside Orchestra-Research/AI-Research-SKILLs, a repository that contains 41 catalogued skills in total. The repository's 11,807 GitHub stars apply to that whole collection, not to this skill on its own.
