Scalable data processing for ML workloads. Streaming execution across CPU/GPU, supports Parquet/CSV/JSON/images. Integrates with Ray Train, PyTorch, TensorFlow. Scales from single machine to 100s of nodes. Use for batch inference, data preprocessing, multi-modal data loading, or distributed ETL pipelines.
pytorch-fsdp2
pytorch-fsdp2 is an open-source data skill for Claude Code and compatible agents, published by Orchestra-Research. Its author describes it as: “Adds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and distributed checkpointing. Use when models exceed single-GPU memory or when you need DTensor-based…”. The project has 12k stars on GitHub and is available under the MIT license. Add it to your setup with `/plugin marketplace add Orchestra-Research/AI-Research-SKILLs`.
What pytorch-fsdp2 does
This skill teaches a coding agent how to **add PyTorch FSDP2** to a training loop with correct initialization, sharding, mixed precision/offload configuration, and checkpointing.
Installation
Add pytorch-fsdp2 to your agent with:
/plugin marketplace add Orchestra-Research/AI-Research-SKILLsAlways review a skill's source before installing it. This command comes from the skill's public repository; the linked repo is the source of truth for exact setup steps.
What's inside
The SKILL.md for pytorch-fsdp2 is organised into these sections:
- When to use this skill
- Alternatives (when FSDP2 is not the best fit)
- Contract the agent must follow
- Step-by-step procedure
- 0) Version & environment sanity
- 1) Initialize distributed and set device
- 2) Build model on meta device (recommended for very large models)
- 3) Apply fullyshard() bottom-up (wrapping policy = “apply where needed”)
- 4) Configure reshardafterforward for memory/perf trade-offs
- 5) Mixed precision & offload (optional but common)
- 6) Optimizer, gradient clipping, accumulation
- 7) Checkpointing: prefer DCP or distributed state dict helpers
When to use it
Reach for pytorch-fsdp2 when you want data help from your agent without writing the same instructions every session. Load the skill and the agent picks it up automatically for relevant tasks.
Strengths
- Clear MIT license — safe to read and adapt
- Ships in Orchestra-Research/AI-Research-SKILLs, an established project with 11,807 GitHub stars
- Actively maintained (recent commits)
Topics
Frequently asked questions
- What does pytorch-fsdp2 do?
- Adds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and distributed checkpointing. Use when models exceed single-GPU memory or when you need DTensor-based sharding with DeviceMesh.
- How do I install pytorch-fsdp2?
- Run /plugin marketplace add Orchestra-Research/AI-Research-SKILLs in your agent, then reload your skills. Review the source at https://github.com/Orchestra-Research/AI-Research-SKILLs before installing.
- Is pytorch-fsdp2 free to use?
- Yes. pytorch-fsdp2 is free and open source under the MIT license, so you can read, run, and adapt it within that license's terms.
- Where does pytorch-fsdp2 come from?
- pytorch-fsdp2 ships inside Orchestra-Research/AI-Research-SKILLs, a repository that contains 41 catalogued skills in total. The repository's 11,807 GitHub stars apply to that whole collection, not to this skill on its own.
Related skills
More Data →Anthropic's method for training harmless AI through self-improvement. Two-phase approach - supervised learning with self-critique/revision, then RLAIF (RL from AI Feedback). Use for safety alignment, reducing harmful outputs without human labels. Powers Claude's safety system.
Expert guidance for fine-tuning LLMs with Axolotl - YAML configs, 100+ models, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, multimodal support
GPU-accelerated data curation for LLM training. Supports text/image/video/audio. Features fuzzy deduplication (16× faster), quality filtering (30+ heuristics), semantic deduplication, PII redaction, NSFW detection. Scales across GPUs with RAPIDS. Use for preparing high-quality training datasets, cleaning web data, or deduplicating large corpora.
Expert data processing specialist with intelligent DuckDB/Polars selection for maximum performance. Always includes numpy, never uses pandas, runs everything through uv. Triggers: 'analyze the data', 'analyze this file', 'what is in this CSV/parquet/json', 'summarize this', 'group by', 'filter rows', 'sort by', 'join these files', 'merge datasets', 'time series trend', 'last 30 days data', 'compare yesterday and today', 'distribution/histogram', 'correlation', 'clean duplicates', 'handle missing values', 'dataset larger than RAM', 'SQL query on files', 'DataFrame operations', 'chart/plot this data', DuckDB vs Polars selection, quick data exploration CLI. NOT for plain text/code inspection, configs, or tiny inline math.