ray-train

Distributed training orchestration across clusters. Scales PyTorch/TensorFlow/HuggingFace from laptop to 1000s of nodes. Built-in hyperparameter tuning with Ray Tune, fault tolerance, elastic scaling. Use when training massive models across multiple machines or running distributed hyperparameter sweeps.

by Orchestra-Research 12k GitHub stars MIT Updated Jun 16, 2026
ray-train skill by Orchestra-Research
ray-train — Workflow skill by Orchestra-Research

Overview

ray-train is an open-source workflow skill for Claude Code and compatible agents, published by Orchestra-Research. Its author describes it as: “Distributed training orchestration across clusters. Scales PyTorch/TensorFlow/HuggingFace from laptop to 1000s of nodes. Built-in hyperparameter tuning with Ray Tune, fault tolerance, elastic scaling. Use when trainin…”. The project has 12k stars on GitHub and is available under the MIT license. Add it to your setup with `/plugin marketplace add Orchestra-Research/AI-Research-SKILLs`.

What ray-train does

Ray Train scales machine learning training from single GPU to multi-node clusters with minimal code changes.

What’s inside

The SKILL.md for ray-train is organised into these sections:

  • Quick start
  • Common workflows
  • Workflow 1: Scale existing PyTorch code
  • Workflow 2: HuggingFace Transformers integration
  • Workflow 3: Hyperparameter tuning with Ray Tune
  • Workflow 4: Checkpointing and fault tolerance
  • Workflow 5: Multi-node training
  • When to use vs alternatives
  • Common issues
  • Advanced topics
  • Hardware requirements
  • Resources

Installation

Add ray-train to your agent with:

/plugin marketplace add Orchestra-Research/AI-Research-SKILLs

Review a skill’s source before installing it — skills can instruct your agent to run commands. This command comes from the skill’s public repository, which is the source of truth for setup steps. New to skills? Read how to install Claude Code skills.

Supported agents

  • Claude Code — ships a claude code plugin marketplace
  • Codex — declared in the repository topics

License

ray-train is published under the MIT license.

Strengths

  • Clear MIT license — safe to read and adapt within its terms
  • Ships in Orchestra-Research/AI-Research-SKILLs, an established project with 11,807 GitHub stars
  • Works with Claude Code, Codex

Author

Orchestra-Research 41 skills in the catalog · GitHub profile ↗
View skills

Topics

Need a skill like this, tailored to your team?Build a custom skill →

Frequently asked questions

What does ray-train do?
Distributed training orchestration across clusters. Scales PyTorch/TensorFlow/HuggingFace from laptop to 1000s of nodes. Built-in hyperparameter tuning with Ray Tune, fault tolerance, elastic scaling. Use when training massive models across multiple machines or running distributed hyperparameter sweeps.
How do I install ray-train?
Run /plugin marketplace add Orchestra-Research/AI-Research-SKILLs in your agent, then reload your skills. Review the source at https://github.com/Orchestra-Research/AI-Research-SKILLs before installing.
Is ray-train free to use?
Yes. ray-train is free and open source under the MIT license, so you can read, run, and adapt it within that license's terms.
Which agents does ray-train work with?
ray-train lists Claude Code and Codex — named in its SKILL.md, its plugin marketplace or its repository topics. Skills use the portable SKILL.md format, so other agents that read that format may load it too.
Where does ray-train come from?
ray-train ships inside Orchestra-Research/AI-Research-SKILLs, a repository that contains 41 catalogued skills in total. The repository's 11,807 GitHub stars apply to that whole collection, not to this skill on its own.

Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.

Claude CodeCodex
12k GitHub stars MIT Updated Jun 16, 2026 Free

Expert guidance for distributed training with DeepSpeed - ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, sparse attention

Claude CodeCodex
12k GitHub stars MIT Updated Jun 16, 2026 Free

Trains large language models (2B-462B parameters) using NVIDIA Megatron-Core with advanced parallelism strategies. Use when training models >1B parameters, need maximum GPU efficiency (47% MFU on H100), or require tensor/pipeline/sequence/context/expert parallelism. Production-ready framework used for Nemotron, LLaMA, DeepSeek.

Claude CodeCodex
12k GitHub stars MIT Updated Jun 16, 2026 Free

Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.

Claude CodeCodex
12k GitHub stars MIT Updated Jun 16, 2026 Free

Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and multimodal models. MLSys 2024 Best Paper Award winner.

Claude CodeCodex
12k GitHub stars MIT Updated Jun 16, 2026 Free