Expert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization
peft-fine-tuning
peft-fine-tuning is an open-source workflow skill for Claude Code and compatible agents, published by Orchestra-Research. Its author describes it as: “Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or…”. The project has 12k stars on GitHub and is available under the MIT license. Add it to your setup with `/plugin marketplace add Orchestra-Research/AI-Research-SKILLs`.
What peft-fine-tuning does
Fine-tune LLMs by training <1% of parameters using LoRA, QLoRA, and 25+ adapter methods.
Installation
Add peft-fine-tuning to your agent with:
/plugin marketplace add Orchestra-Research/AI-Research-SKILLsAlways review a skill's source before installing it. This command comes from the skill's public repository; the linked repo is the source of truth for exact setup steps.
What's inside
The SKILL.md for peft-fine-tuning is organised into these sections:
- When to use PEFT
- Quick start
- Installation
- LoRA fine-tuning (standard)
- QLoRA fine-tuning (memory-efficient)
- LoRA parameter selection
- Rank (r) - capacity vs efficiency
- Alpha (loraalpha) - scaling factor
- Target modules by architecture
- Loading and merging adapters
- Load trained adapter
- Merge adapter into base model
When to use it
Reach for peft-fine-tuning when you want workflow help from your agent without writing the same instructions every session. Load the skill and the agent picks it up automatically for relevant tasks.
Strengths
- Clear MIT license — safe to read and adapt
- Ships in Orchestra-Research/AI-Research-SKILLs, an established project with 11,807 GitHub stars
- Actively maintained (recent commits)
Topics
Frequently asked questions
- What does peft-fine-tuning do?
- Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.
- How do I install peft-fine-tuning?
- Run /plugin marketplace add Orchestra-Research/AI-Research-SKILLs in your agent, then reload your skills. Review the source at https://github.com/Orchestra-Research/AI-Research-SKILLs before installing.
- Is peft-fine-tuning free to use?
- Yes. peft-fine-tuning is free and open source under the MIT license, so you can read, run, and adapt it within that license's terms.
- Where does peft-fine-tuning come from?
- peft-fine-tuning ships inside Orchestra-Research/AI-Research-SKILLs, a repository that contains 41 catalogued skills in total. The repository's 11,807 GitHub stars apply to that whole collection, not to this skill on its own.
Related skills
More Workflow →Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and multimodal models. MLSys 2024 Best Paper Award winner.
Trains large language models (2B-462B parameters) using NVIDIA Megatron-Core with advanced parallelism strategies. Use when training models >1B parameters, need maximum GPU efficiency (47% MFU on H100), or require tensor/pipeline/sequence/context/expert parallelism. Production-ready framework used for Nemotron, LLaMA, DeepSeek.
Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.
Expert guidance for distributed training with DeepSpeed - ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, sparse attention