Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.
awq-quantization
awq-quantization is an open-source workflow skill for Claude Code and compatible agents, published by Orchestra-Research. Its author describes it as: “Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with b…”. The project has 12k stars on GitHub and is available under the MIT license. Add it to your setup with `/plugin marketplace add Orchestra-Research/AI-Research-SKILLs`.
What awq-quantization does
4-bit quantization that preserves salient weights based on activation patterns, achieving 3x speedup with minimal accuracy loss.
Installation
Add awq-quantization to your agent with:
/plugin marketplace add Orchestra-Research/AI-Research-SKILLsAlways review a skill's source before installing it. This command comes from the skill's public repository; the linked repo is the source of truth for exact setup steps.
What's inside
The SKILL.md for awq-quantization is organised into these sections:
- When to use AWQ
- Quick start
- Installation
- Load pre-quantized model
- Quantize your own model
- AWQ vs GPTQ vs bitsandbytes
- Kernel backends
- GEMM (default, batch inference)
- GEMV (single-token generation)
- Marlin (Ampere+ GPUs)
- ExLlamaV2 (AMD compatible)
- HuggingFace Transformers integration
When to use it
Reach for awq-quantization when you want workflow help from your agent without writing the same instructions every session. Load the skill and the agent picks it up automatically for relevant tasks.
Strengths
- Clear MIT license — safe to read and adapt
- Ships in Orchestra-Research/AI-Research-SKILLs, an established project with 11,807 GitHub stars
- Actively maintained (recent commits)
Topics
Frequently asked questions
- What does awq-quantization do?
- Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and multimodal models. MLSys 2024 Best Paper Award winner.
- How do I install awq-quantization?
- Run /plugin marketplace add Orchestra-Research/AI-Research-SKILLs in your agent, then reload your skills. Review the source at https://github.com/Orchestra-Research/AI-Research-SKILLs before installing.
- Is awq-quantization free to use?
- Yes. awq-quantization is free and open source under the MIT license, so you can read, run, and adapt it within that license's terms.
- Where does awq-quantization come from?
- awq-quantization ships inside Orchestra-Research/AI-Research-SKILLs, a repository that contains 41 catalogued skills in total. The repository's 11,807 GitHub stars apply to that whole collection, not to this skill on its own.
Related skills
More Workflow →Expert guidance for distributed training with DeepSpeed - ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, sparse attention
Trains large language models (2B-462B parameters) using NVIDIA Megatron-Core with advanced parallelism strategies. Use when training models >1B parameters, need maximum GPU efficiency (47% MFU on H100), or require tensor/pipeline/sequence/context/expert parallelism. Production-ready framework used for Nemotron, LLaMA, DeepSeek.
Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.
Expert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization