← Explore all skills

MACHINE LEARNING / huggingface

TRL Training API

Select TRL trainers and dataset shapes, then configure SFT or online preference/reward training in Python or the CLI.

“Post-train LLMs with TRL (Transformers Reinforcement Learning) — SFT, DPO, GRPO, KTO, and reward-model training. Use when writing or debugging training code with the TRL”

01 / THE REASONING

Why this made the selection.

  • It maps each stable TRL trainer to a dataset shape and says the chat template is applied by the library.
  • RewardTrainer trains a scalar reward model rather than a policy, and experimental trainers are marked unstable.
  • Maps trainer choices to language-modeling, paired/unpaired preference and prompt-only data.
  • Details reward-function inputs and generation-batch constraints instead of treating all methods as interchangeable.

02 / THE REVIEW RECORD

What we actually inspected.

Source review has boundaries.
A clear record is more useful than a “safe” badge.

Material inspected

  • skills/trl-training/SKILL.md
  • README.md
  • LICENSE

Our findings

  • Each listed method pairs a Trainer class with a Config dataclass that extends TrainingArguments.
  • Datasets are loaded through the datasets library, so training data can come from the Hub rather than a local file.
  • The chat template is applied automatically, and the skill says never to apply it yourself.
  • RewardTrainer does not train a policy, and experimental trainers are called unstable.
  • Definition distinguishes scalar reward-model training from policy training.
  • GRPO rewards accept extra dataset columns and return one reward per completion; conversational completions are message lists.

Not established by this review

  • The resource was not executed and its tools were not called.
  • No skill execution, authenticated service operation, or outcome benchmark was performed.

The review applies to the material and revision named here. A newer upstream release can change its behavior.

03 / PUT IT TO WORK

Add a skill to your workflow.

Upstream setup instructions ↗

DOCUMENTED COMMAND

npx skills add huggingface/skills/skills/trl-training

Run in your terminal to choose your AI tools, project or global scope, and installation method. This short command follows the current upstream version; the review covers the revision shown above.

Optional: install the reviewed revision

Keeps the original source and installer version where recorded, while leaving your AI tools and installation scope for you to choose.

npx skills add https://github.com/huggingface/skills/tree/c3ff942be3e07116b475253f0a8fc4f54682c1a8/skills/trl-training
  1. Install the pinned skill directory interactively, retaining its bundled references and scripts.
  2. Choose the method from the dataset contract and inspect current trainer/config fields.
  3. Use a small representative training pass; check reward cardinality and generation-batch divisibility before scaling.

Before you start

  • Python with compatible TRL/Transformers and optional PEFT/vLLM
  • Task-appropriate data, model access and compute

04 / FOLLOW THE EVIDENCE

The source trail.

Our notes are separate from the original resource.
Check upstream before adopting a new version.

  1. Pinned skill definition ↗Checked

    https://github.com/huggingface/skills/blob/c3ff942be3e07116b475253f0a8fc4f54682c1a8/skills/trl-training/SKILL.md

    Supports: summary, whySelected, bestFor, limitations, install, access, review

  2. https://github.com/huggingface/skills/blob/c3ff942be3e07116b475253f0a8fc4f54682c1a8/README.md

    Supports: install

  3. Apache-2.0 licence ↗Checked

    https://github.com/huggingface/skills/blob/c3ff942be3e07116b475253f0a8fc4f54682c1a8/LICENSE

    Supports: license

Evidence & Ask