slime is an LLM training framework for RL scaling, providing two core capabilities:
- High-Performance Training: Supports efficient training in various modes by connecting Megatron with SGLang;
- Flexible Data Generation: Enables arbitrary training data generation workflows through custom data generation interfaces and server-based engines.
slime's design goal is to make these two capabilities reinforce each other without turning the system into a heavy stack of disconnected trainers, rollout services, and agent frameworks. Megatron training, SGLang rollout, custom data generation, reward computation, verifier feedback, and environment interaction all flow through the same training / rollout / Data Buffer path.
This makes slime one of the most battle-tested open RL post-training frameworks: small enough to understand and extend, but validated through complete training loops behind SOTA-level model releases.
- Production experience: slime is the RL framework behind GLM-5.3-Flash, GLM-5.3, GLM-5.2, GLM-5.1, GLM-5, GLM-4.7, GLM-4.6, and GLM-4.5.
- Direct engine integration: Megatron arguments are available directly; SGLang arguments use the
--sglang-prefix. - Customizable data generation: generation functions, reward functions, verifiers, and environments connect through documented interfaces.
- Correctness and reliability: separate rollout-only and train-only debugging, reproducibility controls, recovery, and CPU/GPU tests support long-running experiments.
Alongside the GLM family, slime supports Qwen (Qwen3.8, Qwen3.6, Qwen3.5, Qwen3-Next, Qwen3 MoE, Qwen3, Qwen2.5), DeepSeek (V3, V3.1, R1), and Llama 3. See the model configurations in scripts/models and the training examples.
Use Megatron arguments directly for parallelism, optimizers, checkpoints, and model configuration. Prefix installed SGLang arguments with --sglang-; for example, --mem-fraction-static becomes --sglang-mem-fraction-static.
For more involved deployments:
- SGLang Config: YAML configuration for server groups, multiple models, and per-group overrides.
- PD Disaggregation: separate prefill and decode resources.
- Delta Weight Sync: send changed weight bytes over shared storage.
- External Rollout Engines: connect serving processes managed outside the training job, including disk-based updates across different GPU fleets.
CPU tests cover core behavior and customization contracts. GPU tests exercise dense and MoE training, rollout deployment, checkpointing, precision, fully asynchronous rollout, distillation, and debug replay. See CI for the test matrix and how to run it.
Engineering guides: Debugging, Reproducibility, Fault Tolerance, Tracing, and Profiling.
- slime: An SGLang-Native Post-Training Framework for RL Scaling
- Agent-Oriented Design: An Asynchronous and Decoupled Framework for Agentic RL
- slime v0.1.0: Redefining High-Performance RL Training Frameworks
- Architecture Overview
- Quick Start
- Ecosystem Built on slime
- Arguments Walkthrough
- Code Reading Path
- Developer Guide
- FAQ & Acknowledgements
Module Descriptions:
- training (Megatron): Responsible for the main training process, reads data from the Data Buffer, and synchronizes parameters to the rollout module after training.
- rollout (SGLang + router): Generates new data (including rewards/verifier outputs) and stores it in the Data Buffer. Custom generate functions can wrap this with multi-turn loops, tool calls, environment/sandbox interaction, and verifier-based reward.
- data buffer: A bridge module that manages prompt initialization, custom data, and rollout generation methods (including agentic workflows that produce samples through the same interface).
The default payload transport is Ray object-store. Selecting --rollout-data-transport straw uses straw for persistent prompt tasks, rollout continuations and training batches on shared JuiceFS storage. Ray carries control messages and references; generation and training processes read and write packed payloads directly. See the straw usage and recovery guide.
Follow the Quick Start Guide for environment setup, data preparation, and your first training run.
We also provide examples for some use cases not covered in the quick start guide; please check examples.
These agentic RL examples use the standard rollout and data buffer interfaces:
examples/multi_agent: Multi-agent generation via--custom-generate-function-pathinside the standard rollout loop.examples/search-r1: Search/RAG-style multi-turn generation via--custom-generate-function-path.examples/fully_async: Fully-async rollout, useful for long-tail agentic generation where some samples take much longer than others.examples/coding_agent_rl: Qwen3.8-27B coding-agent RL on Xiaomi MiMo SWE data, using sunabako sandboxes, official test rewards, and token-correct trajectories.
See the Customization Guide for which interface to use for a given agentic workflow.
These are not just demos. They are independent systems that use slime as a reusable RL substrate for production-scale post-training, agentic RL, domain RL, and rollout-system research.
Dressage is an agentic RL training framework built on slime by Alibaba Accio, centered on unified RL for blackbox agents (e.g., OpenCode, OpenClaw) and white loops across any sandbox environment (e.g., bwrap, E2B, Kubernetes). It decouples interaction semantics, execution placement, and token-level trajectory capture through Paddock, Sandbox, and Proxy layers, adapting agent workflows without rewriting their internal loops. Dressage records token-wise logprobs, loss masks, weight versions, and MoE routing, then uses TITO and segment-aware training to turn long-horizon tool interactions into stable RL samples.
Miles is an RL post-training framework for large-scale models, built on slime by RadixArk. It stays closely aligned with slime's upstream development while extending it with enterprise-oriented features: deeper SGLang integration, operational tooling, deployment support, and optimizations for new models and hardware. Miles also adds a growing set of production features, including LoRA, TITO, and low-precision training.
vime is a post-training framework built on slime and maintained by the vLLM project. It keeps slime's Megatron training stack, Data Buffer dataflow, and custom data-generation design, with its main change being a rollout backend swapped to vLLM with vllm-router. Starting from an existing slime launch script, adjusting only rollout-related parameters is enough to quickly run training with vime.
Relax (Reinforcement Engine Leveraging Agentic X-modality) is an omni-modal agentic RL framework open-sourced by the RedAI Infra team, built upon the slime infrastructure stack that combines Ray, Megatron-LM, and SGLang. Relax adopts a service-oriented architecture on Ray Serve with Megatron-LM and SGLang as training/inference backends. It uses TransferQueue to fully decouple Actor, Rollout, ActorFwd, Reference, and Advantage computation onto independent GPU clusters, and introduces DCS (Distributed Checkpoint Service) — an NCCL-broadcast weight-sync engine that streams updated Actor weights to Rollout/ActorFwd/Reference asynchronously and overlaps the transfer with the next training step, enabling fully-async training at configurable staleness. Relax supports end-to-end RL for text, vision, and audio (including Qwen3-Omni) and agentic multi-turn rollouts.
OpenClaw-RL is an RL server for personalized OpenClaw agents. It hosts the OpenClaw model and improves it from prior conversations across deployments, while slime's decoupled RL infrastructure prevents training from interfering with API serving. It supports two automatic optimization methods: GRPO with binary feedback inferred from subsequent states, and on-policy distillation that extracts hindsight hints from later feedback for the current policy.
P1 is a family of open-source physics reasoning models trained entirely through reinforcement learning. P1 leverages slime as the RL post-training framework, and introduces a multi-stage RL training algorithm that progressively enhances reasoning ability through adaptive learnability adjustment and stabilization mechanisms. Empowered by this training paradigm, P1 delivers breakthrough performance in open-source physics reasoning.
RLVE introduces an approach using verifiable environments that procedurally generate problems and provide algorithmically verifiable rewards, to scale up RL for language models (LMs). With joint training across 400 verifiable environments, RLVE enables each environment to dynamically adapt its problem difficulty distribution to the policy model's capabilities as training progresses.
TritonForge leverages slime's SFT and RL capabilities to train LLMs that automatically generate optimized GPU kernels. By using a two-stage training approach—supervised fine-tuning followed by reinforcement learning with multi-turn compilation feedback—TritonForge achieves remarkable results in converting PyTorch operations into high-performance Triton kernels.
APRIL introduces a system-level optimization that seamlessly integrates with slime to accelerate the rollout generation phase in RL training. By intelligently over-provisioning requests and actively managing partial completions, APRIL addresses the long-tail generation bottleneck that typically consumes over 90% of RL training time.
qqr (a.k.a. hilichurl) is a lightweight extension for slime designed to evolve open-ended agents. It implements the ArenaRL algorithm to tackle discriminative collapse through tournament-based relative ranking (e.g., Seeded Single-Elimination, Round-Robin) and seamlessly integrates the Model Context Protocol (MCP). qqr leverages slime's high-throughput training capabilities to enable scalable, distributed evolution of agents in standardized, decoupled tool environments.
ART (AgentCore RL Toolkit) is an SDK that adapts production agents for RL training on AWS Bedrock AgentCore Runtime. AgentCore Runtime provides auto-scaled and sandboxed agent execution environments well-suited for running many parallel agent rollouts securely. Using ART, user only needs to apply a decorator (@app.rollout_entrypoint) to their agent codes for RL adaption while the same production agent harness is reused directly, where token capture for RL is handled at model gateway layer. ART uses slime as one option of training backends, enabling users to easily optimizing the production agent model with RL training algorithms in slime.
Together, these projects show the main idea behind slime: one high-performance RL kernel can support frontier model post-training, online agent optimization, verifiable environments, omni-modal rollouts, kernel-generation agents, and rollout-system research without changing the core training loop.
Arguments in slime are divided into three categories:
- Megatron arguments: slime reads Megatron arguments directly. You can configure Megatron by passing arguments like
--tensor-model-parallel-size 2. - SGLang arguments: All arguments for the installed SGLang are supported through pass-through. These arguments must be prefixed with
--sglang-. For example,--mem-fraction-staticshould be passed as--sglang-mem-fraction-static. - slime-specific arguments: Please refer to: slime/utils/arguments.py
For complete usage instructions, please refer to the Usage Documentation.
Start from the training loop and follow the calls only as deep as needed:
train.py: train
├─ slime/ray/placement_group.py Ray resource and worker initialization
├─ slime/ray/rollout.py RolloutManager.generate: rollout orchestration
│ └─ slime/rollout/sglang_rollout.py Sample generation and reward computation
└─ slime/ray/actor_group.py RayTrainGroup.async_train: training dispatch
└─ slime/backends/megatron_utils/actor.py
├─ model.py Megatron model execution
└─ loss.py RL losses and advantages
On a first pass, treat slime/utils/arguments.py as the configuration entry point. The deployment details in slime/backends/sglang_utils/ and the weight-sync implementations under slime/backends/megatron_utils/update_weight/ can also wait until you need to change those areas.
-
Read the contribution guidelines before submitting an issue or PR.
-
Use pre-commit to ensure code style consistency for your commits:
apt install pre-commit -y
pre-commit install
# run pre-commit to ensure code style consistency
pre-commit run --all-files --show-diff-on-failure --color=always- For debugging tips, please refer to the Debugging Guide
- For frequently asked questions, please see the Q&A
- Special thanks to the following projects & communities: SGLang, Megatron‑LM, mbridge, OpenRLHF, veRL, Pai-Megatron-Patch and others.
- To quote slime, please use:
@misc{slime_github,
author = {Zilin Zhu and Chengxing Xie and Xin Lv and slime Contributors},
title = {slime: An LLM post-training framework for RL Scaling},
year = {2025},
howpublished = {\url{https://github.com/THUDM/slime}},
note = {GitHub repository. Corresponding author: Xin Lv},
urldate = {2025-06-19}
}