Show HN: NanoRL – RL training for LLMs in ~1,800 lines
One RL training loop that runs CartPole on a laptop and disaggregated async RLVR on a GPU cluster. About 1,800 lines across 7 files. No Ray, no TRL, no DeepSpeed. It's small enough to read in a...
Summary not yet generated for this article.
Original Text
The smallest async RL trainer I could write: one loop that runs REINFORCE on CartPole on a laptop and async GRPO on a cluster
(e.g. 8xH100 trainer, 8 vLLM workers, ran as a [SkyPilot job group](https://docs.skypilot.ai/en/latest/examples/job-groups.html) on k8s ).
All without Ray or TRL or DeepSpeed etc., workers talk to the trainer over stdlib HTTP.
Comments (0 of ) ⌛