The 4-Bitter Lesson: Balancing Stability and Performance in NVFP4 RL

We have developed and shared a low-precision RL recipe preserving higher-precision training dynamics. In this recipe, we needed to address instability from the forward pass due to policy quantization errors, from the backward pass due to gradient mismatches, and at the intersection of both due to a small set of particularly sensitive weights. We explain how we addressed each and validated the final recipe.