Fixing Tool-Call Accuracy in vLLM
52 points by coffeemug
Tool call accuracy is critical for effective open-weight AI models, as inaccuracies can lead to nonsensical context. A fork of vLLM has achieved 100% tool call schema accuracy for the Kimi and DeepSeek models. The recent vLLM 0.31 update introduced constrained decoding, which ensures schema adherence and allows users to reach full accuracy with these models.