GitHub - Fortitude-Group/OmnisBench: Open, reproducible benchmark for LLM routing efficiency: verify routing-savings claims yourself. On a fresh split the models can't have memorised, ideal routing beats the frontier model at about 60% lower cost, and every number re-grades offline. Apache-2.0.

Show HN: OmnisBench, a re-gradable, open LLM routing benchmark on fresh tasks

Open, reproducible benchmark for LLM routing efficiency: verify routing-savings claims yourself. On a fresh split the models can't have memorised, ideal routing beats the frontier model at abou...
3 points by fortitudedev 36 days ago github.com 1 comments
Summary not yet generated for this article.

Comments (1 of ) ⌛

Loading more comments... ⌛
Scroll down to load more comments
No comments available.
Type to search HN and engineering blog articles
Press Escape to close  |  ↑↓ to navigate  |  Enter to open