Show HN: Changed how I use agent harnesses

Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS. - maximhq/bifrost
2 points by Swapnoneel 80 days ago github.com 0 comments
Summary not yet generated for this article.

Original Text

I’ve been using OpenCode for quite a while now, and I’m in love with it. The subscription cost of Claude Code or Codex doesn’t make much sense to me, so I prefer the free tier of OpenCode Zen!

But for planning and drafting I do need capable models, which I grab from Zenmux or similar providers, and those are heavily rate-limited. I’d get most of the way through a task and hit a limit, which meant stopping, opening the config, swapping the provider, restarting, and picking the task back up from wherever it died. Three or four times per complex task. Losing the quota was annoying. Losing my place in the task was worse.

What I settled on was putting a gateway in front of everything and pointing OpenCode at that instead of at any single provider. One base URL in the OpenCode config, and the routing happens behind it. When one provider 429s, the request goes to the next one with headroom.

I tried LiteLLM first. It works, and if your stack is already Python it’s probably the more obvious pick.

But, I recently found out about Bifrost, and switched from LiteLLM. What made it fit my setup is that it’s a single Go binary with a local SQLite config store, so there’s no separate database service to run alongside it. I’ve got 9 providers configured with fallbacks between them, which is the part that actually fixed the interruptions.

Sharing in case it helps anyone with the same problem. Took me a while to land on this.

Comments (0 of ) ⌛

Loading more comments... ⌛
Scroll down to load more comments
No comments available.
Type to search HN and engineering blog articles
Press Escape to close  |  ↑↓ to navigate  |  Enter to open