Show HN: FlexInference LLM Router
FlexInference is a deadline-aware, OpenAI-compatible LLM router for OpenAI, Gemini, and Anthropic, with cloud routing via Amazon Bedrock and Google Vertex AI.
Summary not yet generated for this article.
Original Text
Hey everyone, just added Bedrock and Vertex support to FlexInference. It now can do provider fallback on 429 and 5xx. Plus there is upstream retries you can setup via SDK. This way you can do routing for free through OpenAI, Anthropic, Google, Vertex, and Bedrock. Working on Foundry support next.
Comments (0 of ) ⌛