Configure model fallback chains for high availability — if the primary LLM fails, automatically try the next model in the chain.
!Model Fallback Configuration
When a request is sent:
1. Try the primary model first
2. If it fails (rate limit, timeout, error), try fallback 1
3. Continue down the chain until a model responds
4. If all models fail, return an error
The ModelFallbackManager tries to discover models automatically:
ModelFallbackManager
1. LiteLLM — If installed, reads litellm.model_list for configured models
litellm.model_list
2. Static catalog — Falls back to common models (gpt-4o, gpt-4o-mini, claude-3-5-sonnet, gemini-2.0-flash)
| Endpoint | Method | Description |
|----------|--------|-------------|
| /api/models | GET | List available models + current fallback chain |
/api/models
| /api/models/fallback | PUT | Set the fallback chain |
/api/models/fallback