What is Provider Fallback?
Retrying a failed model call on a backup provider or model instead of erroring the session. Fallbacks trade exact behavior continuity for uptime, so builders log which leg actually served each turn.
Example
If the primary chat provider times out mid-build, the session retries on a backup provider chat config and keeps working — the meter records the fallback model rates for the affected turns.
What people get wrong
Assuming fallback output matches the primary exactly. Different models phrase, format, and reason differently, so verify resumed work instead of trusting continuity.
Related terms
Model Routing
Sending each request to the cheapest model that can handle it: light chat models for drafting, premium tool models for code actions. BYOB separates chat and tool configs per provider so sessions route without user intervention.
Model Lock
Keeping model selection stable for a chat session so decisions and style stay coherent. Mid-session model drift changes code idioms silently; locking prevents it.
Token Metering
Per-leg usage accounting that turns raw token counts into billable credits using each model price table plus markup. BYOB meters input, output, cached, and thinking tokens separately, with a minimum charge flooring tiny calls.
Temperature
Sampling randomness for generation: low values make output focused and repeatable, high values make it varied and surprising. BYOB chat configs default near 0.5 while tool calls use 0.3 for steadier structured actions.
Thinking Budget
The token allowance reserved for internal model reasoning before answering. Larger budgets suit hard multi-step problems; BYOB prices thinking tokens separately so runaway reasoning shows up in metering.
Structured Output
Forcing model replies into a machine-readable shape such as JSON with fixed fields. Required wherever AI output feeds code, databases, or UI components, since free text breaks parsers the moment formatting drifts.