What is Model Routing?
Sending each request to the cheapest model that can handle it: light chat models for drafting, premium tool models for code actions. BYOB separates chat and tool configs per provider so sessions route without user intervention.
Example
A BYOB session drafts copy with a light chat model and switches to the stronger tool model only for code edits and structured actions — routing keeps everyday turns cheap while reserving firepower for hard steps.
What people get wrong
Pinning every call to the flagship model. Maximum capability on trivial turns burns budget that routing would have saved with zero quality loss.
Related terms
Provider Fallback
Retrying a failed model call on a backup provider or model instead of erroring the session. Fallbacks trade exact behavior continuity for uptime, so builders log which leg actually served each turn.
Model Lock
Keeping model selection stable for a chat session so decisions and style stay coherent. Mid-session model drift changes code idioms silently; locking prevents it.
Temperature
Sampling randomness for generation: low values make output focused and repeatable, high values make it varied and surprising. BYOB chat configs default near 0.5 while tool calls use 0.3 for steadier structured actions.
Prompt Pipeline
A staged chain of model calls where each stage output feeds the next step: classify, then draft, then verify. Pipelines beat single giant prompts on reliability, and each stage can use the cheapest capable model.
Thinking Budget
The token allowance reserved for internal model reasoning before answering. Larger budgets suit hard multi-step problems; BYOB prices thinking tokens separately so runaway reasoning shows up in metering.
Structured Output
Forcing model replies into a machine-readable shape such as JSON with fixed fields. Required wherever AI output feeds code, databases, or UI components, since free text breaks parsers the moment formatting drifts.