What is Temperature?
Sampling randomness for generation: low values make output focused and repeatable, high values make it varied and surprising. BYOB chat configs default near 0.5 while tool calls use 0.3 for steadier structured actions.
Related terms
Top-k and Top-p
Sampling filters that limit which tokens the model may pick: top-k keeps the k likeliest tokens, top-p keeps the smallest set covering probability p. They trim the weird tail without flattening style like low temperature does.
Model Routing
Sending each request to the cheapest model that can handle it: light chat models for drafting, premium tool models for code actions. BYOB separates chat and tool configs per provider so sessions route without user intervention.
System Prompt
The persistent instruction block prepended to every request that sets role, rules, and output format. It outranks user text in well-built stacks, which is why builders version it like code, not copy.
Thinking Budget
The token allowance reserved for internal model reasoning before answering. Larger budgets suit hard multi-step problems; BYOB prices thinking tokens separately so runaway reasoning shows up in metering.
Structured Output
Forcing model replies into a machine-readable shape such as JSON with fixed fields. Required wherever AI output feeds code, databases, or UI components, since free text breaks parsers the moment formatting drifts.
JSON Schema Mode
A structured-output setting where the caller supplies a JSON Schema and the model must conform to it. Types, required fields, and enums are validated automatically, which removes most post-processing repair code.