What is Thinking Budget?
The token allowance reserved for internal model reasoning before answering. Larger budgets suit hard multi-step problems; BYOB prices thinking tokens separately so runaway reasoning shows up in metering.
Related terms
Token Metering
Per-leg usage accounting that turns raw token counts into billable credits using each model price table plus markup. BYOB meters input, output, cached, and thinking tokens separately, with a minimum charge flooring tiny calls.
Chain of Thought
Asking the model to reason step by step before giving the final answer, which raises accuracy on math, planning, and debugging. Keep the reasoning in thinking tokens and return only the conclusion to users.
Context Threshold
The input-size line where a provider switches a model to higher long-context pricing. BYOB model configs store the threshold with above-threshold rates, so sessions crossing it pay more per token without changing models.
Temperature
Sampling randomness for generation: low values make output focused and repeatable, high values make it varied and surprising. BYOB chat configs default near 0.5 while tool calls use 0.3 for steadier structured actions.
Structured Output
Forcing model replies into a machine-readable shape such as JSON with fixed fields. Required wherever AI output feeds code, databases, or UI components, since free text breaks parsers the moment formatting drifts.
JSON Schema Mode
A structured-output setting where the caller supplies a JSON Schema and the model must conform to it. Types, required fields, and enums are validated automatically, which removes most post-processing repair code.