Routing
model supports two ways of calling: send model: "auto" and JoyToken selects the final model and provider for the request (recommended), or send a concrete model ID from the catalog to pin the request to that model.
Common Requests
Smart Routing
Send model: "auto" and JoyToken’s smart routing engine takes over. It combines multi-dimensional request features, semantic rules, and LLM review to identify system settings, tool intent, context, and user needs, determine the task type and complexity, and apply the matching routing strategy.
For complex requests, the system automatically builds a task plan — breaking the request down, matching models and tiers to each task, and aggregating the results. Simple requests go straight to single-model selection.
Within policy constraints, the system schedules intelligently across a continuously expanding pool of online models and capabilities, weighing quality, cost, latency, and real-time success rate, and keeps service quality and continuity through automatic circuit breaking, traffic switching, and cross-tier fallback.
Pin a Specific Model
Pass a modelName from GET /api/v1/models as the request model. JoyToken then skips scoring and routing and sends the request straight to that model.
Tiers
tier narrows the candidate pool for an auto request. Omit it to let JoyToken decide.
Orchestration (Multi-task Planning)
For complex prompts, JoyToken may handle a single model: "auto" request in orchestration mode: it first builds a task plan, runs each task with a selected sub-model, then aggregates a final answer. You do not opt in explicitly; the gateway decides based on the request.
How to detect it
Task lifecycle
In orchestration mode each task is billed separately (one metadata/billing entry per task) and may use a different sub-model. Parse choices[].message.content as JSON before rendering, and sum billing.credits_used across all task metadata entries.
