Model routing chooses which language model to use for each query. By sending easy queries to cheaper models and hard queries to stronger ones, it can significantly reduce inference cost while maintaining high accuracy. However, most existing routers treat this as a fixed choice among a small set of models, which makes them hard to adapt to new models or changing budget constraints.

In this paper, we propose SCOPE (Scalable and Controllable Outcome Performance Estimator), a routing framework that goes beyond model selection by predicting their cost and performance. Trained with reinforcement learning, SCOPE makes reasoning-based predictions by retrieving how models behave on similar problems, rather than relying on fixed model names, enabling it to work with new, unseen models. Moreover, by explicitly predicting how accurate and how expensive a model will be, it turns routing into a dynamic decision problem, allowing users to easily control the trade-off between accuracy and cost.

Traditional routers memorize which model to use for each type of question. SCOPE takes a fundamentally different approach: instead of memorizing model assignments, it reasons about model behaviors by analyzing how models have performed on similar questions in the past. This enables SCOPE to:

• Work with any model, including those never seen during training
• Adapt to any budget constraint at inference time
• Provide controllable trade-offs between accuracy and cost

SCOPE operates in three stages: (1) Fingerprint Construction - retrieving similar questions from an anchor bank to construct a model fingerprint; (2) Performance Prediction - using chain-of-thought reasoning to predict correctness and token length; (3) Decision & Calibration - combining predictions with anchor-based calibration to select the optimal model.

Experiments show that SCOPE is more than just a cost-saving tool. It flexibly adapts to user needs: it can boost accuracy by up to 25.7% when performance is the priority, or cut costs by up to 95.1% when efficiency matters most.

We evaluate SCOPE under different trade-off coefficients α against baseline routers on both Test Set and OOD Set. SCOPE consistently outperforms existing routing methods including SVM Router, MLP Router, KNN Router, Graph Router, and xRouter across different settings.

SCOPE enables fine-grained budget control by allowing users to specify their cost constraints. The framework automatically adjusts the routing strategy to maximize accuracy within the given budget, providing a smooth trade-off curve between cost and performance.

Compared to test-time-scaling (TTS) approaches that try all models, SCOPE achieves significant token savings while maintaining comparable accuracy. The savings increase as the model pool size grows.

We conduct extensive ablation studies to validate our design choices. SCOPE outperforms baseline routing strategies including random selection and highest-cost-first approaches.