Connecting an LLM is enough to make an application generate text. It is not enough to make an AI agent complete a real-world task.
Ask a model to summarize a document already in its context, and the model layer is sufficient. Ask it to compare today's competitor prices, enrich a company record, check a social account, or choose the best API for an unfamiliar job, and the missing half becomes obvious: the agent needs a reliable way to reach external systems.
A production agent therefore makes two separate decisions:
- Which model should handle this step?
- Which tool or API should provide the data or action?
AIHubMix addresses the first decision with unified model access and request-time model routing. Monid addresses the second with runtime tool discovery, schema inspection, and pay-per-call execution. The products sit on different layers of the same architecture.
Disclosure: This article was created as part of a content collaboration with Monid. AIHubMix is the model platform discussed below; Monid is the tool platform. Each can be used independently.
The two routing problems inside an AI agent
An agent run is rarely one homogeneous model call. A research agent may classify a request, search for current information, extract structured facts, compare results, and write a final answer. Those steps require different capabilities.
The same is true outside the model. A company-research task may need a search API today, a company-enrichment endpoint tomorrow, and a browser automation tool next week. If every model and tool is hard-coded at build time, each new task becomes an integration project.
The two layers are parallel:
| Model layer | Tool layer | |
|---|---|---|
| Core decision | Which model should answer? | Which API should be called? |
| Selection time | Per request | Per task, at runtime |
| Input | Prompt, modality, quality and latency needs | Goal, required data or action, schema and price |
| Output | A model completion | External data or an executed action |
| Example | AIHubMix | Monid |
This separation matters. A better model cannot create access to live data, and a larger tool catalog cannot reason over the data it returns. The agent needs both capabilities, with a clear contract between them.
Monid presents the complementary tool-side view in Why an AI Agent Needs Two Integrations. From the model side, the architectural lesson is the same: keep model selection and tool selection independent, then optimize each layer for its own job.
Why one fixed model becomes expensive in an agent loop
Using one model everywhere looks simple. In practice, it forces every step to accept the same tradeoff among capability, latency, and price.
Consider a market-research agent:
- Intent classification is short and mechanical.
- Tool selection needs reliable instruction following.
- Extracting fields from returned JSON is mostly transformation.
- The final report may require stronger reasoning and better writing.
Sending all four steps to the most capable model wastes money on routine work. Sending all four to the cheapest model can reduce the quality of the only output the user reads. As an agent loop grows, that compromise is repeated across every call.
AIHubMix provides an OpenAI-compatible endpoint across a broad model catalog. An existing OpenAI SDK integration can point to AIHubMix by changing the API key and base_url:
from openai import OpenAI
client = OpenAI(
api_key="<AIHUBMIX_API_KEY>",
base_url="https://aihubmix.com/v1",
)
response = client.chat.completions.create(
model="auto:balanced",
messages=[
{"role": "user", "content": "Classify this request and propose the next step."}
],
)
Setting model to auto moves model selection into the request path. The router analyzes the task and resolves it to a suitable model. A policy suffix makes the optimization goal explicit:
| Router value | Priority | Typical agent step |
|---|---|---|
auto |
Cost first | Batch work and routine transformations |
auto:balanced |
Capability, cost, and latency | General-purpose agent work |
auto:quality_first |
Capability first | Complex reasoning and final deliverables |
auto:latency_critical |
Speed first | Interactive loops and lightweight planning |
Routing does not add a separate fee. The request is billed at the list price of the model that actually handled it. The resolved model and routing details are exposed in the response, including the X-Aihubmix-Router-Resolved-Model header, so the decision remains observable rather than becoming a black box.
The full behavior, supported endpoints, policies, and current limitations are documented in the AIHubMix LLM Router guide.
Model routing does not give an agent live data
After model routing is configured, the agent can choose a better brain for each step. It still cannot know what changed after the model's training data, access a private business system, or perform an action in another application unless a tool provides that capability.
This is where many agent projects accumulate brittle code. A team connects one search API, then one scraping API, then one enrichment API. Each integration introduces another account, credential, request format, error model, and billing relationship. Tool descriptions are often copied into a system prompt and slowly become stale.
The failure mode is dangerous because it can look successful. A model may produce a plausible call against an outdated schema, receive an incomplete response, and continue as though the task succeeded. Runtime schema inspection is safer than asking the model to remember an API contract from training or from an old prompt.
The tool layer should therefore answer three questions before execution:
- What tool can satisfy this goal?
- What schema and price apply right now?
- What result did the call actually return?
Tool discovery is the second routing layer
Monid turns tool access into a discover-inspect-run workflow. Instead of requiring the developer to predict every API an agent may need, the agent can search a catalog in natural language, inspect a candidate's contract, and execute the selected endpoint.
The basic flow looks like this:
# 1. Find tools that match the goal
monid discover -q "find current product pricing from a public web page"
# 2. Read the selected endpoint's schema and pricing
monid inspect -p PROVIDER_SLUG -e ENDPOINT_PATH
# 3. Execute only after the agent has checked the contract
monid run -p PROVIDER_SLUG -e ENDPOINT_PATH \
--query '{"url":"https://example.com/product"}'
Discovery and inspection let the agent compare options before it spends anything. Execution is billed according to the selected endpoint's pricing model. The developer keeps one integration while the agent gains access to tools across multiple providers.
For agent runtimes that can read setup instructions, Monid also publishes a machine-readable skill:
Set up https://monid.ai/SKILL.md
The Monid workflow documentation explains the catalog, inspection, and execution stages in more detail.
How the two layers work together
The model gateway and tool layer should remain separate components with a small, explicit handoff:
- The agent receives a user goal.
- AIHubMix routes a planning call to an appropriate model.
- The plan identifies missing information or a required external action.
- Monid discovers candidate tools and exposes their schemas and prices.
- The agent selects and runs a tool within its permissions and budget.
- The tool returns facts or an action result.
- AIHubMix routes the synthesis call according to the required quality, cost, or latency.
- The agent returns an answer grounded in the tool result.
In simplified Python, the model-side calls can remain unchanged while the tool result is inserted as context:
from openai import OpenAI
client = OpenAI(
api_key="<AIHUBMIX_API_KEY>",
base_url="https://aihubmix.com/v1",
)
# A fast model is sufficient for a lightweight planning step.
plan = client.chat.completions.create(
model="auto:latency_critical",
messages=[
{
"role": "user",
"content": "Plan how to compare the current prices of these products.",
}
],
)
# Your agent uses Monid to discover, inspect, and run an appropriate tool.
# Replace these placeholders with the structured result returned by that call.
tool_result = {
"source": "<source-url>",
"data": "<structured-tool-result>",
}
report = client.chat.completions.create(
model="auto:quality_first",
messages=[
{
"role": "system",
"content": (
"Write a concise comparison. Use only the supplied tool result, "
"preserve source URLs, and state when a value is missing."
),
},
{"role": "user", "content": str(tool_result)},
],
)
The important detail is not the number of lines. It is that neither choice needs to be permanently embedded in application logic. The model can change as the prompt changes, and the tool can change as the task changes.
Cost controls belong at both layers
Model and tool costs use different units, so they should be measured separately.
At the model layer, the resolved model determines token pricing. AIHubMix makes that decision traceable and lets developers choose a routing policy or restrict the models an API key is allowed to use. Routine steps can favor cost or latency, while user-facing outputs can favor quality.
At the tool layer, an endpoint may charge per call or per result. Monid exposes pricing during inspection, before execution. An agent can reject an endpoint that exceeds its budget, prefer a verified option, or ask for approval before an unusually expensive operation.
Useful production controls include:
- A model allowlist or price ceiling for each API key.
- A maximum tool-call budget per task.
- Provider or endpoint allowlists for regulated data.
- Logs that connect the model routing decision to the tool call and final answer.
- Explicit confirmation before irreversible or sensitive actions.
- Output validation so tool data is treated as untrusted input, not as instructions.
This split also makes cost debugging easier. If a run becomes expensive, token logs show whether the agent reasoned too much, while tool logs show whether it fetched too much. The fixes are different, and the architecture should preserve that distinction.
Reliability requires fresh contracts and visible decisions
Dynamic selection should not mean unpredictable behavior.
On the model side, AIHubMix reports the resolved model and routing policy for each request. Session stickiness can preserve model consistency and prompt-cache benefits across multi-turn work, while fallback behavior can move away from an unhealthy model when necessary.
On the tool side, the agent inspects the current endpoint schema before making a paid call. Discovery results include the information needed to compare candidates, and the result of the actual call becomes the only external evidence passed into the final completion.
Together, these controls create a useful audit trail:
user goal
-> routing decision and resolved model
-> discovered tool candidates
-> inspected schema and price
-> selected endpoint and result
-> final model decision and grounded response
That trail is more valuable than merely having access to many models or many APIs. It explains why the agent made each choice and what evidence supported its answer.
When you do not need both layers
Not every workflow benefits from runtime choice.
If an application sends one stable prompt to one benchmarked model, specifying that model directly is simpler and more deterministic than routing. If a scheduled pipeline always calls one known API, integrating that API directly may be clearer than adding a discovery layer.
The two-layer architecture earns its place when variety is part of the workload:
- Prompts differ enough that the best model changes by step.
- Agents make multiple model calls and need to control cumulative cost or latency.
- Required tools cannot be fully predicted at build time.
- External data must be current and its source must be visible.
- The team wants to add capabilities without adding a new vendor integration for each one.
Use the model layer, tool layer, or both according to the decisions your application actually needs to make.
Build the agent around decisions, not dependencies
A production AI agent is not defined by how many models or APIs it can reach. It is defined by whether it can choose the right capability for the current step, operate within a budget, and explain what happened.
AIHubMix gives the agent a unified model endpoint and request-time routing across cost, quality, and latency priorities. Monid gives it runtime tool discovery, current schemas, visible pricing, and execution across external providers.
One layer decides how the agent thinks. The other decides how it finds out and acts. Keeping those decisions separate produces an agent that is easier to extend, observe, and control.
Start with the AIHubMix quick start, then connect the tool side through Monid.
FAQ
What is model routing for AI agents?
Model routing selects a model for each request based on factors such as task type, capability, cost, and latency. With AIHubMix, setting model to auto or a policy such as auto:quality_first enables request-time selection while preserving an OpenAI-compatible API.
Why does an AI agent need tools if the LLM already has knowledge?
An LLM generates answers from the context it receives and what it learned during training. It cannot reliably know current prices, query a private database, or perform an external action without a connected tool. Tools provide live data and execution; the model plans and interprets the result.
Is a model gateway the same as an MCP server or tool platform?
No. A model gateway routes inference requests to models. MCP servers and tool platforms expose external capabilities to an agent. They solve complementary integration problems and can be used together.
Can AIHubMix and Monid be used independently?
Yes. AIHubMix can route model calls for applications with no dynamic tool requirements. Monid can provide tool discovery to agents using another model provider or gateway. Using both is useful when an agent needs runtime choice at both layers.
How can I keep automatic routing auditable?
Log the resolved model, routing policy, tool candidate, inspected price, selected endpoint, and returned result for each task. AIHubMix exposes model routing details in the response, while Monid separates discovery and inspection from execution so the tool decision can be recorded before the paid call.
