Our first pick is Langfuse, best for open source tracing and evals. Portkey is better for multi-model routing and guardrails, and LangSmith is better for LangChain and LangGraph teams. The remaining 3 options are covered below with prices and limits.
| Product | Ideal for | Price from | Free version | |
|---|---|---|---|---|
| Open source tracing and evals | $29 a month | Yes | Visit | |
| Multi-model routing and guardrails | $49 per month | Yes | Visit | |
| LangChain and LangGraph teams | $39 per seat/month | Yes | Visit | |
| Open source evals and monitoring | $19 per month | Yes | Visit | |
| Agent observability and evaluation | See website | Yes | Visit | |
| Production agent evaluation | $50 per month | Yes | Visit |
If you are comparing LLM monitoring tools, this page covers 6 tools in one place: what each is best for, what it costs to start, whether it has a free plan, and the main limit to know about.

Ideal for: Open source tracing and evals

For open source tracing and evals, Langfuse is a natural candidate. Units include traces, observations and scores, so agent runs with numerous tool calls consume units quicker than straightforward chat calls.
For most small and mid-sized teams, the hosted service is the more straightforward start. Langfuse pricing: $29 per month. Langfuse has a free plan.
Where it is weaker: Unit based pricing takes care to forecast.

Ideal for: Multi-model routing and guardrails

Portkey makes the most sense for multi-model routing and guardrails. Portkey is part gateway and part observability tool, which sets it apart from pure tracing products. The Developer plan is free with a restricted log allowance and short retention, whereas the premium plan begins at 49 dollars a month with a much bigger log allowance and longer retention.
Teams that only need tracing may find Portkey broader than necessary. But teams running multiple models in production frequently value the control layer more than the dashboard alone. Semantic caching can cut repeated token spend, and virtual keys with budgets help stop runaway usage. On cost, Portkey starts like this: $49 per month. You can start on Portkey without paying.
Main advantages: guardrails and prompt management. What to check first: Free log allowance is small for production use.

Ideal for: LangChain and LangGraph teams
LangSmith serves LangChain and LangGraph teams. A trace displays each model call, tool call and retrieval step in order, which makes it far simpler to find why an agent looped, picked the wrong tool or returned a weak answer. Beyond that, teams outside the LangChain world can still use LangSmith through its SDKs.
But they may get less value from the framework specific parts. In daily use, the evaluation side lets teams score outputs against datasets, collect human feedback and compare prompt versions before a change reaches users. Also, large teams should model seat count and trace volume together, and set retention and sampling rules early. Price to start with LangSmith: $39 per seat/month. A free plan is available with LangSmith.
Downsides: Per-seat pricing brings up for large teams.

Ideal for: Open source evals and monitoring
Pick Opik if this describes you: open source evals and monitoring. Opik, built by Comet, merges tracing, evaluation and production monitoring with an open source core that teams can run themselves. For teams that want open source freedom with a low cost hosted start, Opik is a solid shortlist pick.
Worth adding: production monitoring layers on scoring, alerts and cost views. So a regression found in live traffic can become a test case for the next release. Beyond that, self-hosting hands full data control, but it also means running and updating the backend. Entry price for Opik: $19 per month. You can start on Opik without paying.
Where it falls short: Self-hosting needs operational effort.

Ideal for: Agent observability and evaluation
Choose HoneyHive for agent observability and evaluation. That lack of a published self-serve price is a drawback for teams that want to budget without a sales call.
HoneyHive does not show a starting price in this comparison, so check the vendor site for current pricing. Free access exists with HoneyHive.
Points against: Smaller teams may need a sales conversation to size cost.

Ideal for: Production agent evaluation
Arize AI is a good fit for production agent evaluation. Arize AX is aimed at teams that require to watch agents in production and keep improving them, not just log calls. Since pricing is based on span and data volume instead of seats, product managers, support staff and engineers can review traces without changing the bill.
What to check first: Span volume for agents can grow rapidly.
That matters once quality work spreads beyond engineering. Also, the system also contains Phoenix, a free self-hosted choice for tracing and evals, which is handy for a local pilot. More specifically, multi-step agents produce many spans per conversation, so a pilot should measure real span counts before sizing a plan. On cost, Arize AI starts like this: $50 per month. A free plan is available with Arize AI.
Selection started from tools commonly considered for LLM monitoring tools, then narrowed to 6 that have clear pricing information or a clear quote process. Where a vendor does not publish a price, the entry says quote. Where a price depends on seats or billing term, the entry says so. We ordered the tools by general fit, then by how clearly pricing is shown, then by free access. A buyer with unusual needs may reorder them.
No single tool suits every team. Langfuse comes first because it fits open source tracing and evals, while Portkey suits multi-model routing and guardrails and LangSmith suits LangChain and LangGraph teams. Match the tool to your own size, budget and workflow before deciding.
Not all of them. HoneyHive publishes no starting price, so expect a quote request for it. The rest, Langfuse included, show a starting price, though seats and billing terms still change the final cost.
LLM monitoring is the practice of tracing model calls, measuring latency, cost and errors, and scoring output quality in production. It helps teams find failures, compare prompt and model changes, and stop small quality drops from reaching plenty of users.
On this list, 6 of 6 tools include a free plan, among them Langfuse, Portkey, LangSmith and Opik. Caps on seats, usage or features are common, so test the free plan against your real workload.
Langfuse sits first and Arize AI last because the order follows typical fit for LLM monitoring tools, then price clarity and free access. It is a judgement call, so read each profile for fit and limits.