Portkey leads the list, best for AI gateway and governance. Look at Braintrust for Evals and experiment management, and at Arize AX for enterprise observability and evals. The remaining 3 options are covered below with prices and limits.
| Product | Ideal for | Price from | Free version | |
|---|---|---|---|---|
| AI gateway and governance | See website | Yes | Visit | |
| Evals and experiment management | $249 a month | Yes | Visit | |
| Enterprise observability and evals | $50 a month | Yes | Visit | |
| Open source tracing and evals | $19 per month | Yes | Visit | |
| Open source agent observability | $29 a month | Yes | Visit | |
| OpenTelemetry tracing and quality checks | See website | Yes | Visit |
This Ainotely guide covers 6 LLM tracking tools. Each profile lists a starting price, free plan details, strengths and drawbacks, so you can drop poor fits quickly.

Ideal for: AI gateway and governance

Portkey targets AI gateway and governance. Governance features handle access control, PII redaction, budget limits and audit logs, which matter for enterprises running plenty of teams on shared models.
Where it falls short: Homepage now presents Portkey as PRISMA AIRS AI Gateway, so branding is in transition.
For a single app, a lighter tracer may be enough. Portkey shows no starting price here, so confirm the cost on the vendor site. Portkey includes a free tier.
What users get: governance and prompt management in one system.

Ideal for: Evals and experiment management

Think of Braintrust for Evals and experiment management. Braintrust positions itself as the active observability platform for agents, and its strength is the loop from observation to improvement.
Watch-outs: Best value for teams already running structured evals.
Beyond that, the free Starter level includes small use, but Pro opens at $249 a month, which prices it above more straightforward tracing tools. More specifically, traces are searchable across millions of logs, evals run against versioned datasets with LLM, code or human scoring, and experiments compare prompts and models side by side. Braintrust starts at: $249 per month. You can start on Braintrust without paying.

Ideal for: Enterprise observability and evals

Arize AX targets enterprise observability and evals. For organizations that require compliance, scale and a serious evaluation program around production agents, Arize AX is the most enterprise ready choice here.
Smaller teams may find the evaluation design work heavier than a straightforward tracing tool. More specifically, pricing starts with a Pro level around $50 a month, with enterprise terms negotiated beyond that. The starting price for Arize AX: $50 per month. Arize AX has a free plan.
Drawbacks to consider: Setup and evaluation design take time to get right.

Ideal for: Open source tracing and evals
Opik is designed for open source tracing and evals. Opik, built by Comet, is an open source AI observability and evaluation product that logs every agent step, from user input to retrieval and tool calls, then scores outcomes with more than 30 metrics including relevance, hallucination, task completion and more.
Test suites define plain text assertions for what decent looks like, an AI assistant recommends code fixes for failures, and production monitoring alerts when traces fail criteria. Cloud pricing starts with a free level and a Pro plan around $19 a month, with self hosting available from the open source code. The starting price for Opik: $19 per month. You can start on Opik without paying.
Drawbacks to consider: Newer brand under Comet, so some buyers will check roadmap stability.

Ideal for: Open source agent observability
Langfuse is built with open source agent observability in mind. Langfuse has become the default open source choice for LLM observability. Its pitch is the full loop: trace production behavior, build datasets from real traffic, run experiments and evals, then ship prompt and model changes with evidence.
What to check first: Usage based pricing needs monitoring at high volume.
It supports OpenTelemetry instrumentation plus more than 100 integrations. So it works across languages, frameworks and model providers without locking a team into one stack. Beyond that, the trade off is operational. Worth adding: the free level handles 50,000 observations a month with no card required, which is enough for a prototype or a small production app. The starting price for Langfuse: $29 per month. You can start on Langfuse without paying. Strengths: Self hosting choice with full data control.

Ideal for: OpenTelemetry tracing and quality checks
Traceloop is designed for OpenTelemetry tracing and quality checks. Traceloop is a LLM reliability product built on OpenTelemetry, with OpenLLMetry as its open source instrumentation SDK.
Where it falls short: Pricing is not published on the homepage.
In daily use, pricing is not published, so teams should request it early. Check the vendor site for Traceloop pricing; none is listed in this comparison. Traceloop includes a free tier.
What Traceloop does well: Quality checks run automatically on real data.
Selection started from tools commonly considered for LLM tracking tools, then narrowed to 6 that have clear pricing information or a clear quote process. Where a price is missing, the vendor does not list one publicly or ties it to a quote. Always confirm before budgeting. Order on the page follows fit for the typical buyer first, then price transparency and free access. Your own requirements should decide the final pick.
On published monthly entry prices, Opik is the lowest here at $19 per month. Entry price is only part of the cost, because seats, add-ons and billing terms can change the total.
Portkey sits first and Traceloop last because the order follows typical fit for LLM tracking tools, then price clarity and free access. It is a judgement call, so read each profile for fit and limits.
Yes. Langfuse and Opik both offer open source self hosting, and Arize Phoenix offers open source tracing. From a buyer's view, self hosting provides full data control and can suit regulated environments. But it brings operational work for scaling, backups and upgrades.
Often, yes. Portkey, Braintrust, Arize AX and others have a free plan. Use your own data and a real task during the test, and check what the free tier or trial limits before you rely on it.
No single tool suits every team. Portkey comes first because it fits AI gateway and governance, while Braintrust suits Evals and experiment management and Arize AX suits enterprise observability and evals. Match the tool to your own size, budget and workflow before deciding.
LLM tracking, also called LLM observability, means recording and analyzing what happens when an application calls a language model: prompts, responses, tool calls, retrieval steps, latency, token use, cost and errors. It provides teams the visibility to debug failures, control spend and measure quality.