LLM Tracking Tools: 6 Options Side by Side (2026)

A
By the Ainotely Editorial Team.
Updated November 2026. 7 min read. Prices and features researched from each vendor's official pages and app store listings at time of writing.
LLM Tracking Tools
Quick picks
  1. 1Portkey logo
    PortkeyAI gateway and governance
    Visit
  2. 2Braintrust logo
    BraintrustEvals and experiment management
    Visit
  3. 3Arize AX logo
    Arize AXEnterprise observability and evals
    Visit
  4. 4Opik logo
    OpikOpen source tracing and evals
    Visit
  5. 5Langfuse logo
    LangfuseOpen source agent observability
    Visit
Short version:
In this guide Direct answerQuick comparisonThe 6 tools in detailHow we choseWhat to confirm before you sign upFrequently asked questionsSources

Direct answer

Portkey leads the list, best for AI gateway and governance. Look at Braintrust for Evals and experiment management, and at Arize AX for enterprise observability and evals. The remaining 3 options are covered below with prices and limits.

Quick comparison

ProductIdeal forPrice fromFree version
PortkeyAI gateway and governanceSee websiteYesVisit
BraintrustEvals and experiment management$249 a monthYesVisit
Arize AXEnterprise observability and evals$50 a monthYesVisit
OpikOpen source tracing and evals$19 per monthYesVisit
LangfuseOpen source agent observability$29 a monthYesVisit
TraceloopOpenTelemetry tracing and quality checksSee websiteYesVisit

The 6 tools in detail

This Ainotely guide covers 6 LLM tracking tools. Each profile lists a starting price, free plan details, strengths and drawbacks, so you can drop poor fits quickly.

Portkey logo
#1

Portkey

Ideal for: AI gateway and governance

Portkey website
portkey.ai homepage

Portkey targets AI gateway and governance. Governance features handle access control, PII redaction, budget limits and audit logs, which matter for enterprises running plenty of teams on shared models.

Where it falls short: Homepage now presents Portkey as PRISMA AIRS AI Gateway, so branding is in transition.

For a single app, a lighter tracer may be enough. Portkey shows no starting price here, so confirm the cost on the vendor site. Portkey includes a free tier.

What users get: governance and prompt management in one system.

The good and the bad
  • Caching and routing cut model spend
  • Gateway with observability, guardrails
  • Gateway and governance depth can exceed small team needs
Braintrust logo
#2

Braintrust

Ideal for: Evals and experiment management

Braintrust website
braintrust.dev homepage

Think of Braintrust for Evals and experiment management. Braintrust positions itself as the active observability platform for agents, and its strength is the loop from observation to improvement.

Watch-outs: Best value for teams already running structured evals.

Beyond that, the free Starter level includes small use, but Pro opens at $249 a month, which prices it above more straightforward tracing tools. More specifically, traces are searchable across millions of logs, evals run against versioned datasets with LLM, code or human scoring, and experiments compare prompts and models side by side. Braintrust starts at: $249 per month. You can start on Braintrust without paying.

The good and the bad
  • Evals and prompt management for entire teams
  • Most solid experiment and dataset workflow in this list
  • Pro pricing starts high for small teams
Arize AX logo
#3

Arize AX

Ideal for: Enterprise observability and evals

Arize AX website
arize.com homepage

Arize AX targets enterprise observability and evals. For organizations that require compliance, scale and a serious evaluation program around production agents, Arize AX is the most enterprise ready choice here.

Smaller teams may find the evaluation design work heavier than a straightforward tracing tool. More specifically, pricing starts with a Pro level around $50 a month, with enterprise terms negotiated beyond that. The starting price for Arize AX: $50 per month. Arize AX has a free plan.

Drawbacks to consider: Setup and evaluation design take time to get right.

The good and the bad
  • Observability plus evals on one product at enterprise scale
  • Open source Phoenix tracing for local development
  • Enterprise focus means higher cost at scale
Opik logo
#4

Opik

Ideal for: Open source tracing and evals

Opik is designed for open source tracing and evals. Opik, built by Comet, is an open source AI observability and evaluation product that logs every agent step, from user input to retrieval and tool calls, then scores outcomes with more than 30 metrics including relevance, hallucination, task completion and more.

Test suites define plain text assertions for what decent looks like, an AI assistant recommends code fixes for failures, and production monitoring alerts when traces fail criteria. Cloud pricing starts with a free level and a Pro plan around $19 a month, with self hosting available from the open source code. The starting price for Opik: $19 per month. You can start on Opik without paying.

Drawbacks to consider: Newer brand under Comet, so some buyers will check roadmap stability.

The good and the bad
  • Prompt management and monitoring in one place
  • True open source core with an ample free level
  • Advanced features sit on premium levels
Langfuse logo
#5

Langfuse

Ideal for: Open source agent observability

Langfuse is built with open source agent observability in mind. Langfuse has become the default open source choice for LLM observability. Its pitch is the full loop: trace production behavior, build datasets from real traffic, run experiments and evals, then ship prompt and model changes with evidence.

What to check first: Usage based pricing needs monitoring at high volume.

It supports OpenTelemetry instrumentation plus more than 100 integrations. So it works across languages, frameworks and model providers without locking a team into one stack. Beyond that, the trade off is operational. Worth adding: the free level handles 50,000 observations a month with no card required, which is enough for a prototype or a small production app. The starting price for Langfuse: $29 per month. You can start on Langfuse without paying. Strengths: Self hosting choice with full data control.

The good and the bad
  • Free tier with 50,000 observations a month
  • Open source tracing, evals
  • Self hosting brings operational work
Traceloop logo
#6

Traceloop

Ideal for: OpenTelemetry tracing and quality checks

Traceloop is designed for OpenTelemetry tracing and quality checks. Traceloop is a LLM reliability product built on OpenTelemetry, with OpenLLMetry as its open source instrumentation SDK.

Where it falls short: Pricing is not published on the homepage.

In daily use, pricing is not published, so teams should request it early. Check the vendor site for Traceloop pricing; none is listed in this comparison. Traceloop includes a free tier.

What Traceloop does well: Quality checks run automatically on real data.

The good and the bad
  • SOC 2 and HIPAA compliant with cloud
  • On prem and air gapped choices
  • Smaller ecosystem than Langfuse or LangSmith

How we chose

Selection started from tools commonly considered for LLM tracking tools, then narrowed to 6 that have clear pricing information or a clear quote process. Where a price is missing, the vendor does not list one publicly or ties it to a quote. Always confirm before budgeting. Order on the page follows fit for the typical buyer first, then price transparency and free access. Your own requirements should decide the final pick.

What to confirm before you sign up

Frequently asked questions

What is the cheapest option for LLM tracking tools in this list?

On published monthly entry prices, Opik is the lowest here at $19 per month. Entry price is only part of the cost, because seats, add-ons and billing terms can change the total.

How are the tools for LLM tracking tools ordered?

Portkey sits first and Traceloop last because the order follows typical fit for LLM tracking tools, then price clarity and free access. It is a judgement call, so read each profile for fit and limits.

Can I self host a LLM tracking tool?

Yes. Langfuse and Opik both offer open source self hosting, and Arize Phoenix offers open source tracing. From a buyer's view, self hosting provides full data control and can suit regulated environments. But it brings operational work for scaling, backups and upgrades.

Can I try these options before paying?

Often, yes. Portkey, Braintrust, Arize AX and others have a free plan. Use your own data and a real task during the test, and check what the free tier or trial limits before you rely on it.

Is Portkey the right pick for everyone comparing LLM tracking tools?

No single tool suits every team. Portkey comes first because it fits AI gateway and governance, while Braintrust suits Evals and experiment management and Arize AX suits enterprise observability and evals. Match the tool to your own size, budget and workflow before deciding.

What is LLM tracking?

LLM tracking, also called LLM observability, means recording and analyzing what happens when an application calls a language model: prompts, responses, tool calls, retrieval steps, latency, token use, cost and errors. It provides teams the visibility to debug failures, control spend and measure quality.

Sources

  1. https://portkey.ai
  2. https://braintrust.dev
  3. https://arize.com
  4. https://opik.com
  5. https://langfuse.com
  6. https://traceloop.com