Key takeaways
- LangSmith works best if you build on LangChain or LangGraph. Teams with mixed stacks, strict self-hosting needs, or growing seat and trace costs usually start looking elsewhere.
- Langfuse (MIT, from $29/month) is the safest default for most teams. Arize Phoenix is the pick if you want OpenTelemetry-native tracing with no lock-in.
- Braintrust and Confident AI treat evaluation as the main event, but both comparison articles we read are written by the vendors themselves, so weigh their claims about LangSmith accordingly.
- Pricing models differ more than headline prices do. LangSmith bills per seat plus per trace, Braintrust bills by processed GB and scores, Langfuse by units. Estimate your own volume before you commit.
- If your real problem is AI search visibility for your brand rather than debugging your own LLM app, that is a different category of tool (covered near the end).
Why people leave LangSmith
LangSmith is good at what it was built for. Set one environment variable and a LangChain app starts sending traces. The eval tooling plugs straight into the same ecosystem. If your whole stack is LangChain or LangGraph, you may never need to leave.
The friction shows up later. These are the reasons that come up most:
- Framework gravity. Custom orchestration, direct provider SDK calls, or a mix of frameworks all need more manual instrumentation.
- Seat and usage costs. The Plus plan is $39 per seat per month with 10k base traces, then pay-as-you-go. Usage is now metered in LangChain Standard Units at $1.00 per LSU, with extra traces listed at 0.005 LSU each. Fleet, Sandboxes, an LLM Gateway and tuned evaluators are metered too. It is hard to forecast a bill with that many meters.
- Self-hosting is gated. Self-hosted and hybrid deployment sit in the Enterprise tier, priced by quote. For teams with strict data rules, that is a conversation with sales before you can even try it.
- Vendor lock-in worries. Even though the SDKs are MIT, the platform is proprietary.
To be fair, LangSmith has a free Developer tier (5k base traces, one seat) and VC-backed startups can get up to $10,000 in credits. It is not a bad product. It is a product with a specific shape.
What to compare before you switch
Feature lists look alike, so focus on the things that actually differ:
- Tracing model. Is it OpenTelemetry-native, or does it use a proprietary SDK? OTel means you can move your data later.
- Evaluation depth. LLM-as-a-judge, datasets, human annotation queues, CI gating, and whether you can score individual spans in an agent trace.
- Deployment. Self-host, hybrid, or SaaS only. Data region options matter for EU and HIPAA work.
- Pricing unit. Seats, traces, spans, GB, or scores. This decides what happens to your bill when traffic grows ten times.
- Retention. Braintrust's Starter keeps 14 days, Arize's free tier 15 days, Langfuse Hobby 30 days. Short retention hurts if you debug slow-burning regressions.
- Who uses it. If product managers and QA need to run evals without code, check the no-code workflow honestly.
The alternatives at a glance
| Tool | Open source | Best for | Entry pricing |
|---|---|---|---|
| Langfuse | Yes (MIT) | Most teams, self-hosting, prompt management | Free Hobby, Core $29/mo |
| Arize Phoenix / AX | Phoenix is open source | OpenTelemetry-native tracing and evals | AX Free, AX Pro $50/mo |
| Braintrust | No | Eval-driven release workflows, CI gates | Free Starter, Pro $249/mo |
| Confident AI | DeepEval is open source | Evaluation-first teams, no-code evals | Check vendor |
| Opik (Comet) | Yes (Apache 2.0) | Self-hosted, no per-seat pricing | Free self-host, Pro $39/mo cloud |
| Helicone | Yes | Fast gateway-style setup and caching | Check vendor |
| MLflow | Yes (Apache 2.0) | Teams already using it for ML tracking | Not disclosed |
| OpenObserve | Yes (AGPL-3.0) | LLM spans next to infrastructure telemetry | Self-host free |
| Galileo | No | Real-time guardrails and production evaluators | Check vendor |
| Agenta | Yes | Prompt playground plus evals plus observability | Check vendor |
I did not verify pricing for Helicone, Opik's enterprise tier, Confident AI, Galileo, Datadog, W&B Weave or MLflow, so confirm those on each vendor's page.
Langfuse: the default open-source choice
If someone asked me for one name, this is it. Langfuse is MIT licensed, works with most frameworks, and covers prompt management, a playground, online and offline evals, LLM-as-judge, datasets, human annotation, custom dashboards and OpenTelemetry. LiteLLM proxy logging is supported too.
Pricing is in units:
- Hobby: free, 50k units per month, 30 days of data, 2 users
- Core: $29/month, 100k units then $8 per extra 100k, 90 days, unlimited users
- Pro: $199/month, 3 years of data access, SOC2, ISO27001, HIPAA-ready region
- Teams add-on: $300/month for enterprise SSO and fine-grained RBAC
- Enterprise: $2,499/month with audit logs, SCIM and SLAs
You can choose US, EU or JP data regions, or self-host. The pricing page also points to a Langfuse v4 changelog dated 2026-08-17 that claims real-time ingestion and up to 165x faster performance. I have not benchmarked that, so treat it as a vendor claim.
The trade-off: Braintrust's own comparison says self-hosting Langfuse means running your own infrastructure, and another comparison notes it takes real engineering time to set up properly. Both are fair. Free software still has an ops bill.
There is also a direct Langfuse vs. LangSmith comparison on Langfuse's site, which is worth reading with the usual vendor caveat.
Arize Phoenix and AX: OpenTelemetry first
Phoenix is the open-source, local-first option for tracing, evaluation, experimentation and prompt iteration. The Firecrawl roundup names it the top evaluation pick because it is OpenTelemetry-native, so you are not tied to a proprietary format.
The commercial path is Arize AX:
- AX Free: 25k spans per month, 1 GB ingestion, 15-day retention, unlimited users and evals
- AX Pro: $50/month, 50k spans, 10 GB, 30-day retention
- AX Enterprise: custom, SaaS or self-hosted, SSO, audit logs, HIPAA
Paid tiers list trace evals, session evals, agent-as-judge, prompt learning optimization and an assistant called Alyx. Data regions are US, EU or CA. The sensible pattern is to start on Phoenix locally and move to AX only if you need the hosted features.
Braintrust: evals that gate releases
Braintrust's pitch is a loop: production traces convert into eval cases, pull requests run evals through GitHub Actions, and score thresholds can block a merge. It also has an assistant called Loop that lets non-engineers analyze logs and build scorers in plain language.
A caution here. The claim that LangSmith results "inform engineers but do not automatically block bad releases" comes from Braintrust's own article. Check it against your actual LangSmith setup before you believe it.
Pricing is where people get surprised. It is based on processed data and scores, not traces:
- Starter: free, $10 of model credits, 1 GB processed (then $4/GB), 10k scores, 14-day retention, unlimited users
- Pro: $249/month, 5 GB (then $3/GB), 50k scores, 30-day retention, up to 180 days at $0.50/GB/month
- Enterprise: custom, S3 export, SAML SSO, BAA/HIPAA option, on-prem or hosted
Unlimited users is nice if many people need to look at results. Large trace payloads, like long RAG contexts, can inflate GB fast. Qualifying startups can get 6 to 12 months free.
Confident AI and DeepEval
Confident AI is built around evaluation, powered by the open-source DeepEval framework, which ships 50+ metrics. Its main argument is that observability alone does not tell you whether outputs are good. It supports span-level evaluation, so you can test retrieval and generation separately in a RAG trace, and multi-turn simulation for agents.
It also pitches no-code workflows so product managers and QA can run evaluations without engineering help. The company cites a case study where an enterprise voice AI firm, Humach, shipped 300% faster after switching. That is a vendor case study, so take it as a data point rather than a benchmark. I did not verify Confident AI's pricing.
Opik, MLflow and other open-source options
Opik from Comet is Apache 2.0. Self-hosting has no feature restrictions and unlimited spans, which makes the no-per-seat model attractive for cost-conscious teams. The managed cloud has a free tier and Pro at $39/month with unlimited team members.
MLflow (Apache 2.0) makes sense if your team already tracks classical ML experiments there. It covers tracing, evals, a gateway and agent serving in one place. The catch from one comparison is a steep learning curve for the full feature set.
OpenObserve (AGPL-3.0) is a different idea. It is a general observability backend that stores LLM spans next to your logs, metrics and traces, so "was that slow response the model or the vector database" becomes one query. Evals exist but are an enterprise feature and narrower than a dedicated eval platform. Good if you already run infrastructure observability and do not want LLM telemetry living in a separate silo.
Agenta combines a prompt playground, prompt management, evals and observability, open source and self-hostable, though it lacks native CI gating according to Braintrust's comparison.
Helicone is the fast-setup, gateway-style option with caching.
Galileo focuses on production evaluators and guardrails, with 20+ built-in metrics covering RAG, agents and safety. Fiddler AI suits larger enterprises that want ML and LLM monitoring together with compliance governance. TruLens is a good fit for RAG quality metrics.
If you want to browse more options, the software directory at surferstack.com is a good place to start. Prompt management products like PromptLayer and Humanloop also sit nearby in the catalog.
Already paying for Datadog or New Relic?
Then start there. The Firecrawl roundup's advice is blunt: use what you have. Both vendors now offer LLM monitoring, and having traces in the same place as the rest of your stack saves you from a new login. The limit is evaluation depth, which usually is not as strong as dedicated tools.
Which tool for which situation
| Situation | Start with |
|---|---|
| Mostly LangChain/LangGraph, small team | Stay on LangSmith Developer or Plus |
| Mixed frameworks, want open source | Langfuse |
| Need OTel-native tracing, no lock-in | Arize Phoenix |
| Want evals to block bad merges | Braintrust |
| RAG or agent quality is the main worry | Confident AI with DeepEval |
| Strict data control, no per-seat pricing | Opik self-hosted or OpenObserve |
| Already run ML experiments | MLflow |
| Need a fast gateway with caching | Helicone |
Cost traps to check first
- Seats. LangSmith Plus is per seat. If PMs and support need access, count them.
- Payload size. Braintrust bills on GB, so large prompts and retrieved contexts cost more.
- Retention. 14 to 30 days on free tiers is enough for debugging, not for trend analysis.
- Eval runs. LangSmith meters some evaluators per run, such as 0.015 LSU for Perceived Error evaluations. Run a realistic month in a trial.
- Self-hosting labor. Free licenses still need someone to run them.
- Bulk export. LangSmith's feature matrix seems to list bulk data export on a higher tier, but our scrape cut off, so verify before relying on it.
I could not find reliable material on migrating off LangSmith, so budget time for it. The practical advice is to adopt OpenTelemetry instrumentation first, then dual-write traces to the new tool for a couple of weeks and compare results before cutting over.
A different problem: seeing how AI describes your brand
None of the tools above answer a question marketers often ask: when someone asks ChatGPT or Perplexity about my category, does the answer mention us? That is AI search visibility, not LLM app observability. Tools like Promptwatch track brand mentions and citations across AI search engines, which is a separate job from tracing your own application.
For the research side, Promptwatch's data on which sources ChatGPT cites by content type is a useful starting point, and the GEO software directory at bestgeosoftware.com lists more options.
Bottom line
If you live in LangChain, LangSmith is still fine. If you do not, try Langfuse first and Phoenix second, both of which have free paths and low lock-in. Pick Braintrust or Confident AI when the evaluation workflow matters more than the tracing. Then run a two-week pilot with real traffic, because the pricing unit, not the headline price, is what decides your bill.