Top LLM Observability and Evaluation Platforms in 2026: Langfuse, LangSmith, Braintrust, Arize, and More Compared

Introduction

In the rapidly evolving landscape of Large Language Models (LLMs), observability and evaluation have become critical for maintaining performance, debugging issues, and ensuring reliable deployments. This tutorial will guide you through setting up and using LangSmith, one of the leading LLM observability platforms, to monitor and evaluate your LLM applications. You'll learn how to trace LLM calls, evaluate outputs, and visualize performance metrics to ensure your applications are running smoothly.

By the end of this tutorial, you'll have a working LangSmith integration that allows you to track LLM interactions, measure response quality, and debug potential issues in your LLM-powered applications.

Prerequisites

• Basic understanding of Python and LLMs
• Python 3.8 or higher installed
• Access to a LangSmith account (sign up at https://smith.langchain.com/)
• LangChain installed in your environment

Step-by-Step Instructions

1. Install Required Dependencies

First, we need to install the necessary libraries. LangSmith integrates with LangChain, so we'll install both:

pip install langchain langsmith

Why: LangChain provides the core LLM integration tools, while LangSmith offers observability features like tracing and evaluation.

2. Set Up Your LangSmith API Key

You'll need to get your API key from LangSmith. Once you've signed up, go to your account settings and copy your API key. Then, set it as an environment variable:

import os
os.environ["LANGCHAIN_API_KEY"] = "your_api_key_here"

Why: The API key authenticates your application with LangSmith, allowing it to log and trace your LLM interactions.

3. Initialize LangSmith Tracing

Now, we'll initialize LangSmith tracing in your application. This allows LangSmith to track and log all LLM interactions:

from langchain_core.tracers import LangChainTracer
from langchain_core.callbacks import LangChainCallbackHandler

# Initialize the tracer
tracer = LangChainTracer()

# Set up a callback handler
handler = LangChainCallbackHandler()

Why: The tracer records all interactions with LLMs, including prompts, outputs, and timing, which is essential for debugging and performance analysis.

4. Create a Simple LLM Chain

Let's create a basic LLM chain to test our observability setup:

from langchain_openai import OpenAI
from langchain_core.prompts import PromptTemplate
from langchain_core.output_parsers import StrOutputParser

# Initialize the LLM
llm = OpenAI(model="gpt-3.5-turbo", temperature=0.7)

# Define a prompt template
prompt = PromptTemplate.from_template(
    "What is the capital of {country}?"
)

# Create the chain
chain = prompt | llm | StrOutputParser()

Why: This chain demonstrates a basic LLM interaction. By wrapping it with tracing, we can monitor its performance and outputs.

5. Run the Chain with Tracing

Execute the chain while ensuring it's being traced:

# Run the chain with tracing
result = chain.invoke({"country": "France"}, config={"callbacks": [handler]})
print(result)

Why: The callback handler ensures that the chain's execution is logged to LangSmith, making it visible in the LangSmith dashboard.

6. Evaluate Chain Outputs

Now, let's evaluate the outputs of our LLM chain. LangSmith allows us to define evaluation criteria:

from langsmith.evaluation import evaluate

# Define a simple evaluation function
def evaluate_output(run, example):
    # Example: check if the output contains the correct answer
    expected = "Paris"
    actual = run.outputs["text"]
    return {
        "score": 1 if expected.lower() in actual.lower() else 0,
        "comment": "Correct answer" if expected.lower() in actual.lower() else "Incorrect answer"
    }

# Run evaluation
evaluation_results = evaluate(
    chain,
    data=[{"country": "France"}],
    evaluator=evaluate_output
)
print(evaluation_results)

Why: Evaluating outputs helps you measure the quality of LLM responses and identify areas for improvement in your prompts or model selection.

7. View Results in LangSmith Dashboard

After running your chain and evaluations, navigate to the LangSmith dashboard at https://smith.langchain.com/. You'll see your traced runs and evaluation metrics. This dashboard provides:

• Tracing of all LLM interactions
• Performance metrics and latency data
• Evaluation results and scoring
• Debugging tools to inspect individual runs

Why: The dashboard provides a centralized view of all your LLM activities, making it easier to monitor, debug, and optimize your applications.

Summary

In this tutorial, you've learned how to integrate LangSmith into your LLM applications for observability and evaluation. You've set up tracing to monitor LLM interactions, created a simple chain, and evaluated its outputs. By using LangSmith, you now have the tools to track performance, debug issues, and continuously improve your LLM-powered applications.

LangSmith's capabilities extend beyond these basics, offering advanced features like custom evaluation metrics, automated testing, and integration with various LLM providers. As you continue to build more complex applications, these tools will become increasingly valuable for maintaining high-quality LLM deployments.