How to Build an Agent Evaluation System With DeepEval and LangChain

Attach DeepEval's CallbackHandler to your LangChain agent so every run becomes a trace, wrap test inputs as Goldens in an EvaluationDataset, and loop over them with evals_iterator and metrics such as TaskCompletionMetric. Add ToolCorrectnessMetric where expected tools are known, then run the same tests in CI with deepeval test run.

What you are building

An agent evaluation system has four parts: a dataset of realistic tasks, a way to capture what the agent did on each, metrics that score those runs, and a runner that repeats it on every change. DeepEval supplies the capture, metrics, and runner; LangChain supplies the agent. The general method, independent of tooling, is covered in how to evaluate AI agents. This guide is the concrete build.

APIs below follow DeepEval's current LangChain integration docs. The library moves quickly, so check imports against the version you install.

Step 1: install and build a small agent

Install the packages:

pip install -U deepeval langchain langchain-openai

DeepEval's documented example uses LangChain's create_agent with a single tool:

from langchain.agents import create_agent
from langchain_core.tools import tool

@tool
def multiply(a: int, b: int) -> int:
    """Multiply two integers."""
    return a * b

agent = create_agent(
    model="openai:gpt-4o-mini",
    tools=[multiply],
    system_prompt="Be concise.",
)

Replace this with your real agent and tools. Evaluate the agent you ship, not a demo.

Step 2: write goldens, not just prompts

A golden is a test input plus whatever you know about the right outcome. Start with 20 to 50 real tasks pulled from logs or support tickets, covering the common path, edge cases, and requests the agent should refuse.

from deepeval.dataset import EvaluationDataset, Golden

dataset = EvaluationDataset(goldens=[
    Golden(input="What is 8 multiplied by 6?"),
    Golden(input="Multiply 12 by 0 and explain the result."),
])

Write down, for each golden, what success means in a sentence. If you cannot, the task is too vague to grade and the metric will be noise.

Step 3: trace runs and score trajectories

Pass DeepEval's CallbackHandler in the LangChain config. Each run becomes a trace, and the metrics passed to evals_iterator score it end to end:

from deepeval.integrations.langchain import CallbackHandler
from deepeval.metrics import TaskCompletionMetric

for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    agent.invoke(
        {"messages": [{"role": "user", "content": golden.input}]},
        config={"callbacks": [CallbackHandler()]},
    )

DeepEval groups its agent metrics in layers. Reasoning: PlanQualityMetric and PlanAdherenceMetric. Execution: TaskCompletionMetric and StepEfficiencyMetric. All four read the full trace, which is why tracing is required. Task completion catches wrong outcomes; step efficiency catches agents that get there through needless loops.

Step 4: check tool use at the component level

Tool metrics score individual decisions. ToolCorrectnessMetric compares the tools the agent called with the tools you expected, with options to consider ordering or require an exact match. ArgumentCorrectnessMetric is referenceless and judges whether arguments fit the request. DeepEval's tracing helpers in deepeval.tracing, such as next_agent_span and next_llm_span, stage metrics on a specific span inside a run.

For a quick check outside tracing, build a test case directly:

from deepeval.test_case import LLMTestCase, ToolCall
from deepeval.metrics import ToolCorrectnessMetric

case = LLMTestCase(
    input="What is 8 multiplied by 6?",
    actual_output="48",
    tools_called=[ToolCall(name="multiply")],
    expected_tools=[ToolCall(name="multiply")],
)

Only set expected tools where one path is clearly correct. Penalising a valid alternative route teaches you nothing.

Step 5: gate changes in CI

Turn the loop into tests with assert_test(golden=golden, metrics=[TaskCompletionMetric()]) and run them with deepeval test run test_agent.py. Run the suite on every prompt, model, tool, or retrieval change, and track scores over time rather than reacting to single runs; agent outputs vary between identical runs, so repeat flaky cases several times before blaming a change.

Do not trust the judge by default

Task completion, plan, and argument metrics are scored by an LLM judge. That makes them scalable and also fallible: judges favour fluent answers, miss subtle factual errors, and can agree with each other for the wrong reasons, a problem covered in when LLM judges agree. Hand-label a sample of runs, compare with the metric, and set thresholds from that agreement rather than from defaults. Prefer deterministic checks, such as exact tool names or a database state, wherever the task allows.

If the agent retrieves documents, evaluate retrieval separately (see evaluating RAG retrieval quality in production): a wrong answer from a missing passage is a retrieval bug, not a reasoning bug. Retrieval also drives cost, since every unnecessary passage is tokens on every run, and an eval suite runs the agent many times. RDK puts private retrieval in front of the model, cutting token spend 80 to 90 percent on repeated, reference-heavy work, which also makes large eval runs cheaper.

Frequently asked questions

How do I connect DeepEval to a LangChain agent?
Import CallbackHandler from deepeval.integrations.langchain and pass it in the config of each invoke call, as config with a callbacks list. DeepEval then records the agent run, model calls, tool calls, and retriever calls as spans in a trace, which its trajectory metrics such as TaskCompletionMetric use to score the run.
Which DeepEval metrics should an agent eval start with?
Start with TaskCompletionMetric, because outcome is what users feel. Add ToolCorrectnessMetric where one tool path is clearly right, and StepEfficiencyMetric if runs loop or wander. Plan quality and adherence metrics help once the agent plans explicitly. Fewer, well-calibrated metrics beat a long list nobody reads.
Do DeepEval agent metrics need an OpenAI key?
Most agent metrics use an LLM as the judge, so they need access to some judge model. DeepEval's examples use OpenAI models, and the library supports configuring other judge models. Whichever you use, the judge's cost and its accuracy against human labels both belong in your evaluation budget.
How is this different from evaluating RAG retrieval?
Agent evals score whether the whole run reached the right outcome efficiently and used tools correctly. Retrieval evals score whether the right passages were found and ranked, independent of what the model then did. Keep them separate so a failure points to the right layer: search, reasoning, or tool use.