Deepeval on AI Knowledge Base

Command Line Interface

Mon, 01 Jan 0001 00:00:00 +0000

Conversation Simulator

Mon, 01 Jan 0001 00:00:00 +0000

Conversation Simulator Custom Templates

Mon, 01 Jan 0001 00:00:00 +0000

Conversation Simulator Lifecycle Hooks

Mon, 01 Jan 0001 00:00:00 +0000

Conversation Simulator Model Callback

Mon, 01 Jan 0001 00:00:00 +0000

Conversation Simulator Simulation Graph

Mon, 01 Jan 0001 00:00:00 +0000

Conversation Simulator Stopping Logic

Mon, 01 Jan 0001 00:00:00 +0000

Data Privacy

Mon, 01 Jan 0001 00:00:00 +0000

Development

Mon, 01 Jan 0001 00:00:00 +0000

Development

Mon, 01 Jan 0001 00:00:00 +0000

Development

Mon, 01 Jan 0001 00:00:00 +0000

Environment Variables

Mon, 01 Jan 0001 00:00:00 +0000

Evals In Prod

Mon, 01 Jan 0001 00:00:00 +0000

Evals In Prod

Mon, 01 Jan 0001 00:00:00 +0000

Evals In Prod

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation Arena Test Cases

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation Component Level Llm Evals

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation Datasets

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation End To End Llm Evals

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation End To End Multi Turn

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation End To End Single Turn

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation Flags And Configs

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation Introduction

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation Llm Tracing

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation Mcp

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation Multiturn Test Cases

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation Prompts

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation Test Cases

Mon, 01 Jan 0001 00:00:00 +0000

Evaluation Unit Testing In Ci Cd

Mon, 01 Jan 0001 00:00:00 +0000

Faq

Mon, 01 Jan 0001 00:00:00 +0000

Getting Started

Mon, 01 Jan 0001 00:00:00 +0000

This five-minute quickstart is the fastest way into DeepEval: install it, write a test case, pick a metric, and run deepeval test run, which feels like pytest for LLM outputs. The critical thing to set up first is an OPENAI_API_KEY, because nearly all DeepEval metrics are LLM-as-a-judge evaluators that call a model under the hood. If a run appears stuck, suspect rate limits or quota rather than a framework bug, the most common early gotcha. DeepEval covers similar ground to RAGAS but with a pytest-style assertion workflow; read the metrics introduction next.

Getting Started Agents

Mon, 01 Jan 0001 00:00:00 +0000

Getting Started Chatbots

Mon, 01 Jan 0001 00:00:00 +0000

Getting Started Llm Arena

Mon, 01 Jan 0001 00:00:00 +0000

Getting Started Mcp

Mon, 01 Jan 0001 00:00:00 +0000

Getting Started Rag

Mon, 01 Jan 0001 00:00:00 +0000

Golden Synthesizer

Mon, 01 Jan 0001 00:00:00 +0000

Guides Ai Agent Evaluation

Mon, 01 Jan 0001 00:00:00 +0000

Guides Ai Agent Evaluation Metrics

Mon, 01 Jan 0001 00:00:00 +0000

Guides Answer Correctness Metric

Mon, 01 Jan 0001 00:00:00 +0000

Guides Building Custom Metrics

Mon, 01 Jan 0001 00:00:00 +0000

Guides Llm As A Judge

Mon, 01 Jan 0001 00:00:00 +0000

Guides Llm Observability

Mon, 01 Jan 0001 00:00:00 +0000

Guides Multi Turn Evaluation

Mon, 01 Jan 0001 00:00:00 +0000

Guides Multi Turn Evaluation Metrics

Mon, 01 Jan 0001 00:00:00 +0000

Guides Multi Turn Simulation

Mon, 01 Jan 0001 00:00:00 +0000

Guides Optimizing Hyperparameters

Mon, 01 Jan 0001 00:00:00 +0000

Guides Rag Evaluation

Mon, 01 Jan 0001 00:00:00 +0000

Guides Rag Triad

Mon, 01 Jan 0001 00:00:00 +0000

Guides Red Teaming

Mon, 01 Jan 0001 00:00:00 +0000

Guides Regression Testing In Cicd

Mon, 01 Jan 0001 00:00:00 +0000

Guides Tracing Ai Agents

Mon, 01 Jan 0001 00:00:00 +0000

Guides Tracing Multi Turn

Mon, 01 Jan 0001 00:00:00 +0000

Guides Tracing Rag

Mon, 01 Jan 0001 00:00:00 +0000

Guides Using Custom Embedding Models

Mon, 01 Jan 0001 00:00:00 +0000

Guides Using Custom Llms

Mon, 01 Jan 0001 00:00:00 +0000

Guides Using Synthesizer

Mon, 01 Jan 0001 00:00:00 +0000

Improvement

Mon, 01 Jan 0001 00:00:00 +0000

Improvement

Mon, 01 Jan 0001 00:00:00 +0000

Improvement

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

Mon, 01 Jan 0001 00:00:00 +0000

Introduction Comparisons

Mon, 01 Jan 0001 00:00:00 +0000

Introduction Design Philosophy

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Answer Relevancy

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Arena G Eval

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Argument Correctness

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Bias

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Contextual Precision

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Contextual Recall

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Contextual Relevancy

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Conversation Completeness

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Conversational Dag

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Conversational G Eval

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Custom

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Dag

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Exact Match

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Faithfulness

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Goal Accuracy

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Hallucination

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Introduction

Mon, 01 Jan 0001 00:00:00 +0000

This page introduces DeepEval’s fifty-plus metrics, each scored from 0 to 1 with reasoning, and it matters because choosing the right metrics is the whole game in LLM evaluation. The key discipline the docs push is restraint: use no more than about five metrics, roughly two or three generic plus one or two custom to your use case, so you prioritize what truly matters instead of drowning in numbers. Because the metrics are LLM-as-a-judge, expect real cost and some run-to-run variance. This parallels RAGAS’s metric suite; read getting-started first if you have not run an evaluation yet.

Metrics Json Correctness

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Knowledge Retention

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Llm Evals

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Mcp Task Completion

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Mcp Use

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Misuse

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Multi Turn Mcp Use

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Non Advice

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Pattern Match

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Pii Leakage

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Plan Adherence

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Plan Quality

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Prompt Alignment

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Ragas

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Role Adherence

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Role Violation

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Step Efficiency

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Summarization

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Task Completion

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Tool Correctness

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Tool Use

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Topic Adherence

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Toxicity

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Turn Contextual Precision

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Turn Contextual Recall

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Turn Contextual Relevancy

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Turn Faithfulness

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Turn Relevancy

Mon, 01 Jan 0001 00:00:00 +0000

Miscellaneous

Mon, 01 Jan 0001 00:00:00 +0000

Multimodal Metrics Image Coherence

Mon, 01 Jan 0001 00:00:00 +0000

Multimodal Metrics Image Editing

Mon, 01 Jan 0001 00:00:00 +0000

Multimodal Metrics Image Helpfulness

Mon, 01 Jan 0001 00:00:00 +0000

Multimodal Metrics Image Reference

Mon, 01 Jan 0001 00:00:00 +0000

Multimodal Metrics Text To Image

Mon, 01 Jan 0001 00:00:00 +0000

Prompt Optimization Copro

Mon, 01 Jan 0001 00:00:00 +0000

Prompt Optimization Gepa

Mon, 01 Jan 0001 00:00:00 +0000

Prompt Optimization Introduction

Mon, 01 Jan 0001 00:00:00 +0000

Prompt Optimization Miprov2

Mon, 01 Jan 0001 00:00:00 +0000

Prompt Optimization Simba

Mon, 01 Jan 0001 00:00:00 +0000

Synthesizer Generate From Contexts

Mon, 01 Jan 0001 00:00:00 +0000

Synthesizer Generate From Docs

Mon, 01 Jan 0001 00:00:00 +0000

Synthesizer Generate From Goldens

Mon, 01 Jan 0001 00:00:00 +0000

Synthesizer Generate From Scratch

Mon, 01 Jan 0001 00:00:00 +0000

Synthetic Data Generation Introduction

Mon, 01 Jan 0001 00:00:00 +0000

Troubleshooting

Mon, 01 Jan 0001 00:00:00 +0000

Tutorial Introduction

Mon, 01 Jan 0001 00:00:00 +0000

Tutorial Setup

Mon, 01 Jan 0001 00:00:00 +0000

Vibe Coder Quickstart

Mon, 01 Jan 0001 00:00:00 +0000

Vibe Coding

Mon, 01 Jan 0001 00:00:00 +0000