LangSmith CLI ↗

langchain guide intermediate testing workflows
no
Summary: Query and manage LangSmith projects, traces, runs, datasets, evaluators, experiments, and threads from the terminal

Original Documentation

Documentation Index#

Fetch the complete documentation index at: https://docs.langchain.com/llms.txt Use this file to discover all available pages before exploring further.

Query and manage LangSmith projects, traces, runs, datasets, evaluators, experiments, and threads from the terminal

The LangSmith CLI is a fast, agent-friendly command-line tool for working with your LangSmith data and workflows directly from the terminal. It’s designed for both humans and AI coding agents to list, filter, retrieve, and export data—with predictable JSON output by default and a pretty table mode for humans.

Built for agents and scripts: defaults to JSON, supports clean stdout/stderr separation, and offers --yes flags for non-interactive use.

Installation#

Use one of the follow methods for installation:

curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh
INSTALL_DIR=$HOME/.local/bin \
  curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh
# Download the latest binary for your platform
# https://github.com/langchain-ai/langsmith-cli/releases
go install github.com/langchain-ai/langsmith-cli/cmd/langsmith@latest

Authentication#

Set your API key as an environment variable:

export LANGSMITH_API_KEY="lsv2_..."

Optional defaults:

export LANGSMITH_ENDPOINT="https://api.smith.langchain.com"  # self-hosted/hybrid
export LANGSMITH_PROJECT="my-default-project"                 # default project for queries

Or pass them as flags when running commands:

langsmith --api-key lsv2_... trace list --project my-app

Quickstart#

The following commands cover the core resource types—projects, traces, runs, datasets, experiments, and threads:

# List tracing projects (sessions)
langsmith project list

# List recent traces in a project
langsmith trace list --project my-app --limit 5

# Get a specific trace with full detail
langsmith trace get <trace-id> --project my-app --full

# List LLM runs with token counts
langsmith run list --project my-app --run-type llm --include-metadata

# Datasets and experiments
langsmith dataset list
langsmith experiment list --dataset my-eval-set

# Conversation threads in a project
langsmith thread list --project my-chatbot

Output formats#

  • Default: JSON to stdout for easy piping and scripting
  • Pretty tables: --format pretty for human-readable tables and trees
  • Write to file: -o <path>
langsmith trace list --project my-app                  # JSON array to stdout
langsmith --format pretty trace list --project my-app  # tables/trees
langsmith trace list --project my-app -o traces.json   # write JSON to file

Commands overview#

The CLI groups functionality by resource. Each command supports filters like --limit, --last-n-minutes, and more.

project—list tracing projects#

langsmith project list                    # default limit: 20
langsmith project list --name-contains chatbot
langsmith --format pretty project list

trace—query and export traces#

langsmith trace list --project my-app --limit 50 --last-n-minutes 60
langsmith trace list --project my-app --error --include-metadata
langsmith trace get <trace-id> --project my-app --full
langsmith trace export ./traces --project my-app --limit 20 --full

run—query individual runs#

langsmith run list --project my-app --run-type llm
langsmith run list --project my-app --run-type tool --name search
langsmith run get <run-id> --full
langsmith run export llm_calls.jsonl --project my-app --run-type llm --full

thread—query conversation threads#

langsmith thread list --project my-chatbot --last-n-minutes 120
langsmith thread get <thread-id> --project my-chatbot --full

dataset—manage evaluation datasets#

langsmith dataset list --name-contains eval
langsmith dataset get my-dataset
langsmith dataset create --name my-eval-set --description "QA pairs for v2"
langsmith dataset export my-dataset ./data.json --limit 500

evaluator—manage evaluators#

langsmith evaluator list
langsmith evaluator upload evals.py --name accuracy --function check_accuracy --dataset my-eval-set
langsmith evaluator delete accuracy --yes

experiment—results and summaries#

langsmith experiment list --dataset my-eval-set
langsmith experiment get my-experiment-2024-01-15

Edit this page on GitHub or file an issue.

Connect these docs to Claude, VSCode, and more via MCP for real-time answers.

Link last verified June 7, 2026. View original ↗
Source: LangChain Docs
Link last verified: 2026-04-05