← All posts
·17 min read

Claude Code with Langfuse: LLM Traces That Make Sense

Claude CodeLangfuseLLM ObservabilityTracing
Claude Code with Langfuse: LLM traces that make sense

Why Langfuse without CLAUDE.md produces traces you cannot filter

Langfuse is the open-source LLM observability standard for AI applications in 2026. It captures every prompt, response, token count, latency, and cost, then lets you slice the data by user, session, model, prompt version, or custom score. The platform is excellent. The integration is straightforward. The problem is that Langfuse only works as well as the structure you give it. Sessions, users, tags, and scores are not automatic. If Claude Code instruments your application without those constraints, the result is a flat list of generation events that captures everything but lets you filter nothing. You have terabytes of data and no way to ask "which prompt version did user X see when they hit the rate limit?"

The most common Claude defaults that produce useless Langfuse data: every LLM call lands as a top-level trace with no parent, instead of nesting as a span or generation under a request-scoped trace, the userId and sessionId fields are omitted so cross-conversation analytics are impossible, prompts are sent as inline strings instead of fetched from Langfuse Prompt Management so version comparisons cannot be run, the LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY are read inconsistently across server and client code, and scores are never attached, so there is no quality signal to filter on.

This guide covers the CLAUDE.md configuration that locks Claude Code into Langfuse's correct model: every request opens a trace, every LLM call is a generation under that trace, sessions bind across multi-turn conversations, prompts are pinned to versioned references, and scores attach when user feedback or evaluator results arrive. If you are building on the Anthropic SDK directly, Claude Code with Claude API covers the message-pattern instrumentation. For trace correlation with infrastructure telemetry, Claude Code with Honeycomb covers the OpenTelemetry bridge that Langfuse can complement.

The Langfuse CLAUDE.md template

The CLAUDE.md at your project root is read at the start of every Claude Code session. For a Langfuse-instrumented LLM application it needs to declare: the SDK version, the environment variable names for the keys, the trace hierarchy, the session and user binding rules, the prompt management policy, and the hard rules that block the mistakes Claude makes most often.

# Langfuse rules

## Stack
- langfuse ^3.x (TypeScript/Node) OR langfuse ^2.x (Python)
- langfuse-langchain for LangChain auto-instrumentation (optional)
- LANGFUSE_PUBLIC_KEY in .env (pk-lf-...)
- LANGFUSE_SECRET_KEY in .env (sk-lf-...)
- LANGFUSE_BASEURL in .env (cloud.langfuse.com or self-hosted URL)

## Project structure
- src/lib/langfuse.ts          , Langfuse client singleton
- src/lib/llm/                  , LLM call wrappers that emit generations
- src/lib/scoring/              , Score helpers (user feedback, evaluator outputs)
- src/lib/prompts/              , Prompt name constants (NEVER inline prompt text)

## Trace hierarchy (MANDATORY)
Every request follows this structure:
- trace          , per-request, opened in route handler middleware
  - span         , for internal steps (retrieval, parsing, validation)
    - generation , per LLM call (model, prompt, completion, tokens, cost)
  - score        , attached to trace, span, OR generation depending on what is scored

## Required fields per object
trace: { id, name, userId, sessionId, metadata, tags }
generation: { name, model, modelParameters, input, output, usage, prompt }
score: { traceId, name, value, comment? }

## Hard rules
- NEVER call langfuse.generation() without an enclosing trace (parent: trace.id)
- NEVER omit userId on traces in production (cross-conversation analytics break)
- NEVER omit sessionId on multi-turn conversations
- NEVER inline prompt strings, ALWAYS fetch via langfuse.getPrompt(name, version)
- NEVER instantiate new Langfuse() inline, use the singleton at src/lib/langfuse.ts
- NEVER expose LANGFUSE_SECRET_KEY to the client bundle (server-only)
- ALWAYS call langfuse.flushAsync() before serverless function exit

Three rules here prevent the majority of useless-data outcomes Claude generates without them.

The trace hierarchy rule is the most impactful for query-ability. Without it, Claude produces code where every LLM call is its own top-level event. In Langfuse's UI this looks correct: the traces list shows hundreds of entries, each with model, tokens, and latency. But when you try to filter "show me all generations from one user request that resulted in a refund", there is no parent trace to group them under. The hierarchy lets you collapse all the LLM calls, retrieval steps, and validation logic of a single user request into one inspectable trace.

The session and user binding rule prevents the "we have data but cannot answer questions" problem. Langfuse's strength is cross-session analytics. "Show me the average completion length for premium users over the last week." "Which model version had the highest user thumbs-down rate." Both queries require userId and sessionId on every trace. Claude omits them by default because they are optional in the SDK type signatures. CLAUDE.md needs to declare them mandatory.

The prompt management rule prevents the inability to A/B test or roll back. When Claude inlines prompts as string literals, every prompt change requires a code change, a deploy, and a wait. Langfuse Prompt Management lets you edit prompts in the UI, version them, and reference them by name and version in code. The version pin lets you A/B test (50% of users get version 4, 50% get version 5) and roll back (revert to version 4 without a deploy). Inline strings break all of this.

Install and key setup

Install the Langfuse SDK:

npm i langfuse

Add the keys to your environment file. For Next.js, only the public key can ship to the client bundle, the secret key is server-only:

# .env.local
LANGFUSE_PUBLIC_KEY=pk-lf-your-public-key-here
LANGFUSE_SECRET_KEY=sk-lf-your-secret-key-here
LANGFUSE_BASEURL=https://cloud.langfuse.com
# OR for self-hosted:
# LANGFUSE_BASEURL=https://langfuse.your-domain.com

Create the singleton client that every instrumented call imports from:

// src/lib/langfuse.ts
import { Langfuse } from 'langfuse';

if (!process.env.LANGFUSE_PUBLIC_KEY || !process.env.LANGFUSE_SECRET_KEY) {
  throw new Error('Langfuse keys are not defined');
}

export const langfuse = new Langfuse({
  publicKey: process.env.LANGFUSE_PUBLIC_KEY,
  secretKey: process.env.LANGFUSE_SECRET_KEY,
  baseUrl: process.env.LANGFUSE_BASEURL ?? 'https://cloud.langfuse.com',
});

// Flush on process exit to avoid losing in-flight events
process.on('beforeExit', async () => {
  await langfuse.flushAsync();
});

The startup check surfaces missing keys immediately when the server boots rather than at the moment of the first instrumented call. The beforeExit handler is critical for serverless deployments where the function exits as soon as the response is returned and the in-memory event buffer can be lost.

Add the singleton pattern to CLAUDE.md so Claude never instantiates a new client inline:

## Client singleton (ENFORCE)
- The only Langfuse client in the project lives at src/lib/langfuse.ts
- Every file that emits Langfuse events does: import { langfuse } from '@/lib/langfuse'
- Claude MUST NOT write: const langfuse = new Langfuse({...}) inside route handlers or LLM wrappers
- Singleton handles process.exit flushing automatically

The trace, span, generation hierarchy

A request that triggers an LLM call with retrieval should produce one trace with two children: a span for the retrieval step and a generation for the LLM call. The full structure:

// src/app/api/chat/route.ts
import { NextRequest, NextResponse } from 'next/server';
import { langfuse } from '@/lib/langfuse';
import { openai } from '@/lib/openai';
import { getRelevantDocs } from '@/lib/retrieval';

export async function POST(req: NextRequest) {
  const { message, userId, sessionId } = await req.json();

  const trace = langfuse.trace({
    name: 'chat-completion',
    userId,
    sessionId,
    input: { message },
    metadata: { route: '/api/chat', requestTime: new Date().toISOString() },
    tags: ['production', 'chat-app'],
  });

  // Step 1: retrieval (span)
  const retrievalSpan = trace.span({
    name: 'vector-retrieval',
    input: { query: message },
  });

  const docs = await getRelevantDocs(message);

  retrievalSpan.end({
    output: { docCount: docs.length, topScore: docs[0]?.score },
  });

  // Step 2: LLM call (generation)
  const prompt = await langfuse.getPrompt('chat-system-prompt', undefined, {
    type: 'chat',
  });

  const systemMessage = prompt.compile({ docs: docs.map(d => d.content).join('\n\n') });

  const generation = trace.generation({
    name: 'gpt-4o-chat',
    model: 'gpt-4o',
    modelParameters: { temperature: 0.2, maxTokens: 1024 },
    input: [
      { role: 'system', content: systemMessage },
      { role: 'user', content: message },
    ],
    prompt,
  });

  const response = await openai.chat.completions.create({
    model: 'gpt-4o',
    temperature: 0.2,
    max_tokens: 1024,
    messages: [
      { role: 'system', content: systemMessage },
      { role: 'user', content: message },
    ],
  });

  const assistantMessage = response.choices[0].message.content;

  generation.end({
    output: assistantMessage,
    usage: {
      input: response.usage?.prompt_tokens,
      output: response.usage?.completion_tokens,
      total: response.usage?.total_tokens,
    },
  });

  trace.update({ output: { assistantMessage } });

  await langfuse.flushAsync();

  return NextResponse.json({ assistantMessage, traceId: trace.id });
}

Three things to notice:

  • The trace, span, and generation are created from langfuse.trace(), trace.span(), and trace.generation(). Each child object is created from its parent. Claude often calls langfuse.span() and langfuse.generation() directly, which creates orphan events with no parent trace.
  • The span and generation are explicitly ended with .end({ output, usage }). The end call records the output and the elapsed time. Without it, the event is missing its output field and the duration is recorded as the time the parent trace ends rather than the actual operation time.
  • The trace is updated with the final output via trace.update({ output }). This records the top-level output that appears in the Langfuse traces list. Without it, the trace shows an empty output field.

Add a trace hierarchy section to CLAUDE.md:

## Trace hierarchy

- Open a trace at the start of every request: trace = langfuse.trace({...})
- Spans are created from the trace: span = trace.span({...})
- Generations are created from the trace OR a span: gen = trace.generation({...})
- ALWAYS end spans and generations: .end({ output, usage })
- ALWAYS update the trace with its final output: trace.update({ output })
- ALWAYS pass userId and sessionId to trace()
- ALWAYS flushAsync() before request handler returns in serverless contexts

The traceId returned in the response payload is useful for the frontend. It lets the frontend attach scores (user thumbs up/down) directly to the trace later without needing the server to wait for user feedback.

Sessions and users

A session in Langfuse groups multiple traces under one identifier, typically a conversation. A user in Langfuse groups all sessions and traces under one identifier, typically the authenticated user ID.

For a multi-turn chat application, the frontend passes the same sessionId on every request in a conversation. The server passes it through to langfuse.trace({ sessionId }). The Langfuse UI then groups all messages in that conversation under one collapsible session view, with timestamps, model usage, and total cost.

// Frontend, on conversation start
const sessionId = crypto.randomUUID();

// Every subsequent message in this conversation uses the same sessionId
async function sendMessage(message: string) {
  const response = await fetch('/api/chat', {
    method: 'POST',
    body: JSON.stringify({ message, userId: currentUser.id, sessionId }),
  });
  return response.json();
}

// Backend, on every request
const trace = langfuse.trace({
  name: 'chat-completion',
  userId,                    // groups all sessions for this user
  sessionId,                 // groups all traces in this conversation
  input: { message },
});

The Langfuse UI then offers:

  • Sessions view: every conversation as a collapsible thread with all messages, total token usage, total cost, and elapsed time.
  • Users view: every user with their total sessions, traces, cost, and any attached scores.
  • Cross-filter: "show me all sessions where the user gave a thumbs-down on at least one message" or "show me all users whose average session cost is over $0.50".

None of this works without userId and sessionId on the trace.

Add a sessions and users section to CLAUDE.md:

## Sessions and users

- Frontend assigns sessionId on conversation start (crypto.randomUUID() or similar)
- Frontend passes sessionId on every request in that conversation
- Backend passes userId AND sessionId to langfuse.trace() on every call
- ALWAYS pass authenticated user ID, not anonymous session ID, when user is logged in
- For anonymous users: use a stable browser ID (localStorage UUID), NEVER omit userId entirely
- sessionId can be reused across days for the same conversation thread
- Different conversations from the same user MUST have different sessionIds

Prompt management

Langfuse Prompt Management lets you store prompts in the Langfuse UI, version them, and reference them by name in code. The version pin makes A/B testing trivial (label two versions, route traffic by user bucket) and rollback instant (change the label in the UI, no deploy).

A versioned prompt fetched at request time:

// src/lib/prompts/chat-system.ts
import { langfuse } from '@/lib/langfuse';

export async function getChatSystemPrompt(options: { docs: string }) {
  const prompt = await langfuse.getPrompt('chat-system-prompt', undefined, {
    type: 'chat',
    cacheTtlSeconds: 60,
  });

  const compiledMessages = prompt.compile({ docs: options.docs });

  return { prompt, compiledMessages };
}

The arguments to langfuse.getPrompt():

  • First argument: the prompt name (a string constant, never inline). The name is the canonical identifier in Langfuse.
  • Second argument: the version. undefined fetches the latest production-tagged version. A specific number (e.g. 3) pins to that version. A label (e.g. 'staging') fetches whichever version is tagged with that label.
  • Third argument: options. type: 'chat' for chat-format prompts (returns an array of { role, content } messages), type: 'text' for single-string prompts. cacheTtlSeconds: 60 caches the fetched prompt for 60 seconds to avoid hammering the Langfuse API on every request.

The compiled prompt is then passed to the generation:

const { prompt, compiledMessages } = await getChatSystemPrompt({ docs: contextDocs });

const generation = trace.generation({
  name: 'gpt-4o-chat',
  model: 'gpt-4o',
  input: compiledMessages,
  prompt,  // attaches the prompt reference (name + version)
});

The prompt reference attached to the generation lets Langfuse correlate completions with the specific prompt version that produced them. The traces list can then be filtered "show me all generations using chat-system-prompt v4 with negative user feedback" or "compare cost per generation across chat-system-prompt v3 and v4".

Add a prompt management section to CLAUDE.md:

## Prompt management

- NEVER inline prompt strings in code, ALWAYS use langfuse.getPrompt(name, version)
- Prompt names live in src/lib/prompts/ as exported constants
- Fetch with cacheTtlSeconds: 60 to avoid per-request API calls
- Compile templates with prompt.compile({ variable: value })
- Attach the prompt reference to the generation: trace.generation({ prompt, ... })
- Use undefined for version to fetch latest production-tagged
- Use a number to pin to a specific version (deterministic, no UI override)
- Use a string label ('staging', 'production') to fetch the labelled version

Scoring

Scores attach quality signals to traces, spans, or generations. They are the primary mechanism for filtering and ranking traces by user feedback, evaluator output, or any other quality metric.

Scoring from a user feedback endpoint:

// src/app/api/feedback/route.ts
import { NextRequest, NextResponse } from 'next/server';
import { langfuse } from '@/lib/langfuse';

export async function POST(req: NextRequest) {
  const { traceId, value, comment } = await req.json();

  langfuse.score({
    traceId,
    name: 'user-rating',
    value,                  // 0 = thumbs down, 1 = thumbs up (or numeric 1-5)
    comment,                // optional free-text feedback
    dataType: 'NUMERIC',    // NUMERIC, CATEGORICAL, or BOOLEAN
  });

  await langfuse.flushAsync();

  return NextResponse.json({ ok: true });
}

Scoring from an evaluator that runs after the response:

// src/lib/scoring/relevance-evaluator.ts
import { langfuse } from '@/lib/langfuse';
import { openai } from '@/lib/openai';

export async function scoreRelevance(traceId: string, question: string, answer: string) {
  const evaluation = await openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [
      {
        role: 'system',
        content: 'Score how well the answer addresses the question. Reply with only a number from 0 to 1.',
      },
      { role: 'user', content: `Question: ${question}\n\nAnswer: ${answer}` },
    ],
    temperature: 0,
  });

  const score = parseFloat(evaluation.choices[0].message.content ?? '0');

  langfuse.score({
    traceId,
    name: 'relevance',
    value: score,
    dataType: 'NUMERIC',
    comment: `Auto-scored by gpt-4o-mini`,
  });
}

Score schemas should be defined consistently across the application. The Langfuse UI does not enforce a schema (you can write name: 'user-rating' on one trace and name: 'rating' on another), but downstream filters and dashboards break when names drift. Lock the score names in a constants file:

// src/lib/scoring/score-names.ts
export const SCORE_NAMES = {
  USER_RATING: 'user-rating',
  RELEVANCE: 'relevance',
  FACTUAL_ACCURACY: 'factual-accuracy',
  LATENCY_BUCKET: 'latency-bucket',
  HALLUCINATION: 'hallucination',
} as const;

export type ScoreName = (typeof SCORE_NAMES)[keyof typeof SCORE_NAMES];

Add a scoring section to CLAUDE.md:

## Scoring

- All score names live in src/lib/scoring/score-names.ts as exported constants
- NEVER use inline string names, ALWAYS reference SCORE_NAMES.X
- dataType: NUMERIC for 0-1 floats, CATEGORICAL for enum-like, BOOLEAN for binary
- User feedback: bind to traceId returned in the API response
- Evaluator output: bind to traceId, run async after response
- ALWAYS flushAsync() before exiting the scoring endpoint
- Score names should be kebab-case and consistent across the application

OpenAI SDK integration via wrapper

Langfuse ships an OpenAI SDK wrapper that auto-instruments every openai.chat.completions.create() call as a generation under the current trace. This removes the need to manually call trace.generation() for each LLM call.

// src/lib/openai.ts
import OpenAI from 'openai';
import { observeOpenAI } from 'langfuse';

const baseOpenAI = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

export const openai = observeOpenAI(baseOpenAI);

Now every openai.chat.completions.create() call automatically:

  • Creates a generation under the current trace (if one exists)
  • Records the model, parameters, input messages, and completion
  • Captures the token usage from the OpenAI response
  • Records the latency

To pass trace context to the auto-instrumented call, use the langfuse parameter on the create call:

const trace = langfuse.trace({ name: 'chat', userId, sessionId });

const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [...],
  // @ts-expect-error - langfuse wrapper accepts extra fields
  langfuseTrace: trace,
  langfuseGenerationName: 'gpt-4o-completion',
});

The trade-off with the wrapper is reduced control. Manual trace.generation() calls let you split a single OpenAI call into multiple Langfuse generations (one for system message construction, one for the API call, one for response parsing) and attach custom metadata. The wrapper produces one generation per OpenAI call, with standard fields only.

Add an OpenAI wrapper section to CLAUDE.md:

## OpenAI SDK integration

- Use observeOpenAI() wrapper at src/lib/openai.ts for default auto-instrumentation
- The wrapper creates a generation per chat.completions.create() call automatically
- Pass trace context via langfuseTrace and langfuseGenerationName fields
- For complex flows (chain of operations on one OpenAI response), use manual trace.generation()
- Wrapper records model, params, input, output, usage, latency automatically
- Wrapper does NOT record prompt references, attach those manually if needed

Sample rate and PII handling

Production LLM applications generate large volumes of trace data. Two configurations control how much of it lands in Langfuse: the sample rate (record only N% of traces) and the input/output redaction (strip PII before recording).

Sample rate via the client constructor:

export const langfuse = new Langfuse({
  publicKey: process.env.LANGFUSE_PUBLIC_KEY,
  secretKey: process.env.LANGFUSE_SECRET_KEY,
  baseUrl: process.env.LANGFUSE_BASEURL,
  sampleRate: process.env.NODE_ENV === 'production' ? 0.1 : 1.0,
});

sampleRate: 0.1 records 10% of traces in production, 100% in development. The decision is made at trace open time, so all spans and generations under a sampled trace are recorded together (no partial traces).

PII redaction via the SDK option:

import { Langfuse } from 'langfuse';

export const langfuse = new Langfuse({
  publicKey: process.env.LANGFUSE_PUBLIC_KEY,
  secretKey: process.env.LANGFUSE_SECRET_KEY,
  baseUrl: process.env.LANGFUSE_BASEURL,
  mask: (input) => {
    // Redact email addresses
    const text = typeof input === 'string' ? input : JSON.stringify(input);
    return text.replace(/[\w.+-]+@[\w-]+\.[\w.-]+/g, '[email]');
  },
});

The mask function is called on every input and output before it is sent to Langfuse. The function should be deterministic, fast, and never throw. The recommended pattern is to define an explicit list of regex patterns for the PII categories your application handles (email, phone, credit card, SSN) and chain them.

Add a sample rate and PII section to CLAUDE.md:

## Sample rate and PII

- Production sampleRate: 0.1 (10%) for high-volume applications
- Development sampleRate: 1.0 (100%) for full debugging visibility
- Sample decision is per-trace, all children of a sampled trace are recorded
- mask: function in client constructor for PII redaction
- ALWAYS test mask function output before deploying (regex bugs silently leak data)
- Common patterns to redact: email, phone, credit card, SSN, customer-specific identifiers
- NEVER log raw user PII to Langfuse without explicit business justification

Common Claude Code mistakes with Langfuse

Six patterns Claude generates incorrectly without CLAUDE.md constraints, with the correct replacement for each.

1. Orphan events

Claude generates: langfuse.generation({...}) directly on the client without a parent trace.

Correct pattern: const trace = langfuse.trace({...}); const generation = trace.generation({...});

2. Missing userId and sessionId

Claude generates: langfuse.trace({ name: 'chat' }) with no user or session binding.

Correct pattern: langfuse.trace({ name: 'chat', userId, sessionId }).

3. Inline prompt strings

Claude generates: const systemPrompt = 'You are a helpful assistant...' hardcoded in the route handler.

Correct pattern: const prompt = await langfuse.getPrompt('chat-system'); const compiled = prompt.compile({...});

4. Missing .end() on spans and generations

Claude generates: const gen = trace.generation({...}) followed by the LLM call, then no .end().

Correct pattern: gen.end({ output: response.content, usage: response.usage }).

5. No flushAsync before serverless exit

Claude generates: a route handler that calls Langfuse and returns immediately, losing in-flight events.

Correct pattern: await langfuse.flushAsync(); return NextResponse.json({...});

6. Inconsistent score names

Claude generates: langfuse.score({ name: 'rating' }) in one file and langfuse.score({ name: 'user-rating' }) in another.

Correct pattern: import { SCORE_NAMES } from '@/lib/scoring/score-names'; langfuse.score({ name: SCORE_NAMES.USER_RATING });

Add a common mistakes section to CLAUDE.md with these six pairs.

Permission hooks for Langfuse scripts

A Langfuse-instrumented project accumulates scripts: evaluator runs, batch scoring, prompt deployments, trace exports. Some are read-only. Some mutate Langfuse production data. Permission hooks gate the destructive ones.

In .claude/settings.local.json:

{
  "permissions": {
    "allow": [
      "Bash(node scripts/export-traces.js*)",
      "Bash(node scripts/list-prompts.js*)",
      "Bash(node scripts/validate-prompt-yaml.js*)"
    ],
    "deny": [
      "Bash(node scripts/deploy-prompt.js*)",
      "Bash(node scripts/delete-traces.js*)",
      "Bash(node scripts/bulk-score.js*)"
    ]
  }
}

Exporting traces and listing prompts are read-only operations. Deploying a prompt, deleting traces, or running bulk scoring mutate production Langfuse state. The deny list forces Claude to surface those operations as prompts rather than running them as part of an automated workflow.

Building Langfuse instrumentation that actually surfaces issues

The Langfuse CLAUDE.md in this guide produces LLM-app instrumentation where every request opens a trace, every LLM call nests as a generation under that trace, sessions and users are bound on every trace, prompts are fetched from versioned references rather than inlined, scores attach via consistent names, and the in-memory buffer is always flushed before serverless functions exit.

The underlying principle is the same as any observability integration with Claude Code. Langfuse without a CLAUDE.md produces data that looks complete on first inspection but cannot answer the questions you actually need to ask: "which prompt version was active during the latency spike", "which user segment has the highest hallucination rate", "what is the cost-per-conversation breakdown by model". The CLAUDE.md template removes those gaps by making the correct instrumentation pattern the only pattern Claude can generate.

For the next layer of LLM application development, Langfuse works alongside Claude Code with Claude API for direct Anthropic-SDK instrumentation, and Claudify includes a Langfuse-specific CLAUDE.md template with the trace hierarchy, session and user binding, prompt management integration, scoring schemas, and all six common-mistake rules pre-configured.

Get Claudify. Ship LLM traces that you can actually filter, score, and debug.

More like this

🗄️ Integrations & InfrastructureAll topics →

Ready to upgrade your Claude Code setup?

Get Claudify
Featured on Dofollow.Tools AI Toolz Dir Claudify - Featured on Startup Fame