TypeScript

Track and version prompts in TypeScript with ze.prompt()

Installation

npm install zeroeval

Basic Setup

Replace hardcoded prompt strings with ze.prompt(). Your existing text becomes the fallback content that’s used until an optimized version is available.

import * as ze from "zeroeval";
import { OpenAI } from "openai";

ze.init();
const client = ze.wrap(new OpenAI());

const systemPrompt = await ze.prompt({
  name: "support-bot",
  content: "You are a helpful customer support agent for {{company}}.",
  variables: { company: "TechCorp" },
});

const response = await client.chat.completions.create({
  model: "gpt-4",
  messages: [
    { role: "system", content: systemPrompt },
    { role: "user", content: "How do I reset my password?" },
  ],
});

Every call to ze.prompt() is tracked, versioned, and linked to the completions it produces. You’ll see production traces at ZeroEval → Prompts.

When you provide content, ZeroEval automatically uses the latest optimized version from your dashboard if one exists. The content parameter serves as a fallback for when no optimized versions are available yet.

Version Control

Auto-optimization (default)

const prompt = await ze.prompt({
  name: "customer-support",
  content: "You are a helpful assistant.",
});

Uses the latest optimized version if one exists, otherwise falls back to the provided content.

Explicit mode

const prompt = await ze.prompt({
  name: "customer-support",
  from: "explicit",
  content: "You are a helpful assistant.",
});

Always uses the provided content. Useful for debugging or A/B testing a specific version.

Latest mode

const prompt = await ze.prompt({
  name: "customer-support",
  from: "latest",
});

Requires an optimized version to exist. Fails with PromptRequestError if none is found.

Pin to a specific version

const prompt = await ze.prompt({
  name: "customer-support",
  from: "a1b2c3d4...", // 64-char SHA-256 hash
});

Parameters

Parameter	Type	Required	Default	Description
name	string	Yes	—	Task name for this prompt
content	string	No	undefined	Prompt content (fallback or explicit)
from	string	No	undefined	"latest", "explicit", or a 64-char SHA-256 hash
variables	Record<string, string>	No	undefined	Template variables for {{var}} tokens

Return value

Returns Promise<string> — a decorated prompt string with metadata that integrations use to link completions to prompt versions and auto-patch models.

Errors

Error	When
Error	Both content and from provided (except from: "explicit"), or neither
PromptRequestError	from: "latest" but no versions exist
PromptNotFoundError	from is a hash that doesn’t exist

Model Deployments

When you deploy a model to a prompt version in the dashboard, the SDK automatically patches the model parameter in your LLM calls:

const systemPrompt = await ze.prompt({
  name: "support-bot",
  content: "You are a helpful customer support agent.",
});

const response = await client.chat.completions.create({
  model: "gpt-4", // Gets replaced with the deployed model
  messages: [
    { role: "system", content: systemPrompt },
    { role: "user", content: "Hello" },
  ],
});

Manual Prompt-Linked Spans

When you want ze.prompt() to manage a specific LLM interaction but do not want global auto-instrumentation (e.g. your agent makes many LLM calls and only one should be tracked as a prompt generation), disable integrations and create the span yourself.

When to use this

• Your codebase makes many LLM calls but only a subset should appear as prompt completions.
• You use a provider that has no auto-integration (Vapi, a custom HTTP endpoint, etc.).
• You want full control over which codepath produces prompt-linked traces.

Setup

Disable integrations you do not want, then initialize:

import * as ze from 'zeroeval';

ze.init({
  integrations: { openai: false, vercelAI: false },
});

Create a prompt-linked span

1. Call ze.prompt() to register the prompt version and get a decorated string.
2. Use extractZeroEvalMetadata() to split the decorated string into clean content and linkage metadata.
3. Send the clean content to your provider.
4. Wrap the call in ze.withSpan() with kind: 'llm' and the metadata in attributes.zeroeval.

import * as ze from 'zeroeval';
import { extractZeroEvalMetadata } from 'zeroeval';

const decorated = await ze.prompt({
  name: 'customer-support',
  content: 'You are a helpful support agent for {{company}}.',
  variables: { company: 'Acme' },
  from: 'explicit',
});

const { metadata, cleanContent } = extractZeroEvalMetadata(decorated);

await ze.withSpan(
  {
    name: 'llm.chat.completions.create',
    attributes: {
      kind: 'llm',
      task: metadata!.task,
      zeroeval: metadata,
      provider: 'my-custom-provider',
      model: 'gpt-4o',
    },
    inputData: [
      { role: 'system', content: cleanContent },
      { role: 'user', content: 'I need help with my order' },
    ],
    outputData: { role: 'assistant', content: response.text },
  },
  async () => {
    const response = await myProvider.chat([
      { role: 'system', content: cleanContent },
      { role: 'user', content: 'I need help with my order' },
    ]);
    return response;
  }
);

The span will be ingested as an llm span linked to the prompt version. Judge evaluations, feedback, and the prompt completions page all work as if an auto-integration created the span.

extractZeroEvalMetadata is exported from the top-level zeroeval package.

Sending Feedback

Attach feedback to completions to power prompt optimization:

await ze.sendFeedback({
  promptSlug: "support-bot",
  completionId: response.id,
  thumbsUp: true,
  reason: "Clear and concise response",
});

Parameter	Type	Required	Description
promptSlug	string	Yes	Prompt name (same as used in ze.prompt())
completionId	string	Yes	UUID of the completion
thumbsUp	boolean	Yes	Positive or negative feedback
reason	string	No	Explanation of the feedback
expectedOutput	string	No	What the output should have been
metadata	Record<string, unknown>	No	Additional metadata