DOMAIN METRIC PACKS · OPEN DEFINITIONS

Measure what matters.
Know the evidence.

Choose a domain. Inspect each metric, its inputs, and its limits. Then evaluate your own records with an offline, reproducible recipe.

6 domain starter packsJSON + JSONLOffline CLI

CHOOSE YOUR DOMAIN

Start with definitions. Calibrate for your task.

Loading the General metric pack…

Enable JavaScript to inspect metric definitions and switch domains. The packs are also available in the source repository.

Download metric pack Example JSONL

—metric definitions

—computed from inputs

—use supplied evidence

—declared input fields

READ THE EVIDENCEComputed from your inputsImported test resultsSupplied human labels

Metric definitions

Expand a metric to inspect its method and required fields.

MAKE THE INPUTS EXPLICIT

Fields & provenance

Field definitions come directly from the selected pack. A field's presence does not verify its source or label.

Input field types, descriptions, and metrics that use them
Field path	Type	Definition	Used by

Keep source identifiers, test configuration, annotator or reviewer details, and rubric versions with your evaluation records. Human labels and external test results are supplied evidence; this evaluator does not produce or independently verify them.

INTERPRET BEFORE YOU COMPARE

Coverage is part of the result.

Missing or invalid required fields reduce coverage. They are not silently counted as zero-quality answers. Inspect the denominator and unavailable records alongside every metric.

The starter packs do not provide an overall score or industry-approved pass targets. A null target means no target is configured; calibrate thresholds and minimum coverage on your own reviewed evaluation data.

A SMALL, INSPECTABLE STARTING POINT

See the record shape.

The bundled fixture is synthetic. Supplied annotations demonstrate the schema; they are not benchmark results or expert validation.

example.jsonl · first record

YOUR RECORDS. YOUR ENVIRONMENT.

Take the recipe
into your workflow.

Initialize a pack and example locally, then evaluate the JSONL records. Use your own model outputs, references, test results, and reviewed labels for a meaningful run.

New to JEV DataOps? Start here

These commands run on your machine. This catalog only reads static pack files; it does not upload or score your data.

1. Initialize the selected metric pack

jev-dataops init-metrics \
  --domain general \
  --output ./general-metrics

2. Run the offline evaluator

jev-dataops evaluate-metrics \
  --input ./general-metrics/example.jsonl \
  --pack ./general-metrics/metrics.json \
  --output ./general-results