From your first dataset
to a training-ready bundle.

Clone the repository and install it in a Python virtual environment. The default demo path does not need a GPU or an API key.

git clone https://github.com/RenaGao/jev-dataops.git
cd jev-dataops
python3 -m venv .venv
source .venv/bin/activate
pip install -e .
jev-dataops serve

Open http://127.0.0.1:8000. Select Use example to run a small synthetic dataset through the workspace before connecting real services.

Upload UTF-8 JSONL or CSV. The screening workflow accepts text, instruction–response pairs, prompt–response pairs, and conversations. A simple instruction example looks like this:

{"instruction":"Explain operating cash flow.","output":"Cash generated by a company's core operations."}
{"instruction":"What does a Python list store?","output":"An ordered, mutable collection of objects."}

These two lines illustrate the format; they are not a sufficient training dataset. Use enough independent examples to populate train, validation, and test splits. Preserve relevant group or conversation identifiers to reduce leakage.

1. Select a screening engine. Demo uses local rules. A configured JEV provider performs model-based screening.
2. Choose your domain rubric. General, finance, and code provide starting points for domain-aware screening.
3. Set the request budget. Review concurrency and request limits before calling a provider.
4. Review the decisions. Inspect keep, review, and reject outputs plus the data report. Only retained records should enter the next stage.

The self-hosted workspace sends record content to the selected JEV provider when online screening is enabled. Use records you have permission to share.

Choose the training objective and export the screened data. A bundle gives you files and configuration to inspect before a job is launched. SFT export accepts text or conversation data; RL export additionally requires verified answer targets.

pip install -e '.[export]'

For a complete closed-answer example, screen the bundled synthetic arithmetic dataset, which includes independently specified ground_truth strings. A JEV score or an arbitrary response is not a reward target.

jev-dataops export-training \
  --input runs/example/screening/keep.jsonl \
  --output runs/training-bundle \
  --target sft \
  --base-model Qwen/Qwen2.5-0.5B-Instruct \
  --n-gpus 1 \
  --batch-size 4

jev-dataops export-training \
  --input runs/example/screening/keep.jsonl \
  --output runs/training-bundle \
  --target verl-grpo \
  --base-model Qwen/Qwen2.5-0.5B-Instruct \
  --reward-field ground_truth \
  --n-gpus 1 \
  --batch-size 4

jev-dataops export-training \
  --input runs/example/screening/keep.jsonl \
  --output runs/training-bundle \
  --target verl-ppo \
  --base-model Qwen/Qwen2.5-0.5B-Instruct \
  --reward-field ground_truth \
  --n-gpus 1 \
  --batch-size 4

Use a different output directory for each target or experiment, not for each GPU. One bundle is shared across all workers in that run. These examples are single-node training; the public browser demo does not allocate GPUs or execute them. See the multi-GPU setup guide for device selection, batch sizing, and memory requirements.

Copy the bundle once to your Linux/NVIDIA training host if it was exported elsewhere, preserving its directory structure. Run the launcher once from that host: the selected GPU count is already stored in the bundle. Dry-run the launch and confirm the command, paths, reward configuration, and available GPUs before execution.

jev-dataops launch-training \
  --bundle runs/training-bundle \
  --dry-run

The local LoRA workflow measures baseline and trained-model loss on the same held-out test set. Use the ready-to-use domain metric packs to evaluate supplied model outputs, references, imported test results, and reviewed labels offline. Choose general, finance, code, enterprise, legal, or medical definitions. Keep each metric’s coverage alongside its result; the packs do not generate predictions or expert judgments.