The Linter For
Fine-Tuning Data
Parallelogram validates your dataset before you spend GPU hours on it. If it exits 0, your training run won't fail because of data.
Works with
- Claude Code
- Codex
- Cursor
- Windsurf
- Cline
- GitHub Copilot
- Gemini CLI
- Zed
- JetBrains
- Warp
- TRAE
- Hugging Face
- GitHub Actions
Catch It Before The GPU Does
One command between your dataset and your trainer: on your machine, in CI, or called by your coding agent.
$ parallelogram check train.jsonl --eval eval.jsonl
✗ train.jsonl:2 [secrets] Message 0 contains a likely OpenAI API key: sk-pro…
✗ train.jsonl:3 [preference] Chosen and rejected answers are identical
✗ train.jsonl:4 [duplicates] Duplicate of line 1
! train.jsonl:5 [encoding] Message 1 contains likely mojibake: '’'
! train.jsonl:5 [pii] Message 0 contains an email address: t***@gmail.com
✗ train.jsonl:6 [eval-overlap] Prompt also appears in the eval set (eval.jsonl:1)
6 records 2 clean 4 errors 2 warnings exit 2
$ parallelogram check train.jsonl --eval eval.jsonl --fix --output clean.jsonl
✓ duplicates · 1 ✓ encoding · 1 ✓ eval-overlap · 1 ✓ pii · 1 ✓ secrets · 1
✗ dropped: train.jsonl:3 → preference (unfixable)
train.jsonl:4 → duplicates (dropped as fix)
train.jsonl:6 → eval-overlap (dropped as fix)
→ Wrote 3 records to clean.jsonl exit 1
$ parallelogram check clean.jsonl --eval eval.jsonl
3 records 3 clean 0 errors 0 warnings exit 0name: Validate training data
on: [push, pull_request]
jobs:
validate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: Thatayotlhe04/Parallelogram@v1
with:
path: data/train.jsonl
fail-on: error # errors (2) and regressions (3) always fail
# eval: data/eval.jsonl # fail on train/eval leakage
# baseline: baseline/report.json
# The health report lands in the job summary. A dataset PR that breaks
# role order fails the same way a code PR with a failing test does.$ claude mcp add parallelogram -- parallelogram mcp # Claude Code
$ codex mcp add parallelogram -- parallelogram mcp # Codex
→ check_dataset {"path": "train.jsonl", "eval_path": "eval.jsonl", "limit": 3}
{ "summary": { "errors": 4, "warnings": 2 }, "issues_truncated": true,
"by_rule": { "secrets": …, "preference": …, "duplicates": …, "eval-overlap": … },
"issues": [
{ "rule": "secrets", "line": 2, "message": "…OpenAI API key: sk-pro…", "fixable": true },
{ "rule": "preference", "line": 3, "message": "…answers are identical", "fixable": false },
{ "rule": "duplicates", "line": 4, "message": "Duplicate of line 1", "fixable": true } ] }
→ fix_dataset {"path": "train.jsonl", "eval_path": "eval.jsonl", "output": "clean.jsonl"}
{ "summary": { "fixed": 2, "dropped": 3, "emitted": 3 } } clean.jsonl rechecks cleanTen Checks Your Trainer Won't Run
Each one crashes the run, quietly trains the wrong thing, or leaks something it shouldn't.
All rules-
Secrets New in 1.1
API keys, tokens and private keys a model could repeat. Redacted by --fix, never printed in full.
secretserrorfixable -
Personal Data New in 1.1
Email addresses, phone numbers and card numbers. Replaced with placeholders by --fix.
piiwarningfixable -
Eval Leakage New in 1.1
Train records whose content or prompt also sits in your eval set, inflating the score.
eval-overlaperrorfixable -
Preference Pairs New in 1.1
DPO pairs whose chosen and rejected answers are the same, so they teach nothing.
preferenceerror -
Schema
Malformed JSONL, missing fields, wrong types, in any of six formats. Can't be disabled.
schemaerror -
Role Order
Doubled turns, misplaced system prompts, conversations that end on the user.
roleserror -
Empty Content
Blank or whitespace-only turns that teach the model silence.
empty-contenterrorfixable -
Duplicates
Repeated records that push the model toward memorizing instead of generalizing.
duplicateserrorfixable -
Context Window
Records over your token budget, which TRL truncates without telling you.
context-windowerrorfixable -
Encoding
BOM markers and mojibake that end up baked into the weights.
encodingwarningfixable
Check. Fix. Gate.
Every issue gets a line and a rule. Mechanical problems are repaired, the rest are dropped with a reason, and the re-check decides.
- 01 · check
parallelogram check data.jsonlEvery issue with its line number and rule. Exits 0, 1 or 2.
- 02 · fix
parallelogram check data.jsonl --fix --output clean.jsonlMechanical repairs, then a re-check. Anything still broken is dropped, with the reason.
- 03 · gate
uses: Thatayotlhe04/Parallelogram@v1The GitHub Action fails the pull request when dataset quality regresses.
- 1.5.0
For coding agents: an MCP server, a Python API and
--json --limit - 1.1.1
Fixes from an independent review, including secrets quoted in diagnostics
- 1.1.0
Alpaca and DPO formats, secrets and PII checks, train/eval leakage with
--eval - 1.0.0
Stable contract: prompt/completion format, GitHub Action, versioned JSON,
--version
What does "exits 0" actually guarantee?
With all rules on: every record parses, has a valid role order ending on the assistant, has no empty turns or exact duplicates, and fits your token budget (counted exactly with --tokenizer, estimated otherwise). It checks that your data will train, not that it's good data.
Does my data leave my machine?
No. Nothing is uploaded and there is no telemetry. The only network request is opt-in: --tokenizer may download a tokenizer vocabulary once, then caches it.
Which formats are supported?
OpenAI/Qwen chat, ShareGPT, prompt/completion, Alpaca and preference (DPO) pairs, detected per line and mixable in one file. --fix writes each record back in the shape it came in.
Will it catch a leaked API key?
Yes. The secrets check finds keys and tokens for OpenAI, Anthropic, AWS, GitHub, Hugging Face, Slack, Google and Stripe, plus private key blocks. It never prints them in full, and --fix redacts them.
Are token counts exact?
With pip install 'parallelogram[tokenizer]' and --tokenizer, yes (tiktoken or Hugging Face). Without, counts are estimated and overflows are warnings, never errors.
How do I use it in CI?
Add uses: Thatayotlhe04/Parallelogram@v1 to a workflow. It writes a report to the job summary and fails on errors, or on regressions against a --baseline.
Can my coding agent run it?
Yes. parallelogram mcp gives Claude Code, Codex and Cursor three tools: check, fix and list rules. Agents using the shell can run check --json --limit 50, which keeps every count exact but lists only the most important issues. Setup is in the agent guide.
Is it free?
Yes. Apache-2.0, on PyPI and GitHub.