# Length Bias Audit

Audit length bias (verbosity bias) in LLM-as-a-judge evaluations and preference datasets, detecting artificial win-rate inflation driven purely by longer responses rather than superior accuracy.

## Workflow

1. Ingest evaluation battles:
   - Battle identifier
   - Response A length (characters or tokens)
   - Response B length (characters or tokens)
   - Winner label (`model_a`, `model_b`, `tie`)
2. Categorize battles where length difference is material ($\ge 25\%$ length gap).
3. Compute win rate for longer response vs shorter response:
   $$\text{Win Rate}_{\text{longer}} = \frac{N_{\text{longer wins}}}{N_{\text{material battles}}}$$
4. Measure correlation between response length delta and judge score.
5. Flag benchmarks suffering from excessive verbosity bias ($\text{Win Rate}_{\text{longer}} > 65\%$).

## Synthetic Input Format

JSON object with:
- `schemaVersion`: 1
- `benchmark`: string
- `judgeModel`: string
- `battles`: list of objects, each containing:
  - `battleId`: string
  - `lenA`: positive integer
  - `lenB`: positive integer
  - `winner`: `"model_a"` | `"model_b"` | `"tie"`
- `assertions`:
  - `maxLongerWinRate`: optional decimal string (e.g. "0.6500")

## Exit Codes

- `0`: Valid evaluation trace and length bias within threshold.
- `1`: Length bias assertion failed.
- `2`: Invalid input format or safety violation.

## Verification

```sh
python3 scripts/check.py examples/synthetic.json
```
