# Needle-in-a-Haystack Density Audit

Audit long-context retrieval recall across varying context lengths (32k, 128k, 256k, 1M, 2M tokens), needle depth placements (0% to 100%), and distractor similarity levels.

## Workflow

1. Ingest retrieval test matrix:
   - Total context length in tokens ($L$)
   - Needle depth placement percentile ($D \in [0.0, 1.0]$)
   - Retrieved fact exact match / semantic accuracy ($A \in [0.0, 1.0]$)
   - Distractor domain (e.g. random text vs high semantic overlap)
2. Aggregate retrieval accuracy by context tier:
   - Near context ($\le 32\text{k}$)
   - Mid context ($32\text{k} - 128\text{k}$)
   - Deep context ($> 128\text{k}$)
3. Identify "Lost in the Middle" anomalies:
   - Detect severe accuracy dips when needles are placed between depth 30% and 70%.
4. Verify long-context recall meets application threshold (e.g. $\ge 95\%$).

## Synthetic Input Format

JSON object with:
- `schemaVersion`: 1
- `model`: string
- `provider`: string
- `tests`: list of objects, each containing:
  - `contextLength`: positive integer
  - `depthPercent`: non-negative integer or decimal (0 to 100)
  - `recalled`: boolean
- `assertions`:
  - `minOverallRecall`: optional decimal string (e.g. "0.9000")
  - `minMiddleDepthRecall`: optional decimal string (for depths between 30% and 70%)

## Exit Codes

- `0`: Valid matrix and all recall thresholds met.
- `1`: Retrieval recall failure.
- `2`: Invalid input format or safety violation.

## Verification

```sh
python3 scripts/check.py examples/synthetic.json
```