# Parity Strategy

`nirs4all-lite` needs parity gates because it is intended to become the portable
core behind multiple host-language APIs.

## Tiers

1. **Upstream native vs upstream binding**: each upstream project proves its own
   binding parity first, especially `nirs4all-methods`.
2. **nirs4all-lite native vs binding**: lite pipelines produce identical
   results across Rust, Python, R, MATLAB/Octave, and WASM within declared
   tolerance.
3. **nirs4all-lite vs full Python nirs4all**: equivalent pipelines match the
   current Python library before the lite binding can replace any core path.

## Fixture policy

Fixtures must record:

- dataset identity and provenance;
- pipeline descriptor;
- upstream versions or commit SHAs;
- numeric tolerance and dtype;
- platform/runtime notes when relevant.

Golden outputs should be generated from the owning upstream implementation, not
from host-language rewrites.

Pipeline definition fixtures must use the full Python `nirs4all` JSON/YAML
syntax. The parser-level contract accepts JSON/YAML paths, JSON/YAML text,
direct step lists, `pipeline`, and `steps` in every target binding, but
canonical fixtures should keep the same `pipeline` envelope used in
`nirs4all/examples/pipeline_samples`. Execution parity is a separate gate: a
definition that parses is not considered validated until native lite and every
target binding match the full Python `nirs4all` result for the portable operator
subset.

External operators follow the same rule. A language binding can expose operator
metadata before execution support exists, but an operator cannot be marked
`execute-local`, `execute-remote`, or `parity-validated` until a fixture proves
that the host-language adapter delegates to the same upstream behavior.

## Initial parity targets

- PLS pipelines backed by `nirs4all-methods`.
- Format load/record conversion backed by `nirs4all-formats`.
- Dataset materialization backed by `nirs4all-io`.
- Catalog resolution backed by `nirs4all-datasets`.
- DAG execution descriptors backed by `dag-ml` and `dag-ml-data`.
- External operator adapters once the relevant executor can plan or call them.

## Current execution gates

The initial numeric gates compare the JavaScript/WASM, Python, Rust, R, and
MATLAB/Octave bindings against full Python `nirs4all` on four shared JSON/YAML
pipeline fixtures:

- SNV + PLS;
- Savitzky-Golay + PLS;
- Kennard-Stone + SNV + PLS;
- Kennard-Stone + SNV + Savitzky-Golay + PLS component sweep.

The Python oracle is generated by
`scripts/parity/generate_python_oracle.py` into
`tests/parity/expected/portable_python_oracle.json`. It uses the real Python
`nirs4all` operators for Kennard-Stone, SNV, and Savitzky-Golay, plus
`sklearn.cross_decomposition.PLSRegression`. The WASM, Python, Rust, R, and
MATLAB/Octave tests then execute the same fixture descriptors through their
`runPortablePipeline()`, `run_portable_pipeline()`,
`run_portable_pipeline_with_library()`,
`nirs4all_run_portable_pipeline()`, and `nirs4all.runPortablePipeline()` APIs and
compare split indices, targets, RMSE, predictions, and selected sweep component
within the declared tolerances. The WASM gate also reuses the serialized
selected model through `predictPortablePipeline()` and checks that predictions
on the held-out rows match the selected run output.

The shared parser contract also has regression coverage for Savitzky-Golay
defaults and explicit boundary modes. `mode` defaults to `interp` to match full
Python nirs4all, while explicit methods-backed SciPy modes (`mirror`,
`constant`, `nearest`, `wrap`, `interp`) and `cval` are preserved across Python,
Rust, JavaScript/WASM, R, and MATLAB/Octave.

Regenerate the oracle from a workspace that has full Python `nirs4all`
installed:

```bash
PYTHONPATH=/home/delete/nirs4all/nirs4all \
  /home/delete/nirs4all/nirs4all/.venv/bin/python \
  scripts/parity/generate_python_oracle.py
```

Run the WASM parity gate after building/staging `nirs4all-methods` JS/WASM.
Set `NIRS4ALL_METHODS_JS_DIST` when the methods checkout is not a sibling of
this repository. Set `NIRS4ALL_LITE_REQUIRE_METHODS_PARITY=1` to make a
missing methods artefact fail instead of skip:

```bash
NIRS4ALL_METHODS_JS_DIST=/path/to/nirs4all-methods/bindings/js/dist \
NIRS4ALL_LITE_REQUIRE_METHODS_PARITY=1 \
npm test --prefix bindings/wasm
```

Run the strict Python parity gate with `nirs4all-methods` importable and a
local libn4m build:

```bash
PYTHONPATH=bindings/python/src:/path/to/nirs4all-methods/bindings/python/src \
PLS4ALL_LIB_PATH=/path/to/libn4m.so \
NIRS4ALL_LITE_REQUIRE_METHODS_PARITY=1 \
python -m unittest bindings/python/tests/test_execution_parity.py -v
```

Run the strict Rust parity gate with a local libn4m build:

```bash
NIRS4ALL_METHODS_LIB=/path/to/libn4m.so \
LD_LIBRARY_PATH=/path/to/libn4m-directory \
NIRS4ALL_LITE_REQUIRE_METHODS_PARITY=1 \
cargo test -p nirs4all rust_binding_execution_matches_full_python_nirs4all_oracle -- --nocapture
```

Run the strict R parity gate after installing an `n4m` R binding that exposes
the portable preprocessing and splitter surface:

```bash
NIRS4ALL_LITE_PARITY_ORACLE=$PWD/tests/parity/expected/portable_python_oracle.json \
NIRS4ALL_LITE_PARITY_FIXTURES=$PWD/bindings/r/inst/extdata \
NIRS4ALL_LITE_REQUIRE_METHODS_PARITY=1 \
Rscript bindings/r/tests/parity.R
```

Run the strict MATLAB/Octave parity gate after building the `nirs4all-methods`
`+pls4all` MEX shims:

```bash
NIRS4ALL_LITE_PARITY_ORACLE=$PWD/tests/parity/expected/portable_python_oracle.json \
NIRS4ALL_LITE_PARITY_FIXTURES=$PWD/tests/parity/fixtures \
NIRS4ALL_LITE_REQUIRE_METHODS_PARITY=1 \
octave --quiet --eval "addpath('bindings/matlab/tests'); parity"
```