Batch experiments

kd.harness runs a matrix of fits and keeps what it ran. The loop is four calls: an ExperimentPlan of PlanEntry rows states the matrix, run_plan executes it into a fresh directory, EvidenceStore.load reopens that directory and re-verifies it, and build_consensus groups the runs by the law each one selected. The store's guarantees and the meaning of the consensus axes are described under Batch experiments and evidence.

These names are imported from the subpackage rather than the package root: from kd.harness import ExperimentPlan, run_plan. Listed below are the entry points a caller writes; the module also carries the sharding and artifact machinery that run_plan and build_consensus drive on their own.

Name	Summary
`kd.harness.ExperimentPlan(name, entries)`	An ordered execution matrix of plan entries with a content identity.
`kd.harness.PlanEntry(instrument, dataset_ref[, ...])`	One declarative episode: an instrument applied to a dataset at a seed.
`kd.harness.run_plan(plan, datasets[, ...])`	Pre-flight, then run every plan entry serially into a fresh store.
`kd.harness.PlanRunResult(store_root, outcomes)`	The outcome of a whole batch: the store root plus every episode outcome.
`kd.harness.EpisodeOutcome(entry_index, entry[, ...])`	The sealed result of running one plan entry.
`kd.harness.EvidenceStore(root, plan[, ...])`	A fresh-dir, append-only, tamper-evident batch evidence directory.
`kd.harness.build_consensus(store, datasets[, ...])`	Aggregate a sealed evidence store into a consensus report tree.
`kd.harness.ConsensusReport(provenance, datasets[, ...])`	The full consensus report tree for one sealed evidence store.
`kd.harness.render_consensus_markdown(report)`	Render the report as a deterministic Markdown document (pure function).
`kd.harness.SKETCH_SIDECAR_FILENAME`	—

ExperimentPlan `dataclass`

An ordered execution matrix of plan entries with a content identity.

from_dict `classmethod`

from_dict(data: dict[str, Any]) -> ExperimentPlan

Strictly decode a persisted plan payload.

plan_hash

plan_hash() -> str

Return the SHA-256 kd-plan-v1 identity of this plan.

to_dict

to_dict() -> dict[str, Any]

Return the pure content payload (no embedded hash).

PlanEntry `dataclass`

One declarative episode: an instrument applied to a dataset at a seed.

from_dict `classmethod`

from_dict(data: dict[str, Any]) -> PlanEntry

Strictly reconstruct an entry (validation re-runs in __init__).

to_dict

to_dict() -> dict[str, Any]

Return a defensive copy of the full JSON-native entry payload.

run_plan

run_plan(
    plan: ExperimentPlan,
    *,
    datasets: Mapping[str, PDEDataset | TabularDataset],
    store_root: Path,
    model_factory: Callable[..., Any] = Model,
    device: str | None = None,
    recording: RecordingOptions | None = None,
    catalog_path: Path | None = None,
    entry_indices: Sequence[int] | None = None,
    resume_from: Mapping[int, Path | str] | None = None,
    plan_hash: str | None = None,
) -> PlanRunResult

Pre-flight, then run every plan entry serially into a fresh store.

Parameters:

plan (ExperimentPlan) –

The ordered execution matrix to run.
datasets (Mapping[str, PDEDataset | TabularDataset]) –

Maps each dataset_ref to a loaded PDEDataset or a public TabularDataset (normalized once at entry through the same seam as Model.fit).
store_root (Path) –

Fresh directory for the EvidenceStore (must not exist or must be empty; append/resume is out of scope).
model_factory (Callable[..., Any], default: Model ) –

Facade constructor forwarded to run_episode; injectable so unit tests can stub the fit.
device (str | None, default: None ) –

Execution-environment parameter forwarded to run_episode and, from there, conditionally to the facade. None leaves the serial-execution call form literally unchanged.
recording (RecordingOptions | None, default: None ) –

When set, each entry runs inside a standard run dir <store_root>/runs/entry-NNNN/ (store-LOCAL numbering, matching records/entry-NNNN.json) wired per the options.
catalog_path (Path | None, default: None ) –

Cross-run ledger to append one kd-runcat-v1 row per outcome (requires recording; rows carry the GLOBAL entry index and paths relative to the catalog file).
entry_indices (Sequence[int] | None, default: None ) –

GLOBAL plan index for each local entry (a dispatch worker passes its shard's entry_indices); None means this IS the whole plan (global == local).
resume_from (Mapping[int, Path | str] | None, default: None ) –

Map of GLOBAL entry index -> checkpoint path to resume that entry from (retry lanes); forwarded per-entry to run_episode. Dispatch keys are validated against shard coverage at the manifest level.
plan_hash (str | None, default: None ) –

Overrides the hash stamped into catalog rows; defaults to store.plan_hash. Dispatch workers pass the batch manifest's full-plan hash.

Returns:

PlanRunResult –

A PlanRunResult with the store root and one outcome per entry.

Raises:

ValueError –

If pre-flight finds any unknown instrument or unresolved dataset_ref (all violations are collected into the message), if catalog_path is given without recording, or if entry_indices does not match the plan length.

PlanRunResult `dataclass`

The outcome of a whole batch: the store root plus every episode outcome.

EpisodeOutcome `dataclass`

The sealed result of running one plan entry.

EvidenceStore

A fresh-dir, append-only, tamper-evident batch evidence directory.

attempts `property`

attempts: tuple[dict[str, Any], ...]

Return the attempt ledger as a tuple of independent dict copies.

env `property`

env: dict[str, str]

Return a copy of the stored environment fingerprint.

plan `property`

plan: ExperimentPlan

Return the sealed experiment plan.

plan_hash `property`

plan_hash: str

Return the kd-plan-v1 identity of the sealed plan.

records `property`

records: MappingProxyType[int, RunRecord]

Return a read-only view of completed-episode records by entry index.

root `property`

root: Path

Return the store's root directory.

add_outcome

add_outcome(outcome: Any) -> None

Record one episode outcome and rewrite the index atomically.

create `classmethod`

create(
    root: Path, *, plan: ExperimentPlan, env: dict[str, str]
) -> EvidenceStore

Create a fresh store rooted at root.

load `classmethod`

load(root: Path) -> EvidenceStore

Load and fully verify a persisted store (returned READ-ONLY).

build_consensus

build_consensus(
    store: EvidenceStore,
    *,
    datasets: Mapping[str, PDEDataset] | None = None,
    policy: ConsensusPolicy = ConsensusPolicy(),
    instrument_schemas: Sequence[Mapping[str, Any]]
    | None = None,
    context_factory: VerifyContextFactory | None = None,
) -> ConsensusReport

Aggregate a sealed evidence store into a consensus report tree.

ConsensusReport `dataclass`

The full consensus report tree for one sealed evidence store.

render_consensus_markdown

render_consensus_markdown(report: ConsensusReport) -> str

Render the report as a deterministic Markdown document (pure function).

SKETCH_SIDECAR_FILENAME `module-attribute`

SKETCH_SIDECAR_FILENAME: Final[str] = 'sketch.json'

Batch experiments

ExperimentPlan dataclass

from_dict classmethod

plan_hash

to_dict

PlanEntry dataclass

from_dict classmethod

to_dict

run_plan

PlanRunResult dataclass

EpisodeOutcome dataclass

EvidenceStore

attempts property

env property

plan property

plan_hash property

records property

root property

add_outcome

create classmethod

load classmethod

build_consensus

ConsensusReport dataclass

render_consensus_markdown

SKETCH_SIDECAR_FILENAME module-attribute

ExperimentPlan `dataclass`

from_dict `classmethod`

PlanEntry `dataclass`

from_dict `classmethod`

PlanRunResult `dataclass`

EpisodeOutcome `dataclass`

attempts `property`

env `property`

plan `property`

plan_hash `property`

records `property`

root `property`

create `classmethod`

load `classmethod`

ConsensusReport `dataclass`

SKETCH_SIDECAR_FILENAME `module-attribute`