Skip to content

Data requirements

KD reads a set of observations and searches for the equation behind them. For field data that means two things: a set of coordinate axes, and one or more fields sampled on those coordinates. Scalar regression data is a plain X -> y table instead, and carries no axes at all. The derivatives, the candidate terms and the coefficients are all produced by the algorithm.

Build a dataset from arrays

kd.PDEDataset.from_arrays wraps arrays you already have. The key order of coords fixes the axis order, and each field array must have exactly the shape given by the axis lengths in that order.

import numpy as np
import kd

x = np.linspace(0.0, 2 * np.pi, 128, endpoint=False)   # periodic axis, endpoint not repeated
t = np.linspace(0.0, 1.0, 64)
u = np.sin(x)[:, None] * np.exp(-t)[None, :]      # shape (128, 64)

dataset = kd.PDEDataset.from_arrays(
    coords={"x": x, "t": t},
    fields={"u": u},
    lhs="u_t",
    periodic={"x"},
    name="my-data",
)

kd.preview(dataset)

Dataset: my-data
Axes:
     x | n=128  | range [0.000, 6.234] | step 0.04909 (uniform)
     t | n=64   | range [0.000, 1.000] | step 0.01587 (uniform)
Fields:
     u | dtype=float64 | shape=(128, 64) | min=-1.000 max=1.000 mean=0.000 (NaN=0)
LHS: u_t  (field='u', axis='t')
Status: ready to fit

That dataset goes straight into kd.Model.fit(). kd.preview prints, per axis, the point count, range and whether the step is uniform; per field, the dtype, shape, extrema and NaN count; plus the left-hand side it parsed. kd.preview_report returns the same facts as an object you can read from code. It also gives, for each field and each uniformly spaced axis with at least five points, a noise estimate relative to the field's standard deviation and two finite-difference consistency measures that compare central differences with step 1 and step 2 for the first and second derivatives. They are the noise_estimate, first_derivative_gap and second_derivative_gap dictionaries of each FieldReport, keyed by axis name. kd.preview does not print these three values.

kd.Model(savgol_window=w) smooths every field of a grid dataset along every axis with a Savitzky-Golay filter of window length w and polynomial order 3 before the search starts, for any algorithm; Model.train_surrogate trains on the same smoothed data, and the default 0 leaves the data unchanged. kd.data.savgol_smooth(dataset, w) returns the smoothed copy as a new dataset, so its effect can be inspected with kd.preview_report before a fit.

Load a dataset from a file

kd.load takes a catalog id or a file path. A catalog id returns that entry (kd.load("kdv") is kd.load_kdv()). A path is read on evidence, never on a guess: a registered layout that recognizes the file's arrays builds the dataset, otherwise you name the arrays yourself, or you pass a loader function of your own. kd.inspect_file lists every array first, so the mapping can be written from what the file holds.

import kd

report = kd.inspect_file("1D_Burgers_Sols_Nu0.01.hdf5")   # key, shape, dtype, range per array

dataset = kd.load("1D_Burgers_Sols_Nu0.01.hdf5", select={"sample": 0})   # pdebench-1d layout
dataset = kd.load("run.mat", coords={"x": "x", "t": "t"}, fields={"u": "usol"}, lhs="u_t")
dataset = kd.load("run.bin", loader=read_my_format)   # your function returns a PDEDataset

The registered layouts are pdebench-1d and pdebench-cfd (PDEBench HDF5 files; a file with several samples needs select={"sample": i}) and kd-npz (an .npz holding axis_order, one coordinate array per axis and the field arrays). A field stored in another axis order, or with a sample axis, says so with field_axes= and select=. Periodicity is never in a file: state it with periodic={"x"}. The layout and mapping routes end in from_arrays, and the dataset carries source (path, sha256, container, layout, hints) so a program can see where it came from; it stays out of the dataset fingerprint and out of run records.

Axis and field requirements

  • Any number of axes is accepted. The coords mapping fixes both the count and the order, and each field array carries one dimension per axis. The example above has one spatial axis and time; kd.load_burgers_2d() is a bundled dataset with two spatial axes and time, and lap(u) sums the second derivative over whichever spatial axes the dataset has. Which layouts a given algorithm accepts is declared per algorithm and checked at fit() (see below).
  • Each axis is a 1-D increasing array: a decreasing coordinate is rejected by from_arrays itself, at construction.
  • Axes are evenly spaced. Finite-difference stencils are written around a fixed step, so an unevenly spaced axis is rejected when the derivatives are taken (algorithms that differentiate a trained surrogate instead are not bound by this).
  • Fields are floating-point arrays with no NaN and no Inf. from_arrays casts coordinates and fields to float64 by default.
  • Axis names are free-form strings, but derivative symbols (u_xx) are parsed against single-letter axis names, so x, y, t save you trouble.
  • periodic marks the axes with periodic boundaries: they are differentiated with a wrap-around stencil, so boundary points keep the interior accuracy. Do not repeat the endpoint on a periodic axis (use [0, L), not [0, L]).

The left-hand side

lhs spells out the term on the left of the equation using KD's derivative naming convention; KD parses the field, the axis and the order out of it.

  • lhs="u_t" (default): first-order in time, i.e. u_t = f(u, u_x, u_xx, ...). Most algorithms target this form.
  • lhs="u_tt": second order. The bundled wave and klein-gordon datasets are of this kind. DLGA-PDE is the algorithm that fits them: kd.DLGAConfig(target_lhs_order=2), or the ready-made kd.DLGAConfig.wave_preset() / kd.DLGAConfig.kg_preset(). Every other algorithm builds a first-order evaluator only, and raises at fit() on second-order data rather than quietly fitting a different target.
  • A steady problem has no time derivative and therefore no left-hand side: build it with kd.PDEDataset.from_scatter(..., lhs="") and the equation reads as a sum of terms equal to zero. The three bundled Laplace / Poisson datasets are of this kind.

Scattered points

Data that does not sit on a regular grid goes through kd.PDEDataset.from_scatter: every coordinate array and every field array is a 1-D array of length N, one entry per observed point, and the coordinates need not be sorted. kd.PDEDataset.from_xlsx reads the same structure directly out of an xlsx workbook by column header, which fits experimental data already stored as (x, y, u) columns.

Scattered data is handled by EqGPT: its multi-case wave mode and its steady mode declare scattered.

Tabular data

A scalar regression task is a plain X -> y table, and kd.TabularDataset carries one: X is an array of shape (n_samples, n_features), y is a (n_samples,) array of the measured target, var_names gives one name per column of X, and target_name names y. There are no coordinate axes: a row is one observation, not a sample of a field. name, source and description record where the table came from, and are required alongside the four above.

import numpy as np
import kd

pressure = np.linspace(1.0, 5.0, 40)
flow = np.linspace(0.2, 2.0, 40)

table = kd.TabularDataset(
    X=np.column_stack([pressure, flow]),
    y=2.0 * pressure + 3.0,
    var_names=("pressure", "flow"),
    target_name="reactor_yield",
    name="my-table",
    source="lab notebook",
    description="reactor yield against pressure and flow rate",
)

fit() reads the table through one converter, and that is where the names and the values are checked: every name is a Python identifier, distinct from the others, and neither a Python keyword nor one of the names KD reserves for its own functions, derivatives and the constant term one; every value is finite; the columns are read as float64.

That table goes straight into kd.Model.fit(). Each column becomes a field of its own and the target column becomes the left-hand side. No derivatives are taken, so a column named x or t is just another feature. The result is an expression in the feature names rather than a PDE. result_.lhs_label is the target column, and best_expr_ carries the expression together with the constants fitted for it. An algorithm whose tabular mode declares the REGRESSION form returns the same answer on result_.equation as well, the target column on the left of it.

DISCOVER, PySR and PySINDy read this layout: each declares a tabular mode, and KD selects that mode from the dataset you hand to fit(). On a table, the reserved term one is a column of ones with the length of the target column. PySINDy's tabular library starts with one, followed by every monomial of the feature columns up to PySINDyConfig.degree, so an additive constant in the fitted law is the coefficient of one and is printed as a number in result_.equation. KD ships one table of its own, kd.load_tlc_cc(), which returns a TabularDataset ready to fit.

Supported layouts per algorithm

Every mode of every algorithm declares the layouts it accepts (topologies: grid / scattered / tabular) and the equation forms it searches (forms: EVOLUTION / HOMOGENEOUS / REGRESSION, or empty for a mode that returns a bare expression) in kd.instrument_schemas(); the spec strip at the top of each algorithm page shows the same information. When the layout or the left-hand-side order of your data does not match the algorithm, fit() raises and names what that algorithm does support.

To try an algorithm before pointing it at your own data, start from the bundled datasets. For how the derivatives are produced, see derivative sources.