DLGA-PDE

A neural network fits the field and supplies derivatives; a genetic algorithm searches term sets over a candidate library.

algorithm

dlga

score

DLGA fitness min

cost

heavy

derivatives

autograd

equation form

EVOLUTION

data layout

grid

Usage

import kd

dataset = kd.load_burgers()
model = kd.Model(algorithm="dlga", generations=40, seed=1, pop_size=40)
model.fit(dataset)

Main parameters

Parameter	Default	Resume	Description
pop_size	400	resume-safe	Genetic population size.
epsilon	0.001	init-only	Expression-length penalty.
mutation_rate	0.4	resume-safe	Genome mutation probability.
crossover_rate	0.8	resume-safe	Genome crossover probability.
surrogate_lr	0.001	init-only	Neural-surrogate learning rate.

All fields of DLGAConfig

Field	Type	Default
mode	Literal['constant', 'adaptive', 'auto']	constant
library	list[str]	['u', 'u_x', 'u_xx', 'u_xxx']
solver	Literal['svd_null_space', 'ols']	svd_null_space
lhs_auto_select	bool	True
target_lhs_order	int	1
epsilon	float	0.001
pop_size	int	400
seed	int	0
auto_upgrade_threshold	float	0.001
max_modules	int	5
max_module_length	int	5
partial_prob	float	0.6
genes_prob	float	0.6
crossover_rate	float	0.8
mutation_rate	float	0.4
add_rate	float	0.4
delete_rate	float	0.5
surrogate_hidden_sizes	list[int]	[50, 50, 50, 50, 50]
surrogate_activation	Literal['tanh', 'sin', 'relu']	sin
surrogate_lr	float	0.001
surrogate_max_epochs	int	50000
surrogate_patience	int \| None	None
surrogate_val_ratio	float	0.2
surrogate_restore_best	bool	True

Worked example

The run below uses the Burgers dataset that ships with KD: a 256 × 201 grid over x in [-8, 8] and t in [0, 10], whose initial profile steepens into a front at the origin and then spreads out as the diffusive term takes over.

The observed field u over x and t.

The equation returned by the run:

	Equation
Discovered	\(u_t = -0.9993\,u\,u_x +0.0999\,u_{xx}\)
Reference	\(u_t = -u\,u_x +0.1\,u_{xx}\)

Both terms and both coefficients match. The terms were assembled from the default candidate library (u, u_x, u_xx, u_xxx), and the left-hand side was chosen by the run as well: DLGA-PDE fits both u_t and u_tt and keeps the one with the lower nmse.

The discovered equation evaluated against the u_t the search fitted, which is the network's u_t rather than a finite difference of the observations. The first two panels share a color scale and are indistinguishable by eye; the third is their difference, at most 3.1e-02, and it is concentrated along the front.

Result interpretation

model.best_score_ is DLGA-PDE's own fitness, nmse + epsilon * length: lower is better. This run ends at 3.46e-03, of which 4.61e-04 is the nmse and the remaining 3.0e-03 comes from epsilon, at its default of 1e-3, applied to the three tokens of the answer. The fitness combines fit and length, so nmse alone does not determine which of two term sets is preferred.

That nmse is measured against the network's field, which is the field the search operates on. The fitted equation itself is on model.result_.equation, term by term with its coefficient.

The search is recorded as well. kd.VizEngine renders the per-generation record; alongside the universal convergence curve, DLGA-PDE contributes four diagnostic panels of its own (the full list is under Visualization):

The best fitness starts at 0.4489 and comes down in steps to 3.46e-03, the last of them at generation 29 of 40. The axis is logarithmic: the descent covers two orders of magnitude, and on a linear axis every step after the first would read as one flat line.

Left: distinct candidate expressions per generation, between 29 and 39 out of a population of 40. Middle: mean term count of the valid candidates, between 1.78 and 2.68, so the population stays near the length of the answer. Right: mean fitness of the population on a log axis, between 0.86 and 2.3e+03. A single ill-conditioned candidate can raise a generation's mean into the thousands, so the axis is logarithmic and the best curve is drawn separately.

The fourth panel shows the training of the surrogate network, which is independent of the search:

Training and validation MSE over the 50000 epochs the surrogate is given, from 3.2e-01 down to 5.9e-08 across six decades. The two curves track each other the whole way. The vertical line marks the epoch whose weights were kept, 47600, which is the best one rather than the last.

viz = kd.VizEngine(output_dir="out/burgers")
viz.render_all(model.result_, algorithm=model.algorithm_, dataset=dataset)

Method

A neural network fits the observations first. Every derivative the search uses is taken from that network by autograd, so the candidate terms are built on a smooth field rather than on finite differences of the raw data.

A candidate is a genome: a set of terms, each term a product of tokens drawn from the candidate library (u, u_x, u_xx, u_xxx by default). Crossover and mutation produce the next generation, and every candidate is scored by nmse + epsilon * length, which combines the fit error against the surrogate field with a penalty per token. Lower scores are better. One generation is one KD iteration, and generations sets how many are run.

The coefficients in the returned equation come from a platform refit rather than from the algorithm: DLGA-PDE searches for the term set, and the coefficients are fitted on that set afterwards.

References

Xu et al. (2020). "DLGA-PDE: Discovery of PDEs with incomplete candidate library via combination of deep learning and genetic algorithm". J. Comput. Phys. 418, 109584. Paper · arXiv:2001.07305

Code: woshixuhao/DLGA_tutorial (the author's tutorial notebook, on the KdV case), woshixuhao/PIC_code (the author's own code for a later paper, running the same neural-network-plus-genetic-algorithm search)