actgpr package

Submodules

actgpr.acquisition module

Acquisition function module for active GPR optimisation.

Implements the Expected Improvement (EI) acquisition function for selecting the next input point to evaluate.

References

Jones, D. R., Schonlau, M., & Welch, W. J. (1998).: Efficient Global Optimization of Expensive Black-Box Functions. Journal of Global Optimization, 13(4), 455-492. https://doi.org/10.1023/A:1008306431147

class actgpr.acquisition.Acquisition(surrogate, search_bounds, n_candidates=None, seed=25, refinement_stages=None)[source]

Bases: object

Expected Improvement acquisition function for active GPR optimisation.

Stores a reference to the surrogate, search bounds, and candidate count. Scores candidate input points and selects the next input point to evaluate.

Candidates are placed in stages: a coarse set covering the whole search bounds, then one or two refinement sets in a shrinking box around the best point found so far (see refinement_stages). With one input dimension both sets are evenly spaced grids. With more, a grid is unaffordable (500 per axis is 125 million points in 3D), so both sets are drawn from a scrambled Sobol sequence, which covers the space evenly without growing with the number of dimensions.

Public Methods

expected_improvement(f_mean, f_var, current_best): Compute EI scores for an array of candidate points.
find_next_input_point(current_best): Generate candidates, score them, and return the best input point.

expected_improvement(f_mean, f_var, current_best)[source]

Compute the Expected Improvement at candidate points.

Parameters:

f_mean (torch.Tensor of shape (m,)) – Predicted posterior mean at candidate points.
f_var (torch.Tensor of shape (m,)) – Predicted posterior variance at candidate points.
current_best (float) – The smallest objective value observed so far.

Returns:

The EI score for each candidate point.

Return type:

torch.Tensor of shape (m,)

Raises:

ValueError – If f_mean and f_var have different shapes.

find_next_input_point(current_best)[source]

Find the next input point to evaluate by maximising Expected Improvement.

Places candidates across the search bounds, predicts posterior mean and variance using the surrogate, scores them with Expected Improvement, and locates the candidate with the highest score. The coarse candidates’ spacing caps how precisely the true EI maximum can be located, so a finer set is then scored inside a box around the best point, once or twice depending on refinement_stages, and the refined maximum is returned instead.

Parameters:: current_best (float) – The smallest objective value observed so far.
Returns:: The coordinates of the input point with the highest EI score, one per input dimension, refined beyond the coarse candidates’ resolution.
Return type:: tuple of float
Raises:: RuntimeError – If the surrogate has not been fitted prior to calling.

Parameters:

surrogate (GPyTorchSurrogate)
search_bounds (Sequence[float] | Sequence[Sequence[float]])
n_candidates (int | None)
seed (int)
refinement_stages (int | None)

actgpr.mrr module

Minimal Reproducible Run (MRR) file I/O operations.

actgpr.mrr.create_run_dir(base_path, fit_mode, training_iter, ei_threshold, max_iterations, noise, lengthscale, outputscale, n_dims=1)[source]

Create a timestamped run directory with parameters in the name.

Parameters:

base_path (Path) – The root directory where runs are stored (e.g., “results”).
fit_mode (str) – “training” or “notraining”.
training_iter (int | None) – Number of training iterations (if fit_mode is “training”).
ei_threshold (float) – Expected improvement threshold for convergence.
max_iterations (int) – Maximum number of evaluations.
noise (float) – Noise level for the surrogate.
lengthscale (float, sequence of float, or None) – Lengthscale (if fit_mode is “notraining”). A per-dimension sequence is joined with hyphens in the folder name.
outputscale (float | None) – Outputscale (if fit_mode is “notraining”).
n_dims (int, optional) – Number of input dimensions of the run, by default 1. Leads the parameter part of the name, since it is the first thing that distinguishes one run from another.

Returns:

The created run directory path.

Return type:

Path

actgpr.mrr.write_config(run_dir, config)[source]

Write all run parameters to config.json.

Parameters:

run_dir (Path)
config (dict[str, object])

Return type:

None

actgpr.mrr.write_manifest(run_dir)[source]

Compute SHA-256 of config.json and write manifest.json.

Parameters:: run_dir (Path)
Return type:: None

actgpr.mrr.write_meta(run_dir, run_start, run_end, best_x, best_y, n_iterations, stop_reason)[source]

Write environment and output summary to meta.json.

Includes the package name, version, and repository URL so a meta.json file remains identifiable on its own, e.g. if shared or archived separately from this repository.

Parameters:

run_dir (Path)
run_start (datetime)
run_end (datetime)
best_x (Sequence[float])
best_y (float)
n_iterations (int)
stop_reason (str)

Return type:

None

actgpr.mrr.save_hdf5(run_dir, results, config, store_snapshots, final_train_x, final_train_y, best_x, best_y, stop_reason, n_iterations, convergence_snapshot=None, fitted_hyperparameters=None)[source]

Write a self-describing HDF5 file with the run history and results.

Layout

/ (root): Attributes holding the run configuration (n_dims, bounds, thresholds, …).
history/: Per-iteration series, each a dataset with one row per iteration, aligned by the iteration index dataset: new_y, current_best, max_ei, prediction_error, improvement (one value per row), next_point (one column per input dimension), plus lengthscale/outputscale/noise when the surrogate reports them, giving the hyperparameters behind each iteration’s fit (lengthscale has one column per input dimension). Also point_and_output, a heatmap-friendly view putting the evaluated input point and its output in one array, each column scaled to [0, 1] (see its own attributes). This is the single authoritative record of the run’s history. Covers only evaluated iterations. See convergence_snapshot below for the one fit that never reached evaluation.
iterations/iter_NNN/: Written only when store_snapshots is True: the GP snapshot arrays candidates, f_mean, f_var, ei_scores, train_x, train_y for that iteration. Input points (candidates, train_x) have one row per point and one column per input dimension.
final/: Attributes best_x (one value per input dimension), best_y, stop_reason, n_iterations and the final train_x/train_y datasets. When stop_reason is "ei_threshold" and convergence_snapshot is given, also holds the GP/EI state of the fit that triggered convergence, attributes converged_max_ei/converged_next_point and datasets converged_candidates/converged_f_mean/ converged_f_var/converged_ei_scores. That fit’s candidate was never evaluated, so it has no place in history/ or iterations/, so this is the only place it is recorded. When fitted_hyperparameters is given, also holds fitted_lengthscale/fitted_outputscale/fitted_noise, the surrogate’s hyperparameters as the run left them.

param convergence_snapshot:: The GP/EI snapshot of the fit that triggered ei_threshold convergence (OptimisationRun._convergence_snapshot), or None if the run stopped via max_iterations or store_snapshots was False.
type convergence_snapshot:: dict or None, optional
param fitted_hyperparameters:: The surrogate’s final hyperparameters, as returned by its hyperparameters() method. None when the surrogate does not expose one, or was never fitted (a crash before iteration 1). Recording these matters most for with_training runs, where config.json holds only the starting values Adam began from.
type fitted_hyperparameters:: dict or None, optional

Parameters:

run_dir (Path)
results (list[dict[str, object]])
config (dict[str, object])
store_snapshots (bool)
final_train_x (Tensor)
final_train_y (Tensor)
best_x (Sequence[float])
best_y (float)
stop_reason (str)
n_iterations (int)
convergence_snapshot (dict[str, object] | None)
fitted_hyperparameters (dict[str, tuple[float, ...]] | None)

Return type:

None

actgpr.mrr.setup_file_logger(run_dir)[source]

Add a FileHandler to the actgpr logger and return it.

Parameters:: run_dir (Path)
Return type:: FileHandler

actgpr.objective_fn module

Objective function module for active GPR optimisation.

Defines the Objective interface actgpr optimises against, and ObjectiveFn, the convenience wrapper for plain callables.

An Objective is evaluated at one input point per call. The point’s coordinates are passed as separate arguments, one per input dimension, and the call returns one output:

>>> ObjectiveFn(lambda x1, x2: x1 + x2).evaluate(1.0, 2.0)
3.0

Classes

Objective: The interface: any object with evaluate(*x) -> float.
ObjectiveFn: Wraps a plain function of one or more inputs as an Objective.

class actgpr.objective_fn.Objective(*args, **kwargs)[source]

Bases: Protocol

Anything actgpr can minimise: an object with an evaluate method.

This is a typing.Protocol, so conformance is structural: any object providing a matching evaluate satisfies it, with no base class to inherit and no registration step. OptimisationRun never inspects the type of the objective it is given, only calls this method, so wrapping a simulation of your own means writing a class with this one method on it.

ObjectiveFn below satisfies this protocol and is the convenient route for a plain function; it is not the only permitted Objective.

evaluate is called once per input point, with that point’s coordinates as positional arguments, one per input dimension, and must return that point’s single output. A three-input simulation therefore implements def evaluate(self, x1, x2, x3) -> float.

evaluate(*x)[source]

Evaluate the Objective at one input point and return its output.

Parameters:: x (float)
Return type:: float

actgpr.objective_fn.DEFAULT_FUNC(*x)

Evaluate the default Objective: the sum of squared coordinates.

Parameters:: x (float)
Return type:: float

class actgpr.objective_fn.ObjectiveFn(func=None, jitter=0.0, seed=25)[source]

Bases: object

Objective function for active GPR optimisation.

This class represents the real-valued scalar function being optimised. It wraps a plain function taking one argument per input dimension, e.g. def f(x1, x2, x3) -> float. By default it evaluates the sum of squared coordinates, which is f(x) = x² with one input dimension. Optionally adds Gaussian jitter to each evaluation, to simulate the sensor/measurement noise of a real experiment.

Public Methods

evaluate(x1, …, xd): Evaluate the function at one input point with d coordinates.

evaluate(*x)[source]

Evaluate the Objective at one input point.

Parameters:

*x (float) – The coordinates of the input point, one per input dimension. They are passed to func in the same order, so evaluate(x1, x2, x3) calls func(x1, x2, x3).

Returns:

The Objective output at that input point.

Return type:

float

Raises:

ValueError – If no coordinates are provided.
TypeError – If a coordinate cannot be converted to a float, or if func returns a non-numeric value.

Notes

Exceptions raised inside func (e.g. a ValueError from a domain error, or a TypeError because func expects a different number of inputs) propagate unchanged so callers can handle the original error type.

If jitter is non-zero, Gaussian noise with that standard deviation is added to the result after func runs. The wrapped function itself always sees the exact, noise-free input. The noise comes from this ObjectiveFn’s own seeded generator, so repeated calls advance it (the same input evaluated twice gives different noise, as a real sensor would) while two ObjectiveFn objects built with the same seed produce the same sequence.

Examples

>>> ObjectiveFn().evaluate(3.0)
9.0
>>> ObjectiveFn(lambda x1, x2: x1 * x2).evaluate(2.0, 4.0)
8.0

Parameters:

func (Callable[[...], float] | None)
jitter (float)
seed (int)

actgpr.plotting module

Plotting utilities for active GPR optimisation.

Two figures, each reachable two ways. The plot_ prefix draws from a run object you still hold and lives on OptimisationRun; the load_ prefix reads a run back from its log directory and lives here:

the surrogate itself: run.plot_iterations() / load_iterations(dir)
validation metrics: run.plot_metrics() / load_metrics(dir)

Each pair draws the identical figure, so which one to reach for depends solely on what you still have to hand. Everything else in this module is a private helper that these four are built from.

Functions

load_iterations: Reads a run directory and browses its iterations with an interactive slider over the GP fit and the EI landscape, one frame per iteration.
load_metrics: Reads a run directory and plots its validation metrics against iteration, one panel per metric.

actgpr.plotting.load_iterations(run_dir, log_scale=True, show=True)[source]

Browse a saved run’s iterations with the same slider as a live run.

The from-the-logs counterpart to OptimisationRun.plot_iterations(): it reads a run directory’s results.h5 and opens the identical interactive figure, so a run finished last month can be stepped through exactly like one still in memory.

Parameters:

run_dir (Path or str) – The run directory written by OptimisationRun.run().
log_scale (bool, optional) – If True (the default), the EI axis is log-scaled.
show (bool, optional) – If True (the default), calls plt.show().

Returns:

The slider. Assign it to a variable that outlives the call, or matplotlib will garbage collect it and it will stop responding.

Return type:

matplotlib.widgets.Slider

Raises:

FileNotFoundError – If run_dir does not contain a results.h5 file.
RuntimeError – If the run was executed without store_snapshots.
ValueError – If the run has more than one input dimension; its surrogate cannot be drawn as a curve. Use load_metrics for such runs.

actgpr.plotting.load_metrics(run_dir, show=True, log_scale=True)[source]

Plot validation metrics vs. iteration from a saved run’s results.h5.

The from-the-logs counterpart to OptimisationRun.plot_metrics(): it reads the /history series straight from results.h5, with no OptimisationRun object needed, so a past run can be visualised from its run directory alone at any later time. Both open the identical figure.

Parameters:

run_dir (Path or str) – The run directory written by OptimisationRun.run() (the folder containing results.h5, not the file itself).
show (bool, optional) – Whether to call plt.show() immediately, by default True.
log_scale (bool, optional) – If True (the default), the max_ei panel is log-scaled. EI shrinks by orders of magnitude as a run converges, so a linear axis hides the shrinkage that signals convergence.

Returns:

The figure and its 2x2 array of axes, one panel per metric.

Return type:

tuple[Figure, numpy.ndarray]

Raises:

FileNotFoundError – If run_dir does not contain a results.h5 file.

actgpr.run module

Optimisation run module for active GPR optimisation.

Orchestrates the active learning loop: fit surrogate, maximise acquisition function, evaluate objective, repeat until convergence.

class actgpr.run.OptimisationRun(objective, surrogate, search_bounds, initial_train_x, max_iterations, ei_threshold, n_candidates=None, noise=0.0001, store_snapshots=True, run_dir=None, *, _train_hyperparameters=True, _training_iter=50, _lengthscale=1.0, _outputscale=1.0)[source]

Bases: object

Orchestrates the active GPR optimisation loop.

Coordinates the ObjectiveFn, Surrogate, and Acquisition components to iteratively find the minimum of the ObjectiveFn within the search bounds.

The number of input dimensions is set by search_bounds: one (lo, hi) pair per input. Every input point, including each entry of initial_train_x, then has one coordinate per input, and the Objective is called with those coordinates as separate arguments.

The loop terminates when either the maximum EI score falls below ei_threshold (nothing left to gain) or the number of optimisation iterations reaches max_iterations (budget cap), whichever fires first.

Use the classmethods with_training and without_training to construct an OptimisationRun. The raw __init__ is available for advanced use but the classmethods are the preferred entry points.

Public Methods

with_training(): Construct an OptimisationRun that optimises GP hyperparameters each iteration.
without_training(): Construct an OptimisationRun with fixed GP hyperparameters.
run(): Execute the optimisation loop and return the results.
plot_iterations(): Open an interactive matplotlib slider to browse GP snapshots per iteration.
plot_metrics(): Plot this run’s validation metrics against iteration.

classmethod with_training(objective, surrogate, search_bounds, initial_train_x, max_iterations, ei_threshold, n_candidates=None, training_iter=50, noise=0.0001, store_snapshots=True, run_dir=None)[source]

Construct an OptimisationRun that optimises GP hyperparameters.

Each iteration fits the surrogate and optimises the kernel lengthscale, outputscale, and noise variance using Adam.

Parameters:

objective (Objective) – The Objective to minimise: any object exposing evaluate(*x) -> float, called once per input point with that point’s coordinates. Its type is never checked, only that method is called, so a class of your own wrapping a simulation works as well as an ObjectiveFn.
surrogate (GPyTorchSurrogate) – The GP surrogate model used to approximate the objective.
search_bounds (sequence of (lo, hi) pairs, or a single (lo, hi) pair) – The closed interval of each input dimension, in order, e.g. [(-3.0, 3.0), (0.0, 1.0)] for two inputs. The number of pairs is the number of input dimensions. A single (lo, hi) pair is shorthand for one input dimension.
initial_train_x (array-like of shape (n, d), or (n,) when d is 1) – The initial input points to seed the optimisation loop, one row per point with one coordinate per input dimension, e.g. [[0.0, 0.5], [1.0, 0.2]] for two 2D points. With a single input dimension a flat list such as [-3.0, 5.0] is accepted.
max_iterations (int) – Maximum number of active optimisation iterations, meaning GPR fit cycles, to execute (budget cap).
ei_threshold (float) – The loop stops when the maximum EI score falls below this value.
n_candidates (int or None, optional) – Number of candidate points the acquisition function scores in each stage. If None (the default), 500 for a single input and 4000 for more, since the same count spread over several inputs thins out quickly. Raising it further mainly improves the worst case rather than the typical one.
training_iter (int, optional) – Number of hyperparameter optimisation iterations per surrogate fit, by default 50.
noise (float, optional) – Initial observation noise variance for the GP likelihood, by default 1e-4.
store_snapshots (bool, optional) – If True, also stores GP snapshots for interactive plotting via plot_iterations(), by default True. Pass False to skip them (they are the bulk of results.h5’s size). The prediction_error and improvement history used by plotting.load_metrics() is recorded either way, regardless of this flag.
run_dir (Path | str | None)

Returns:

A configured OptimisationRun that will train hyperparameters.

Return type:

OptimisationRun

classmethod without_training(objective, surrogate, search_bounds, initial_train_x, max_iterations, ei_threshold, n_candidates=None, lengthscale=1.0, outputscale=1.0, noise=0.0001, store_snapshots=True, run_dir=None)[source]

Construct an OptimisationRun with fixed GP hyperparameters.

Each iteration fits the surrogate with the given lengthscale, outputscale, and noise, so no hyperparameter optimisation takes place.

Parameters:

objective (Objective) – The Objective to minimise: any object exposing evaluate(*x) -> float, called once per input point with that point’s coordinates. Its type is never checked, only that method is called, so a class of your own wrapping a simulation works as well as an ObjectiveFn.
surrogate (GPyTorchSurrogate) – The GP surrogate model used to approximate the objective.
search_bounds (sequence of (lo, hi) pairs, or a single (lo, hi) pair) – The closed interval of each input dimension, in order, e.g. [(-3.0, 3.0), (0.0, 1.0)] for two inputs. The number of pairs is the number of input dimensions. A single (lo, hi) pair is shorthand for one input dimension.
initial_train_x (array-like of shape (n, d), or (n,) when d is 1) – The initial input points to seed the optimisation loop, one row per point with one coordinate per input dimension, e.g. [[0.0, 0.5], [1.0, 0.2]] for two 2D points. With a single input dimension a flat list such as [-3.0, 5.0] is accepted.
max_iterations (int) – Maximum number of active optimisation iterations, meaning GPR fit cycles, to execute (budget cap).
ei_threshold (float) – The loop stops when the maximum EI score falls below this value.
n_candidates (int or None, optional) – Number of candidate points the acquisition function scores in each stage. If None (the default), 500 for a single input and 4000 for more, since the same count spread over several inputs thins out quickly. Raising it further mainly improves the worst case rather than the typical one.
lengthscale (float or sequence of float, optional) – The RBF kernel lengthscale, by default 1.0. A single value is used for every input dimension; a sequence gives one value per input dimension, in order.
outputscale (float, optional) – The kernel outputscale (signal variance), by default 1.0.
noise (float, optional) – The observation noise variance, by default 1e-4.
store_snapshots (bool, optional) – If True, also stores GP snapshots for interactive plotting via plot_iterations(), by default True. Pass False to skip them (they are the bulk of results.h5’s size). The prediction_error and improvement history used by plotting.load_metrics() is recorded either way, regardless of this flag.
run_dir (Path | str | None)

Returns:

A configured OptimisationRun with fixed hyperparameters.

Return type:

OptimisationRun

run()[source]

Execute the optimisation loop.

Iteratively fits the surrogate, finds the next input point via the acquisition function, and evaluates the objective until convergence.

Returns:

A dictionary containing the optimisation results:

”best_x”: tuple of float, the input point with the lowest Objective value, one coordinate per input dimension (a 1-tuple for a single input).
”best_y”: float, the lowest Objective value found.
”train_x”: torch.Tensor of shape (n, d), all evaluated input points, one row each.
”train_y”: torch.Tensor, all Objective outputs.
”n_iterations”: int, number of loop iterations executed.
”stop_reason”: str, “ei_threshold” if EI dropped below ei_threshold, “max_iterations” if budget cap was reached.

Return type:

dict

Raises:

Exception – Any exception raised while fitting the surrogate or evaluating the objective propagates unchanged. If run_dir was given, a best-effort results.h5/meta.json checkpoint covering every iteration completed before the failure is written first (with stop_reason="crashed"), so a mid-run crash only loses the incomplete iteration rather than the whole run.

plot_metrics(show=True, log_scale=True)[source]

Plot this run’s validation metrics against iteration.

The from-the-run counterpart to plotting.load_metrics, which reads the same series back from a saved results.h5. Both draw the identical figure, one panel per metric. Use this one while the run object is still to hand, including for a run that wrote no MRR record and therefore has no directory to read back.

Parameters:

show (bool, optional) – Whether to call plt.show() immediately, by default True.
log_scale (bool, optional) – If True (the default), the max_ei panel is log-scaled.

Returns:

The figure and its 2x2 array of axes, one panel per metric.

Return type:

tuple[Figure, numpy.ndarray]

Raises:

RuntimeError – If the run has not been executed yet, so there is no history.

plot_iterations(show=True, log_scale=True)[source]

Open an interactive matplotlib figure to browse iterations.

Creates a figure with two subplots (GP predictions on top, EI landscape on bottom) and a slider to scrub through iterations.

The from-the-run counterpart to plotting.load_iterations, which opens the identical figure for a run read back from its logs.

The Slider is kept alive via self._active_slider for as long as the OptimisationRun exists. Matplotlib does not keep its own strong reference to a Slider. If the only reference were a local variable here, it would be garbage collected as soon as this method returns, which happens immediately whenever plt.show() does not block (backend- and environment-dependent). The slider would still be drawn, but would silently stop responding to drags.

Parameters:

show (bool, optional) – Whether to call plt.show() immediately, by default True. Pass False when opening this alongside another figure: plt.show() displays every open figure, so calling it once per figure re-displays the earlier ones. Build both, then call plt.show() once yourself.
log_scale (bool, optional) – If True, draws the EI subplot’s y-axis on a log scale, with the ei_threshold convergence criterion marked as a reference line. EI often shrinks by orders of magnitude as a run converges, which a linear axis compresses into an invisible flat line, so log scale keeps that shrinkage visible, so it is the default. Pass False for a linear EI axis.

Return type:

None

Notes

If the run converged via ei_threshold, the final frame is the fit that triggered convergence. Its next_point was scored but never evaluated, shown with a title noting “(converged, not evaluated)” instead of the usual pred_error/improvement values.

Raises:

RuntimeError – If the run has not been executed yet, or if it was executed with store_snapshots=False so no snapshots were recorded.
ValueError – If the run has more than one input dimension; its surrogate cannot be drawn as a curve. Use plot_metrics() for such runs.

Parameters:

show (bool)
log_scale (bool)

Return type:

None

Parameters:

objective (Objective)
surrogate (GPyTorchSurrogate)
search_bounds (Sequence[float] | Sequence[Sequence[float]])
initial_train_x (torch.Tensor | Sequence[float] | Sequence[Sequence[float]])
max_iterations (int)
ei_threshold (float)
n_candidates (int | None)
noise (float)
store_snapshots (bool)
run_dir (Path | str | None)
_train_hyperparameters (bool)
_training_iter (int)
_lengthscale (float | Sequence[float])
_outputscale (float)

actgpr.surrogate module

Surrogate model module for active GPR optimisation.

class actgpr.surrogate.ExactGPModel(train_x, train_y, likelihood)[source]

Bases: ExactGP

An exact Gaussian Process model with Constant mean and scaled RBF kernel.

This class defines the structural prior components (mean and covariance) of the GP model. The RBF kernel has one lengthscale per input dimension (automatic relevance determination), since inputs of a multi-dimensional problem generally vary on different scales.

Parameters:

train_x (Tensor)
train_y (Tensor)
likelihood (GaussianLikelihood)

forward(x)[source]

Compute the prior distribution at input points x.

Parameters:: x (torch.Tensor of shape (m, d)) – The input points to evaluate the prior mean and covariance at.
Returns:: The prior multivariate normal distribution.
Return type:: gpytorch.distributions.MultivariateNormal

class actgpr.surrogate.GPyTorchSurrogate[source]

Bases: object

A Gaussian Process surrogate model backend wrapper using GPyTorch.

This class packages model definition, parameter optimization, and posterior prediction, hiding GPyTorch-specific API details from external callers.

fit_and_train(train_x, train_y, training_iter=50, lr=0.1, noise=0.0001)[source]

Fit the GP model and optimise hyperparameters automatically.

Optimises the kernel lengthscale, outputscale, and noise variance using PyTorch’s Adam optimiser.

Parameters:

train_x (torch.Tensor of shape (n,) or (n, d)) – The input points where the Objective was evaluated, one row per point.
train_y (torch.Tensor of shape (n,)) – The corresponding Objective outputs.
training_iter (int, optional) – Number of iterations for hyperparameter optimisation, by default 50.
lr (float, optional) – Learning rate for the optimiser, by default 0.1.
noise (float, optional) – Initial observation noise variance for the likelihood, by default 1e-4.

Return type:

None

fit_no_training(train_x, train_y, lengthscale=1.0, outputscale=1.0, noise=0.0001)[source]

Fit the GP model with user-specified hyperparameters (no training).

Sets the kernel lengthscale, outputscale, and noise to the given values and freezes all parameters so no optimisation takes place.

Parameters:

train_x (torch.Tensor of shape (n,) or (n, d)) – The input points where the Objective was evaluated, one row per point.
train_y (torch.Tensor of shape (n,)) – The corresponding Objective outputs.
lengthscale (float or sequence of float, optional) – The RBF kernel lengthscale, by default 1.0. A single value is used for every input dimension; a sequence gives one value per input dimension, in order.
outputscale (float, optional) – The kernel outputscale (signal variance), by default 1.0.
noise (float, optional) – The observation noise variance, by default 1e-4.

Raises:

ValueError – If lengthscale is a sequence whose length is not the number of input dimensions.

Return type:

None

hyperparameters()[source]

Return the GP’s current kernel and likelihood hyperparameters.

After fit_no_training these are the values that were passed in. After fit_and_train they are the values Adam arrived at, which are otherwise not recoverable: config.json is written before the loop starts and can only record the starting point.

Returns:: lengthscale with one value per input dimension, and outputscale and noise with one value each. Every entry is a tuple so that callers can record and render all three alike.
Return type:: dict[str, tuple of float]
Raises:: RuntimeError – If the model has not been fitted yet.

predict(test_x, n_samples=0)[source]

Predict the posterior distributions at test points.

Parameters:

test_x (torch.Tensor of shape (m, d), or (m,) when d is 1) – The input points to predict at, with as many coordinates per point as the training data.
n_samples (int, optional) – Number of samples to draw from the latent function’s predictive posterior, by default 0. Sampling is skipped entirely when 0, which keeps predictions cheap inside the optimisation loop.

Returns:

A dictionary containing prediction components:

”f_preds”: predictive distribution (MultivariateNormal) of the latent function f(test_x). Its full covariance matrix is available on demand as f_preds.covariance_matrix, shape (m, m). It is not computed eagerly, since it grows with m² and the optimisation loop needs only the diagonal.
”observed_pred”: predictive distribution (MultivariateNormal) of observed targets y(test_x) = f(test_x) + noise.
”f_mean”: predicted posterior mean of the latent function, torch.Tensor of shape (m,).
”f_var”: predicted posterior variance of the latent function, torch.Tensor of shape (m,).
”f_samples”: samples drawn from the latent function’s predictive posterior, torch.Tensor of shape (n_samples, m). Only present when n_samples > 0.

Return type:

dict[str, torch.Tensor | gpytorch.distributions.MultivariateNormal]

Raises:

RuntimeError – If fit_and_train() or fit_no_training() has not been called prior to predicting.
ValueError – If test_x does not have one coordinate per input dimension.

Module contents

Active GPR (Gaussian Process Regression) Optimisation package.

This package finds the minimum of an expensive-to-evaluate objective function with one or more inputs and a single scalar output, by iteratively fitting a Gaussian Process surrogate and using active learning. The number of inputs is set by search_bounds, one (lo, hi) pair per input.

Exported classes

OptimisationRun: Orchestrates the active optimisation loop and MRR artifact writes.
ObjectiveFn: Wraps a plain function of one or more inputs as the Objective.
GPyTorchSurrogate: Gaussian Process surrogate backend built on GPyTorch.
Acquisition: Expected Improvement acquisition function.

class actgpr.Acquisition(surrogate, search_bounds, n_candidates=None, seed=25, refinement_stages=None)[source]

Bases: object

Expected Improvement acquisition function for active GPR optimisation.

Stores a reference to the surrogate, search bounds, and candidate count. Scores candidate input points and selects the next input point to evaluate.

Candidates are placed in stages: a coarse set covering the whole search bounds, then one or two refinement sets in a shrinking box around the best point found so far (see refinement_stages). With one input dimension both sets are evenly spaced grids. With more, a grid is unaffordable (500 per axis is 125 million points in 3D), so both sets are drawn from a scrambled Sobol sequence, which covers the space evenly without growing with the number of dimensions.

Public Methods

expected_improvement(f_mean, f_var, current_best): Compute EI scores for an array of candidate points.
find_next_input_point(current_best): Generate candidates, score them, and return the best input point.

expected_improvement(f_mean, f_var, current_best)[source]

Compute the Expected Improvement at candidate points.

Parameters:

f_mean (torch.Tensor of shape (m,)) – Predicted posterior mean at candidate points.
f_var (torch.Tensor of shape (m,)) – Predicted posterior variance at candidate points.
current_best (float) – The smallest objective value observed so far.

Returns:

The EI score for each candidate point.

Return type:

torch.Tensor of shape (m,)

Raises:

ValueError – If f_mean and f_var have different shapes.

find_next_input_point(current_best)[source]

Find the next input point to evaluate by maximising Expected Improvement.

Places candidates across the search bounds, predicts posterior mean and variance using the surrogate, scores them with Expected Improvement, and locates the candidate with the highest score. The coarse candidates’ spacing caps how precisely the true EI maximum can be located, so a finer set is then scored inside a box around the best point, once or twice depending on refinement_stages, and the refined maximum is returned instead.

Parameters:: current_best (float) – The smallest objective value observed so far.
Returns:: The coordinates of the input point with the highest EI score, one per input dimension, refined beyond the coarse candidates’ resolution.
Return type:: tuple of float
Raises:: RuntimeError – If the surrogate has not been fitted prior to calling.

Parameters:

surrogate (GPyTorchSurrogate)
search_bounds (Sequence[float] | Sequence[Sequence[float]])
n_candidates (int | None)
seed (int)
refinement_stages (int | None)

class actgpr.ObjectiveFn(func=None, jitter=0.0, seed=25)[source]

Bases: object

Objective function for active GPR optimisation.

This class represents the real-valued scalar function being optimised. It wraps a plain function taking one argument per input dimension, e.g. def f(x1, x2, x3) -> float. By default it evaluates the sum of squared coordinates, which is f(x) = x² with one input dimension. Optionally adds Gaussian jitter to each evaluation, to simulate the sensor/measurement noise of a real experiment.

Public Methods

evaluate(x1, …, xd): Evaluate the function at one input point with d coordinates.

evaluate(*x)[source]

Evaluate the Objective at one input point.

Parameters:

*x (float) – The coordinates of the input point, one per input dimension. They are passed to func in the same order, so evaluate(x1, x2, x3) calls func(x1, x2, x3).

Returns:

The Objective output at that input point.

Return type:

float

Raises:

ValueError – If no coordinates are provided.
TypeError – If a coordinate cannot be converted to a float, or if func returns a non-numeric value.

Notes

Exceptions raised inside func (e.g. a ValueError from a domain error, or a TypeError because func expects a different number of inputs) propagate unchanged so callers can handle the original error type.

If jitter is non-zero, Gaussian noise with that standard deviation is added to the result after func runs. The wrapped function itself always sees the exact, noise-free input. The noise comes from this ObjectiveFn’s own seeded generator, so repeated calls advance it (the same input evaluated twice gives different noise, as a real sensor would) while two ObjectiveFn objects built with the same seed produce the same sequence.

Examples

>>> ObjectiveFn().evaluate(3.0)
9.0
>>> ObjectiveFn(lambda x1, x2: x1 * x2).evaluate(2.0, 4.0)
8.0

Parameters:

func (Callable[[...], float] | None)
jitter (float)
seed (int)

class actgpr.OptimisationRun(objective, surrogate, search_bounds, initial_train_x, max_iterations, ei_threshold, n_candidates=None, noise=0.0001, store_snapshots=True, run_dir=None, *, _train_hyperparameters=True, _training_iter=50, _lengthscale=1.0, _outputscale=1.0)[source]

Bases: object

Orchestrates the active GPR optimisation loop.

Coordinates the ObjectiveFn, Surrogate, and Acquisition components to iteratively find the minimum of the ObjectiveFn within the search bounds.

The number of input dimensions is set by search_bounds: one (lo, hi) pair per input. Every input point, including each entry of initial_train_x, then has one coordinate per input, and the Objective is called with those coordinates as separate arguments.

The loop terminates when either the maximum EI score falls below ei_threshold (nothing left to gain) or the number of optimisation iterations reaches max_iterations (budget cap), whichever fires first.

Use the classmethods with_training and without_training to construct an OptimisationRun. The raw __init__ is available for advanced use but the classmethods are the preferred entry points.

Public Methods

with_training(): Construct an OptimisationRun that optimises GP hyperparameters each iteration.
without_training(): Construct an OptimisationRun with fixed GP hyperparameters.
run(): Execute the optimisation loop and return the results.
plot_iterations(): Open an interactive matplotlib slider to browse GP snapshots per iteration.
plot_metrics(): Plot this run’s validation metrics against iteration.

classmethod with_training(objective, surrogate, search_bounds, initial_train_x, max_iterations, ei_threshold, n_candidates=None, training_iter=50, noise=0.0001, store_snapshots=True, run_dir=None)[source]

Construct an OptimisationRun that optimises GP hyperparameters.

Each iteration fits the surrogate and optimises the kernel lengthscale, outputscale, and noise variance using Adam.

Parameters:

objective (Objective) – The Objective to minimise: any object exposing evaluate(*x) -> float, called once per input point with that point’s coordinates. Its type is never checked, only that method is called, so a class of your own wrapping a simulation works as well as an ObjectiveFn.
surrogate (GPyTorchSurrogate) – The GP surrogate model used to approximate the objective.
search_bounds (sequence of (lo, hi) pairs, or a single (lo, hi) pair) – The closed interval of each input dimension, in order, e.g. [(-3.0, 3.0), (0.0, 1.0)] for two inputs. The number of pairs is the number of input dimensions. A single (lo, hi) pair is shorthand for one input dimension.
initial_train_x (array-like of shape (n, d), or (n,) when d is 1) – The initial input points to seed the optimisation loop, one row per point with one coordinate per input dimension, e.g. [[0.0, 0.5], [1.0, 0.2]] for two 2D points. With a single input dimension a flat list such as [-3.0, 5.0] is accepted.
max_iterations (int) – Maximum number of active optimisation iterations, meaning GPR fit cycles, to execute (budget cap).
ei_threshold (float) – The loop stops when the maximum EI score falls below this value.
n_candidates (int or None, optional) – Number of candidate points the acquisition function scores in each stage. If None (the default), 500 for a single input and 4000 for more, since the same count spread over several inputs thins out quickly. Raising it further mainly improves the worst case rather than the typical one.
training_iter (int, optional) – Number of hyperparameter optimisation iterations per surrogate fit, by default 50.
noise (float, optional) – Initial observation noise variance for the GP likelihood, by default 1e-4.
store_snapshots (bool, optional) – If True, also stores GP snapshots for interactive plotting via plot_iterations(), by default True. Pass False to skip them (they are the bulk of results.h5’s size). The prediction_error and improvement history used by plotting.load_metrics() is recorded either way, regardless of this flag.
run_dir (Path | str | None)

Returns:

A configured OptimisationRun that will train hyperparameters.

Return type:

OptimisationRun

classmethod without_training(objective, surrogate, search_bounds, initial_train_x, max_iterations, ei_threshold, n_candidates=None, lengthscale=1.0, outputscale=1.0, noise=0.0001, store_snapshots=True, run_dir=None)[source]

Construct an OptimisationRun with fixed GP hyperparameters.

Each iteration fits the surrogate with the given lengthscale, outputscale, and noise, so no hyperparameter optimisation takes place.

Parameters:

objective (Objective) – The Objective to minimise: any object exposing evaluate(*x) -> float, called once per input point with that point’s coordinates. Its type is never checked, only that method is called, so a class of your own wrapping a simulation works as well as an ObjectiveFn.
surrogate (GPyTorchSurrogate) – The GP surrogate model used to approximate the objective.
search_bounds (sequence of (lo, hi) pairs, or a single (lo, hi) pair) – The closed interval of each input dimension, in order, e.g. [(-3.0, 3.0), (0.0, 1.0)] for two inputs. The number of pairs is the number of input dimensions. A single (lo, hi) pair is shorthand for one input dimension.
initial_train_x (array-like of shape (n, d), or (n,) when d is 1) – The initial input points to seed the optimisation loop, one row per point with one coordinate per input dimension, e.g. [[0.0, 0.5], [1.0, 0.2]] for two 2D points. With a single input dimension a flat list such as [-3.0, 5.0] is accepted.
max_iterations (int) – Maximum number of active optimisation iterations, meaning GPR fit cycles, to execute (budget cap).
ei_threshold (float) – The loop stops when the maximum EI score falls below this value.
n_candidates (int or None, optional) – Number of candidate points the acquisition function scores in each stage. If None (the default), 500 for a single input and 4000 for more, since the same count spread over several inputs thins out quickly. Raising it further mainly improves the worst case rather than the typical one.
lengthscale (float or sequence of float, optional) – The RBF kernel lengthscale, by default 1.0. A single value is used for every input dimension; a sequence gives one value per input dimension, in order.
outputscale (float, optional) – The kernel outputscale (signal variance), by default 1.0.
noise (float, optional) – The observation noise variance, by default 1e-4.
store_snapshots (bool, optional) – If True, also stores GP snapshots for interactive plotting via plot_iterations(), by default True. Pass False to skip them (they are the bulk of results.h5’s size). The prediction_error and improvement history used by plotting.load_metrics() is recorded either way, regardless of this flag.
run_dir (Path | str | None)

Returns:

A configured OptimisationRun with fixed hyperparameters.

Return type:

OptimisationRun

run()[source]

Execute the optimisation loop.

Iteratively fits the surrogate, finds the next input point via the acquisition function, and evaluates the objective until convergence.

Returns:

A dictionary containing the optimisation results:

”best_x”: tuple of float, the input point with the lowest Objective value, one coordinate per input dimension (a 1-tuple for a single input).
”best_y”: float, the lowest Objective value found.
”train_x”: torch.Tensor of shape (n, d), all evaluated input points, one row each.
”train_y”: torch.Tensor, all Objective outputs.
”n_iterations”: int, number of loop iterations executed.
”stop_reason”: str, “ei_threshold” if EI dropped below ei_threshold, “max_iterations” if budget cap was reached.

Return type:

dict

Raises:

Exception – Any exception raised while fitting the surrogate or evaluating the objective propagates unchanged. If run_dir was given, a best-effort results.h5/meta.json checkpoint covering every iteration completed before the failure is written first (with stop_reason="crashed"), so a mid-run crash only loses the incomplete iteration rather than the whole run.

plot_metrics(show=True, log_scale=True)[source]

Plot this run’s validation metrics against iteration.

The from-the-run counterpart to plotting.load_metrics, which reads the same series back from a saved results.h5. Both draw the identical figure, one panel per metric. Use this one while the run object is still to hand, including for a run that wrote no MRR record and therefore has no directory to read back.

Parameters:

show (bool, optional) – Whether to call plt.show() immediately, by default True.
log_scale (bool, optional) – If True (the default), the max_ei panel is log-scaled.

Returns:

The figure and its 2x2 array of axes, one panel per metric.

Return type:

tuple[Figure, numpy.ndarray]

Raises:

RuntimeError – If the run has not been executed yet, so there is no history.

plot_iterations(show=True, log_scale=True)[source]

Open an interactive matplotlib figure to browse iterations.

Creates a figure with two subplots (GP predictions on top, EI landscape on bottom) and a slider to scrub through iterations.

The from-the-run counterpart to plotting.load_iterations, which opens the identical figure for a run read back from its logs.

The Slider is kept alive via self._active_slider for as long as the OptimisationRun exists. Matplotlib does not keep its own strong reference to a Slider. If the only reference were a local variable here, it would be garbage collected as soon as this method returns, which happens immediately whenever plt.show() does not block (backend- and environment-dependent). The slider would still be drawn, but would silently stop responding to drags.

Parameters:

show (bool, optional) – Whether to call plt.show() immediately, by default True. Pass False when opening this alongside another figure: plt.show() displays every open figure, so calling it once per figure re-displays the earlier ones. Build both, then call plt.show() once yourself.
log_scale (bool, optional) – If True, draws the EI subplot’s y-axis on a log scale, with the ei_threshold convergence criterion marked as a reference line. EI often shrinks by orders of magnitude as a run converges, which a linear axis compresses into an invisible flat line, so log scale keeps that shrinkage visible, so it is the default. Pass False for a linear EI axis.

Return type:

None

Notes

If the run converged via ei_threshold, the final frame is the fit that triggered convergence. Its next_point was scored but never evaluated, shown with a title noting “(converged, not evaluated)” instead of the usual pred_error/improvement values.

Raises:

RuntimeError – If the run has not been executed yet, or if it was executed with store_snapshots=False so no snapshots were recorded.
ValueError – If the run has more than one input dimension; its surrogate cannot be drawn as a curve. Use plot_metrics() for such runs.

Parameters:

show (bool)
log_scale (bool)

Return type:

None

Parameters:

objective (Objective)
surrogate (GPyTorchSurrogate)
search_bounds (Sequence[float] | Sequence[Sequence[float]])
initial_train_x (torch.Tensor | Sequence[float] | Sequence[Sequence[float]])
max_iterations (int)
ei_threshold (float)
n_candidates (int | None)
noise (float)
store_snapshots (bool)
run_dir (Path | str | None)
_train_hyperparameters (bool)
_training_iter (int)
_lengthscale (float | Sequence[float])
_outputscale (float)

class actgpr.GPyTorchSurrogate[source]

Bases: object

A Gaussian Process surrogate model backend wrapper using GPyTorch.

This class packages model definition, parameter optimization, and posterior prediction, hiding GPyTorch-specific API details from external callers.

fit_and_train(train_x, train_y, training_iter=50, lr=0.1, noise=0.0001)[source]

Fit the GP model and optimise hyperparameters automatically.

Optimises the kernel lengthscale, outputscale, and noise variance using PyTorch’s Adam optimiser.

Parameters:

train_x (torch.Tensor of shape (n,) or (n, d)) – The input points where the Objective was evaluated, one row per point.
train_y (torch.Tensor of shape (n,)) – The corresponding Objective outputs.
training_iter (int, optional) – Number of iterations for hyperparameter optimisation, by default 50.
lr (float, optional) – Learning rate for the optimiser, by default 0.1.
noise (float, optional) – Initial observation noise variance for the likelihood, by default 1e-4.

Return type:

None

fit_no_training(train_x, train_y, lengthscale=1.0, outputscale=1.0, noise=0.0001)[source]

Fit the GP model with user-specified hyperparameters (no training).

Sets the kernel lengthscale, outputscale, and noise to the given values and freezes all parameters so no optimisation takes place.

Parameters:

train_x (torch.Tensor of shape (n,) or (n, d)) – The input points where the Objective was evaluated, one row per point.
train_y (torch.Tensor of shape (n,)) – The corresponding Objective outputs.
lengthscale (float or sequence of float, optional) – The RBF kernel lengthscale, by default 1.0. A single value is used for every input dimension; a sequence gives one value per input dimension, in order.
outputscale (float, optional) – The kernel outputscale (signal variance), by default 1.0.
noise (float, optional) – The observation noise variance, by default 1e-4.

Raises:

ValueError – If lengthscale is a sequence whose length is not the number of input dimensions.

Return type:

None

hyperparameters()[source]

Return the GP’s current kernel and likelihood hyperparameters.

After fit_no_training these are the values that were passed in. After fit_and_train they are the values Adam arrived at, which are otherwise not recoverable: config.json is written before the loop starts and can only record the starting point.

Returns:: lengthscale with one value per input dimension, and outputscale and noise with one value each. Every entry is a tuple so that callers can record and render all three alike.
Return type:: dict[str, tuple of float]
Raises:: RuntimeError – If the model has not been fitted yet.

predict(test_x, n_samples=0)[source]

Predict the posterior distributions at test points.

Parameters:

test_x (torch.Tensor of shape (m, d), or (m,) when d is 1) – The input points to predict at, with as many coordinates per point as the training data.
n_samples (int, optional) – Number of samples to draw from the latent function’s predictive posterior, by default 0. Sampling is skipped entirely when 0, which keeps predictions cheap inside the optimisation loop.

Returns:

A dictionary containing prediction components:

”f_preds”: predictive distribution (MultivariateNormal) of the latent function f(test_x). Its full covariance matrix is available on demand as f_preds.covariance_matrix, shape (m, m). It is not computed eagerly, since it grows with m² and the optimisation loop needs only the diagonal.
”observed_pred”: predictive distribution (MultivariateNormal) of observed targets y(test_x) = f(test_x) + noise.
”f_mean”: predicted posterior mean of the latent function, torch.Tensor of shape (m,).
”f_var”: predicted posterior variance of the latent function, torch.Tensor of shape (m,).
”f_samples”: samples drawn from the latent function’s predictive posterior, torch.Tensor of shape (n_samples, m). Only present when n_samples > 0.

Return type:

dict[str, torch.Tensor | gpytorch.distributions.MultivariateNormal]

Raises:

RuntimeError – If fit_and_train() or fit_no_training() has not been called prior to predicting.
ValueError – If test_x does not have one coordinate per input dimension.