pymor.reductors.neural_network
¶
Remark on the documentation:
Due to an issue in autoapi, the classes NeuralNetworkStatefreeOutputReductor
,
NeuralNetworkInstationaryReductor
, NeuralNetworkInstationaryStatefreeOutputReductor
,
EarlyStoppingScheduler
and CustomDataset
do not appear in the documentation,
see https://github.com/pymor/pymor/issues/1343.
Module Contents¶
Classes¶
Reduced Basis reductor relying on artificial neural networks. 

Output reductor relying on artificial neural networks. 

Reduced Basis reductor for instationary problems relying on artificial neural networks. 

Output reductor relying on artificial neural networks. 

Class for performing early stopping in training of neural networks. 

Class that represents the dataset to use in PyTorch. 
Functions¶
Training algorithm for artificial neural networks. 

Algorithm that performs multiple restarts of neural network training. 
 class pymor.reductors.neural_network.NeuralNetworkReductor(fom, training_set, validation_set=None, validation_ratio=0.1, basis_size=None, rtol=0.0, atol=0.0, l2_err=0.0, pod_params=None, ann_mse='like_basis')[source]¶
Bases:
pymor.core.base.BasicObject
Reduced Basis reductor relying on artificial neural networks.
This is a reductor that constructs a reduced basis using proper orthogonal decomposition and trains a neural network that approximates the mapping from parameter space to coefficients of the fullorder solution in the reduced basis. The approach is described in [HU18].
Parameters
 fom
The fullorder
Model
to reduce. training_set
Set of
parameter values
to use for POD and training of the neural network. validation_set
Set of
parameter values
to use for validation in the training of the neural network. validation_ratio
Fraction of the training set to use for validation in the training of the neural network (only used if no validation set is provided).
 basis_size
Desired size of the reduced basis. If
None
, rtol, atol or l2_err must be provided. rtol
Relative tolerance the basis should guarantee on the training set.
 atol
Absolute tolerance the basis should guarantee on the training set.
 l2_err
L2approximation error the basis should not exceed on the training set.
 pod_params
Dict of additional parameters for the PODmethod.
 ann_mse
If
'like_basis'
, the mean squared error of the neural network on the training set should not exceed the error of projecting onto the basis. IfNone
, the neural network with smallest validation error is used to build the ROM. If a tolerance is prescribed, the mean squared error of the neural network on the training set should not exceed this threshold. Training is interrupted if a neural network that undercuts the error tolerance is found.
 reduce(self, hidden_layers='[(N+P)*3, (N+P)*3]', activation_function=torch.tanh, optimizer=optim.LBFGS, epochs=1000, batch_size=20, learning_rate=1.0, restarts=10, seed=0)[source]¶
Reduce by training artificial neural networks.
Parameters
 hidden_layers
Number of neurons in the hidden layers. Can either be fixed or a Python expression string depending on the reduced basis size respectively output dimension
N
and the total dimension of theParameters
P
. activation_function
Activation function to use between the hidden layers.
 optimizer
Algorithm to use as optimizer during training.
 epochs
Maximum number of epochs for training.
 batch_size
Batch size to use if optimizer allows minibatching.
 learning_rate
Step size to use in each optimization step.
 restarts
Number of restarts of the training algorithm. Since the training results highly depend on the initial starting point, i.e. the initial weights and biases, it is advisable to train multiple neural networks by starting with different initial values and choose that one performing best on the validation set.
 seed
Seed to use for various functions in PyTorch. Using a fixed seed, it is possible to reproduce former results.
Returns
 rom
Reducedorder
NeuralNetworkModel
.
 _compute_sample(self, mu, u=None)[source]¶
Transform parameter and corresponding solution to
NumPy arrays
.
 _compute_layer_sizes(self, hidden_layers)[source]¶
Compute the number of neurons in the layers of the neural network.
 class pymor.reductors.neural_network.NeuralNetworkStatefreeOutputReductor(fom, training_set, validation_set=None, validation_ratio=0.1, validation_loss=None)[source]¶
Bases:
NeuralNetworkReductor
Output reductor relying on artificial neural networks.
This is a reductor that trains a neural network that approximates the mapping from parameter space to output space.
Parameters
 fom
The fullorder
Model
to reduce. training_set
Set of
parameter values
to use for POD and training of the neural network. validation_set
Set of
parameter values
to use for validation in the training of the neural network. validation_ratio
Fraction of the training set to use for validation in the training of the neural network (only used if no validation set is provided).
 validation_loss
The validation loss to reach during training. If
None
, the neural network with the smallest validation loss is returned.
 compute_training_data(self)[source]¶
Compute the training samples (the outputs to the parameters of the training set).
 _compute_layer_sizes(self, hidden_layers)[source]¶
Compute the number of neurons in the layers of the neural network.
 class pymor.reductors.neural_network.NeuralNetworkInstationaryReductor(fom, training_set, validation_set=None, validation_ratio=0.1, basis_size=None, rtol=0.0, atol=0.0, l2_err=0.0, pod_params=None, ann_mse='like_basis')[source]¶
Bases:
NeuralNetworkReductor
Reduced Basis reductor for instationary problems relying on artificial neural networks.
This is a reductor that constructs a reduced basis using proper orthogonal decomposition and trains a neural network that approximates the mapping from parameter and time space to coefficients of the fullorder solution in the reduced basis. The approach is described in [WHR19].
Parameters
 fom
The fullorder
Model
to reduce. training_set
Set of
parameter values
to use for POD and training of the neural network. validation_set
Set of
parameter values
to use for validation in the training of the neural network. validation_ratio
Fraction of the training set to use for validation in the training of the neural network (only used if no validation set is provided).
 basis_size
Desired size of the reduced basis. If
None
, rtol, atol or l2_err must be provided. rtol
Relative tolerance the basis should guarantee on the training set.
 atol
Absolute tolerance the basis should guarantee on the training set.
 l2_err
L2approximation error the basis should not exceed on the training set.
 pod_params
Dict of additional parameters for the PODmethod.
 ann_mse
If
'like_basis'
, the mean squared error of the neural network on the training set should not exceed the error of projecting onto the basis. IfNone
, the neural network with smallest validation error is used to build the ROM. If a tolerance is prescribed, the mean squared error of the neural network on the training set should not exceed this threshold. Training is interrupted if a neural network that undercuts the error tolerance is found.
 _compute_sample(self, mu, u=None)[source]¶
Transform parameter and corresponding solution to
NumPy arrays
.This function takes care of including the time instances in the inputs.
 class pymor.reductors.neural_network.NeuralNetworkInstationaryStatefreeOutputReductor(fom, nt, training_set, validation_set=None, validation_ratio=0.1, validation_loss=None)[source]¶
Bases:
NeuralNetworkStatefreeOutputReductor
Output reductor relying on artificial neural networks.
This is a reductor that trains a neural network that approximates the mapping from parameter space to output space.
Parameters
 fom
The fullorder
Model
to reduce. nt
Number of time steps in the reduced order model (does not have to coincide with the number of time steps in the full order model).
 training_set
Set of
parameter values
to use for POD and training of the neural network. validation_set
Set of
parameter values
to use for validation in the training of the neural network. validation_ratio
Fraction of the training set to use for validation in the training of the neural network (only used if no validation set is provided).
 validation_loss
The validation loss to reach during training. If
None
, the neural network with the smallest validation loss is returned.
 _compute_sample(self, mu)[source]¶
Transform parameter and corresponding output to
NumPy arrays
.This function takes care of including the time instances in the inputs.
 class pymor.reductors.neural_network.EarlyStoppingScheduler(size_training_validation_set, patience=10, delta=0.0)[source]¶
Bases:
pymor.core.base.BasicObject
Class for performing early stopping in training of neural networks.
If the validation loss does not decrease over a certain amount of epochs, the training should be aborted to avoid overfitting the training data. This class implements an early stopping scheduler that recommends to stop the training process if the validation loss did not decrease by at least
delta
overpatience
epochs.Parameters
 size_training_validation_set
Size of both, training and validation set together.
 patience
Number of epochs of nondecreasing validation loss allowed, before early stopping the training process.
 delta
Minimal amount of decrease in the validation loss that is required to reset the counter of nondecreasing epochs.
 __call__(self, losses, neural_network=None)[source]¶
Returns
True
if early stopping of training is suggested.Parameters
 losses
Dictionary of losses on the validation and the training set in the current epoch.
 neural_network
Neural network that produces the current validation loss.
Returns
True
if early stopping is suggested,False
otherwise.
 class pymor.reductors.neural_network.CustomDataset(training_data)[source]¶
Bases:
torch.utils.data.Dataset
Class that represents the dataset to use in PyTorch.
Parameters
 training_data
Set of training parameters and the respective coefficients of the solution in the reduced basis.
 pymor.reductors.neural_network.train_neural_network(training_data, validation_data, neural_network, training_parameters={})[source]¶
Training algorithm for artificial neural networks.
Trains a single neural network using the given training and validation data.
Parameters
 training_data
Data to use during the training phase. Has to be a list of tuples, where each tuple consists of two elements that are either PyTorchtensors (
torch.DoubleTensor
) orNumPy arrays
or pyMOR data structures that haveto_numpy()
implemented. The first element contains the input data, the second element contains the target values. validation_data
Data to use during the validation phase. Has to be a list of tuples, where each tuple consists of two elements that are either PyTorchtensors (
torch.DoubleTensor
) orNumPy arrays
or pyMOR data structures that haveto_numpy()
implemented. The first element contains the input data, the second element contains the target values. neural_network
The neural network to train (can also be a pretrained model). Has to be a PyTorchModule.
 training_parameters
Dictionary with additional parameters for the training routine like the type of the optimizer, the (maximum) number of epochs, the batch size, the learning rate or the loss function to use. Possible keys are
'optimizer'
(an optimizer from the PyTorchoptim
package; if not provided, the LBFGSoptimizer is taken as default),'epochs'
(an integer that determines the number of epochs to use for training the neural network (if training is not interrupted prematurely due to early stopping); if not provided, 1000 is taken as default value),'batch_size'
(an integer that determines the number of samples to pass to the optimizer at once; if not provided, 20 is taken as default value; not used in the case of the LBFGSoptimizer since LBFGS does not support minibatching),'learning_rate'
(a positive real number used as the (initial) step size of the optimizer; if not provided, 1 is taken as default value; thus far, no learning rate schedulers are supported in this implementation), and'loss_function'
(a loss function from PyTorch; if not provided, the MSE loss is taken as default).
Returns
 best_neural_network
The best trained neural network with respect to validation loss.
 losses
The corresponding losses as a dictionary with keys
'full'
(for the full loss containing the training and the validation average loss),'train'
(for the average loss on the training set), and'val'
(for the average loss on the validation set).
 pymor.reductors.neural_network.multiple_restarts_training(training_data, validation_data, neural_network, target_loss=None, max_restarts=10, training_parameters={}, seed=None)[source]¶
Algorithm that performs multiple restarts of neural network training.
This method either performs a predefined number of restarts and returns the best trained network or tries to reach a given target loss and stops training when the target loss is reached.
See
train_neural_network
for more information on the parameters.Parameters
 training_data
Data to use during the training phase.
 validation_data
Data to use during the validation phase.
 target_loss
Loss to reach during training (if
None
, the network with the smallest loss is returned). max_restarts
Maximum number of restarts to perform.
 neural_network
The neural network to train (parameters will be reset after each restart).
Returns
 best_neural_network
The best trained neural network.
 losses
The corresponding losses.
Raises
 NeuralNetworkTrainingFailed
Raised if prescribed loss can not be reached within the given number of restarts.