mindspore.nn.probability.toolbox.UncertaintyEvaluation

class mindspore.nn.probability.toolbox.UncertaintyEvaluation(model, train_dataset, task_type, num_classes=None, epochs=1, epi_uncer_model_path=None, ale_uncer_model_path=None, save_model=False)

Toolbox for Uncertainty Evaluation.

Parameters

• model (Cell) – The model for uncertainty evaluation.

• train_dataset (Dataset) – A dataset iterator to train model.

• task_type (str) – Option for the task types of model - regression: A regression model. - classification: A classification model.

• num_classes (int) – The number of labels of classification. If the task type is classification, it must be set; otherwise, it is not needed. Default: None.

• epochs (int) – Total number of iterations on the data. Default: 1.

• epi_uncer_model_path (str) – The save or read path of the epistemic uncertainty model. Default: None.

• ale_uncer_model_path (str) – The save or read path of the aleatoric uncertainty model. Default: None.

• save_model (bool) – Whether to save the uncertainty model or not, if true, the epi_uncer_model_path and ale_uncer_model_path must not be None. If false, the model to evaluate will be loaded from the the path of the uncertainty model; if the path is not given , it will not save or load the uncertainty model. Default: False.

Supported Platforms:

Ascend GPU

Examples

>>> network = LeNet()
>>> param_dict = load_checkpoint('checkpoint_lenet.ckpt')
>>> load_param_into_net(network, param_dict)
>>> ds_train = create_dataset('workspace/mnist/train')
>>> ds_eval = create_dataset('workspace/mnist/test')
>>> evaluation = UncertaintyEvaluation(model=network,
...                                    train_dataset=ds_train,
...                                    task_type='classification',
...                                    num_classes=10,
...                                    epochs=1,
...                                    epi_uncer_model_path=None,
...                                    ale_uncer_model_path=None,
...                                    save_model=False)
>>> for eval_data in ds_eval.create_dict_iterator(output_numpy=True, num_epochs=1):
...    eval_data = Tensor(eval_data['image'], mstype.float32)
...    epistemic_uncertainty = evaluation.eval_epistemic_uncertainty(eval_data)
...    aleatoric_uncertainty = evaluation.eval_aleatoric_uncertainty(eval_data)
>>> output = epistemic_uncertainty.shape
>>> print(output)
(32, 10)
>>> output = aleatoric_uncertainty.shape
>>> print(output)
(32,)

eval_aleatoric_uncertainty(eval_data)

Evaluate the aleatoric uncertainty of inference results, which also called data uncertainty.

Parameters

eval_data (Tensor) – The data samples to be evaluated, the shape must be (N,C,H,W).

Returns

numpy.dtype, the aleatoric uncertainty of inference results of data samples.

eval_epistemic_uncertainty(eval_data)

Evaluate the epistemic uncertainty of inference results, which also called model uncertainty.

Parameters

eval_data (Tensor) – The data samples to be evaluated, the shape must be (N,C,H,W).

Returns

numpy.dtype, the epistemic uncertainty of inference results of data samples.