Class ModelParallelRunner

Class Documentation

class ModelParallelRunner

The ModelParallelRunner class is used to define a MindSpore ModelParallelRunner, facilitating Model management.

Public Functions

inline Status Init(const std::string &model_path, const std::shared_ptr<RunnerConfig> &runner_config = nullptr)

build a model parallel runner from model path so that it can run on a device.

Parameters

  • model_path – [in] Define the model path.

  • runner_config – [in] Define the config used to store options during model pool init.

Returns

Status.

Status Init(const void *model_data, const size_t data_size, const std::shared_ptr<RunnerConfig> &runner_config = nullptr)

build a model parallel runner from model buffer so that it can run on a device.

Parameters

  • model_data – [in] Define the buffer read from a model file.

  • data_size – [in] Define bytes number of model buffer.

  • runner_config – [in] Define the config used to store options during model pool init.

Returns

Status.

std::vector<MSTensor> GetInputs()

Obtains all input tensors information of the model.

Returns

The vector that includes all input tensors.

std::vector<MSTensor> GetOutputs()

Obtains all output tensors information of the model.

Returns

The vector that includes all output tensors.

Status Predict(const std::vector<MSTensor> &inputs, std::vector<MSTensor> *outputs, const MSKernelCallBack &before = nullptr, const MSKernelCallBack &after = nullptr)

Inference ModelParallelRunner.

Parameters

  • inputs – [in] A vector where model inputs are arranged in sequence.

  • outputs – [out] Which is a pointer to a vector. The model outputs are filled in the container in sequence.

  • before – [in] CallBack before predict.

  • after – [in] CallBack after predict.

Returns

Status.