mindspore_lite.ModelParallelRunner

class mindspore_lite.ModelParallelRunner

The ModelParallelRunner class is used to define a MindSpore ModelParallelRunner, facilitating Model management.

Examples

>>> # only for serving inference
>>> import mindspore_lite as mslite
>>> model_parallel_runner = mslite.ModelParallelRunner()
>>> print(model_parallel_runner)
model_path: .

get_inputs()

Obtains all input tensors of the model.

Returns

list[Tensor], the inputs tensor list of the model.

Examples

>>> # model download link: https://download.mindspore.cn/model_zoo/official/lite/quick_start/mobilenetv2.ms
>>> import mindspore_lite as mslite
>>> context = mslite.Context()
>>> context.append_device_info(mslite.CPUDeviceInfo())
>>> runner_config = mslite.RunnerConfig(context=context, workers_num=4)
>>> model_parallel_runner = mslite.ModelParallelRunner()
>>> model_parallel_runner.init(model_path="mobilenetv2.ms", runner_config=runner_config)
>>> inputs = model_parallel_runner.get_inputs()

get_outputs()

Obtains all output tensors of the model.

Returns

list[Tensor], the outputs tensor list of the model.

Examples

>>> # model download link: https://download.mindspore.cn/model_zoo/official/lite/quick_start/mobilenetv2.ms
>>> import mindspore_lite as mslite
>>> context = mslite.Context()
>>> context.append_device_info(mslite.CPUDeviceInfo())
>>> runner_config = mslite.RunnerConfig(context=context, workers_num=4)
>>> model_parallel_runner = mslite.ModelParallelRunner()
>>> model_parallel_runner.init(model_path="mobilenetv2.ms", runner_config=runner_config)
>>> outputs = model_parallel_runner.get_outputs()

init(model_path, runner_config=None)

build a model parallel runner from model path so that it can run on a device.

Parameters

• model_path (str) – Define the model path.

• runner_config (RunnerConfig, optional) – Define the config used to store options during model pool init. Default: None.

Raises

• TypeError – model_path is not a str.

• TypeError – runner_config is neither a RunnerConfig nor None.

• RuntimeError – model_path does not exist.

• RuntimeError – ModelParallelRunner’s init failed.

Examples

>>> # model download link: https://download.mindspore.cn/model_zoo/official/lite/quick_start/mobilenetv2.ms
>>> import mindspore_lite as mslite
>>> context = mslite.Context()
>>> context.append_device_info(mslite.CPUDeviceInfo())
>>> runner_config = mslite.RunnerConfig(context=context, workers_num=4)
>>> model_parallel_runner = mslite.ModelParallelRunner()
>>> model_parallel_runner.init(model_path="mobilenetv2.ms", runner_config=runner_config)
>>> print(model_parallel_runner)
model_path: mobilenetv2.ms.

predict(inputs, outputs)

Inference ModelParallelRunner.

Parameters

• inputs (list[Tensor]) – A list that includes all input tensors in order.

• outputs (list[Tensor]) – The model outputs are filled in the container in sequence.

Raises

• TypeError – inputs is not a list.

• TypeError – inputs is a list, but the elements are not Tensor.

• TypeError – outputs is not a list.

• TypeError – outputs is a list, but the elements are not Tensor.

• RuntimeError – predict model failed.

Examples

>>> # model download link: https://download.mindspore.cn/model_zoo/official/lite/quick_start/mobilenetv2.ms
>>> # in_data download link: https://download.mindspore.cn/model_zoo/official/lite/quick_start/input.bin
>>> import mindspore_lite as mslite
>>> context = mslite.Context()
>>> context.append_device_info(mslite.CPUDeviceInfo())
>>> runner_config = mslite.RunnerConfig(context=context, workers_num=4)
>>> model_parallel_runner = mslite.ModelParallelRunner()
>>> model_parallel_runner.init(model_path="mobilenetv2.ms", runner_config=runner_config)
>>> inputs = model_parallel_runner.get_inputs()
>>> in_data = np.fromfile("input.bin", dtype=np.float32)
>>> inputs[0].set_data_from_numpy(in_data)
>>> outputs = model_parallel_runner.get_outputs()
>>> model_parallel_runner.predict(inputs, outputs)
>>> for output in outputs:
...     data = output.get_data_to_numpy()
...     print("outputs: ", data)
...
outputs:  [[8.9401474e-05 4.4536911e-05 1.0089713e-04 ... 3.2687691e-05
            3.6021424e-04 8.3650106e-05]]