mindspore.dataset.Dataset.save

Dataset.save(file_name, num_files=1, file_type='mindrecord')

Save the dynamic data processed by the dataset pipeline in common dataset format. Supported dataset formats: 'mindrecord' only. And you can use mindspore.dataset.MindDataset API to read the saved file(s).

Implicit type casting exists when saving data as 'mindrecord' . The transform table shows how to do type casting.

Implicit Type Casting when Saving as mindrecord

Type in dataset	Type in mindrecord	Details
bool	None	Not supported
int8	int32
uint8	bytes(1D uint8)	Drop dimension
int16	int32
uint16	int32
int32	int32
uint32	int64
int64	int64
uint64	None	Not supported
float16	float32
float32	float32
float64	float64
string	string	Multi-dimensional string not supported

Note

1. To save the samples in order, set dataset’s shuffle to False and num_files to 1.

2. Before calling the function, do not use batch operation, repeat operation or data augmentation operations with random attribute in map operation.

3. When array dimension is variable, one-dimensional arrays or multi-dimensional arrays with variable dimension 0 are supported.

4. MindRecord does not support uint64, multi-dimensional uint8(drop dimension) nor multi-dimensional string.

Parameters

• file_name (str) – Path to dataset file.

• num_files (int, optional) – Number of dataset files. Default: 1 .

• file_type (str, optional) – Dataset format. Default: 'mindrecord' .

Examples

>>> import mindspore.dataset as ds
>>> import numpy as np
>>>
>>> def generator_1d():
...     for i in range(10):
...         yield (np.array([i]),)
>>>
>>> # apply dataset operations
>>> d1 = ds.GeneratorDataset(generator_1d, ["data"], shuffle=False)
>>> d1.save('/path/to/save_file')