Template Function mindspore::dataset::TFRecord

Function Documentation

template<typename T = std::shared_ptr<SchemaObj>>
std::shared_ptr<TFRecordDataset> mindspore::dataset::TFRecord(const std::vector<std::string> &dataset_files, const T &schema = nullptr, const std::vector<std::string> &columns_list = {}, int64_t num_samples = 0, ShuffleMode shuffle = ShuffleMode::kGlobal, int32_t num_shards = 1, int32_t shard_id = 0, bool shard_equal_rows = false, const std::shared_ptr<DatasetCache> &cache = nullptr)

Function to create a TFRecordDataset.

Parameters

  • dataset_files – [in] List of files to be read to search for a pattern of files. The list will be sorted in a lexicographical order.

  • schema – [in] SchemaObj or string to schema path. (Default = nullptr, which means that the meta data from the TFData file is considered the schema).

  • columns_list – [in] List of columns to be read (Default = {}, read all columns).

  • num_samples – [in] The number of samples to be included in the dataset (Default = 0 means all samples). If num_samples is 0 and numRows(parsed from schema) does not exist, read the full dataset; If num_samples is 0 and numRows(parsed from schema) is greater than 0, read numRows rows; If both num_samples and numRows(parsed from schema) are greater than 0, read num_samples rows.

  • shuffle – [in] The mode for shuffling data every epoch. (Default = ShuffleMode::kGlobal) Can be any of: ShuffleMode::kFalse - No shuffling is performed. ShuffleMode::kFiles - Shuffle files only. ShuffleMode::kGlobal - Shuffle both the files and samples.

  • num_shards – [in] Number of shards that the dataset should be divided into (Default = 1).

  • shard_id – [in] The shard ID within num_shards. This argument should be specified only when num_shards is also specified (Default = 0).

  • shard_equal_rows – [in] Get equal rows for all shards (Default = False, number of rows of each shard may be not equal).

  • cache – [in] Tensor cache to use (default=nullptr which means no cache is used).

Returns

Shared pointer to the TFRecordDataset.

Example
/* Define dataset path and MindData object */
std::string file_path = "/path/to/tfrecord_file";
std::string schema_path = "/path/to/schema_file";
std::shared_ptr<Dataset> ds = TFRecord({file_path}, schema_path, {"image"});

/* Create iterator to read dataset */
std::shared_ptr<Iterator> iter = ds->CreateIterator();
std::unordered_map<std::string, mindspore::MSTensor> row;
iter->GetNextRow(&row);

/* Note: The columns of generated dataset depend on the source TFRecord files. */
auto image = row["image"];