Function mindspore::dataset::TextFile
Defined in File datasets.h
Function Documentation
Function to create a TextFileDataset.
Note
The generated dataset has one column [‘text’].
- Parameters
dataset_files – [in] List of files to be read to search for a pattern of files. The list will be sorted in a lexicographical order.
num_samples – [in] The number of samples to be included in the dataset (Default = 0 means all samples).
shuffle – [in] The mode for shuffling data every epoch (Default=ShuffleMode.kGlobal). Can be any of: ShuffleMode.kFalse - No shuffling is performed. ShuffleMode.kFiles - Shuffle files only. ShuffleMode.kGlobal - Shuffle both the files and samples.
num_shards – [in] Number of shards that the dataset should be divided into (Default = 1).
shard_id – [in] The shard ID within num_shards. This argument should be specified only when num_shards is also specified (Default = 0).
cache – [in] Tensor cache to use (default=nullptr which means no cache is used).
- Returns
Shared pointer to the TextFileDataset.
样例/* Define dataset path and MindData object */ std::string file_path = "/path/to/text_file_dataset_file"; std::shared_ptr<Dataset> ds = TextFile({file_path}, 2); /* Create iterator to read dataset */ std::shared_ptr<Iterator> iter = ds->CreateIterator(); std::unordered_map<std::string, mindspore::MSTensor> row; iter->GetNextRow(&row); /* Note: In TextFile dataset, each dictionary has key "text" */ auto text = row["text"];