Function mindspore::dataset::CSV
Defined in File datasets.h
Function Documentation
Function to create a CSVDataset.
Note
The generated dataset has a variable number of columns.
- Parameters
dataset_files – [in] List of files to be read to search for a pattern of files. The list will be sorted in a lexicographical order.
field_delim – [in] A char that indicates the delimiter to separate fields (default=’,’).
column_defaults – [in] List of default values for the CSV field (default={}). Each item in the list is either a valid type (float, int, or string). If this is not provided, treats all columns as string type.
column_names – [in] List of column names of the dataset (default={}). If this is not provided, infers the column_names from the first row of CSV file.
num_samples – [in] The number of samples to be included in the dataset. (Default = 0 means all samples).
shuffle – [in] The mode for shuffling data every epoch (Default=ShuffleMode::kGlobal). Can be any of: ShuffleMode::kFalse - No shuffling is performed. ShuffleMode::kFiles - Shuffle files only. ShuffleMode::kGlobal - Shuffle both the files and samples.
num_shards – [in] Number of shards that the dataset should be divided into. (Default = 1)
shard_id – [in] The shard ID within num_shards. This argument should be specified only when num_shards is also specified (Default = 0).
cache – [in] Tensor cache to use.(default=nullptr which means no cache is used).
- Returns
Shared pointer to the CSVDataset
Example/* Define dataset path and MindData object */ std::string train_file = "/path/to/csv_file"; std::vector<std::string> column_names = {"col1", "col2", "col3", "col4"}; std::shared_ptr<Dataset> ds = CSV({train_file}, ',', {}, column_names, 0, ShuffleMode::kFalse); /* Create iterator to read dataset */ std::shared_ptr<Iterator> iter = ds->CreateIterator(); std::unordered_map<std::string, mindspore::MSTensor> row; iter->GetNextRow(&row); /* Note: As we defined before, the dataset has column "col1", "col2", "col3" and "col4" */ auto col1 = row["col1"];