Class PennTreebankDataset

• Defined in File datasets.h

Inheritance Relationships

Base Type

• public mindspore::dataset::Dataset (Class Dataset)

Class Documentation

class PennTreebankDataset : public mindspore::dataset::Dataset

A source dataset for reading and parsing PennTreebank dataset.

Public Functions

PennTreebankDataset(const std::vector<char> &dataset_dir, const std::vector<char> &usage, int64_t num_samples, ShuffleMode shuffle, int32_t num_shards, int32_t shard_id, const std::shared_ptr<DatasetCache> &cache)

Constructor of PennTreebank Dataset.

Parameters

• dataset_dir – [in] Path to the root directory that contains the dataset.

• usage – [in] The type of data list txt file to be read, can be “train”, “test”, ‘valid’ or “all”.

• num_samples – [in] The number of samples to be included in the dataset.

• shuffle – [in] The mode for shuffling data every epoch. Can be any of: ShuffleMode.kFalse - No shuffling is performed. ShuffleMode.kFiles - Shuffle files only. ShuffleMode.kGlobal - Shuffle both the files and samples.

• num_shards – [in] Number of shards that the dataset should be divided into.

• shard_id – [in] The shard ID within num_shards. This argument should be specified only when num_shards is also specified.

• cache – [in] Tensor cache to use.

~PennTreebankDataset() override = default

Destructor of PennTreebankDataset.