Class DistributedSampler
Defined in File samplers.h
Inheritance Relationships
Base Type
public mindspore::dataset::Sampler
(Class Sampler)
Class Documentation
-
class DistributedSampler : public mindspore::dataset::Sampler
A class to represent a Distributed Sampler in the data pipeline.
Note
A Sampler that accesses a shard of the dataset.
Public Functions
-
DistributedSampler(int64_t num_shards, int64_t shard_id, bool shuffle = true, int64_t num_samples = 0, uint32_t seed = 1, int64_t offset = -1, bool even_dist = true)
Constructor.
- Parameters
num_shards – [in] Number of shards to divide the dataset into.
shard_id – [in] Shard ID of the current shard within num_shards.
shuffle – [in] If true, the indices are shuffled (default=true).
num_samples – [in] The number of samples to draw (default=0, return all samples).
seed – [in] The seed in use when shuffle is true (default=1).
offset – [in] The starting position where access to elements in the dataset begins (default=-1).
even_dist – [in] If true, each shard would return the same number of rows (default=true). If false the total rows returned by all the shards would not have overlap.
样例/* creates a distributed sampler with 2 shards in total. This shard is shard 0 */ std::string file_path = "/path/to/test.mindrecord"; std::shared_ptr<Dataset> ds = MindData(file_path, {}, std::make_shared<DistributedSampler>(2, 0, false));
-
~DistributedSampler() = default
Destructor.
-
DistributedSampler(int64_t num_shards, int64_t shard_id, bool shuffle = true, int64_t num_samples = 0, uint32_t seed = 1, int64_t offset = -1, bool even_dist = true)