partition

Partition a label datastore

Syntax

subds = partition(ds,N,index)

subds = partition(ds,'Files',fileIndex)

subds = partition(ds,'Files',filename)

Description

subds = partition(ds,N,index) partitions a datastore ds into N parts and returns the partition corresponding to index.

subds = partition(ds,'Files',fileIndex) partitions the PixelLabelDatastore or pixelLabelImageDatastore by files. The partitioned datastore corresponds to the index in the Files property of the datastore.

subds = partition(ds,'Files',filename) partitions the PixelLabelDatastore or pixelLabelImageDatastore by files. The partitioned datastore corresponds to the file specified by filename.

Examples

Partition a Datastore

Load training data that contains bounding boxes with labels for vehicles.

data = load('vehicleTrainingData.mat');
trainingData = data.vehicleTrainingData;
blds = boxLabelDatastore(trainingData(:,2:end));

Use the partition function to partition the box label datastore. The output contains the first 5 rows of the training data.

subds = partition(blds,59,5);

Read the data. Use the hasdata function to check for data.

while hasdata(subds)
      % Read one row of box labels at a time
      bxLabels = read(subds);
end

Input Arguments

ds — Datastore with labeled data

PixelLabelDatastore object | pixelLabelImageDatastore object | boxLabelDatastore object

Datastore with labeled data for training a semantic segmentation network or an object detection network, specified as a pixelLabelDatastore, pixelLabelImageDatastore, or boxLabelDatastore object.

N — Number of partitions

positive integer

Number of partitions to divide datastore, specified as a positive integer. To obtain an estimate for a reasonable number of partitions, use the numpartitions function.

numWorkers = 3;
p = parpool('local',numWorkers);
n = numpartitions(ds,p);

parfor ii=1:n
    subds = partition(ds,n,ii);
    while hasdata(subds)
        data = read(subds);
    end
end

Example: 3

Data Types: double

index — Index

positive integer

Index, specified as an integer. The index points to a partitioned datastore of the N number of partitioned datastores.

Example: 1

Data Types: double

fileIndex — File index

positive integer

File index of a file stored within the Files property of a pixelLabelDatastore or pixelLabelImageDatastore, specified as a positive integer.

Example: 1

Data Types: double

filename — file name

character vector

File name, specified as a character vector.

Example: 'file1.csv'

Example: '../dir/data/file1.csv'

Example: 'hdfs://myserver:7867/data/file1.txt'

Data Types: char

Output Arguments

subds — Output datastore

datastore object

Output datastore, returned as a datastore object. The output datastore is of the same type as the input datastore, ds.

Version History

Introduced in R2017b