training

Training indices for cross-validation

Syntax

idx = training(c)

idx = training(c,i)

idx = training(c,"all")

Description

idx = training(c) returns the training indices idx for a cvpartition object c of type 'holdout' or 'resubstitution'.

• If c.Type is 'holdout', then idx specifies the observations in the training set.

• If c.Type is 'resubstitution', then idx specifies all observations.

idx = training(c,i) returns the training indices for the repetitions specified in i of a cvpartition object c of type 'kfold' or 'leaveout'.

• If c.Type is 'kfold', then idx(:,j) specifies the observations in training set i(j).

• If c.Type is 'leaveout', then idx(:,j) specifies the observations reserved for training at repetition i(j).

idx = training(c,"all") returns the training indices for all repetitions of a cvpartition object c. Column j in idx indicates the observations in training set j for an object of type 'kfold' or 'leaveout'. (since R2023b)

Examples

Identify Training Indices in Holdout Partition

Identify the observations that are in the training set of a cvpartition object for holdout validation.

Partition 10 observations for holdout validation. Select approximately 30% of the observations to be in the test (holdout) set.

rng(0,"twister") % For reproducibility
c = cvpartition(10,Holdout=0.30)

c = 
Hold-out cross validation partition
    NumObservations: 10
        NumTestSets: 1
          TrainSize: 7
           TestSize: 3
           IsCustom: 0
          IsGrouped: 0
       IsStratified: 0


  Properties, Methods

Identify the training set observations. Observations that correspond to 1s are in the training set.

set = training(c)

set = 10×1 logical array

   1
   1
   1
   0
   1
   1
   1
   1
   0
   0

Visualize the results. All observations except the fourth, ninth, and tenth are in the training set.

h = heatmap(double(set),ColorbarVisible="off");
sorty(h,"1","ascend")
ylabel("Observation")
title("Training Set Observations")

Identify Training Indices in k-Fold Partition

Identify the observations that are in the training sets of a cvpartition object for 3-fold cross-validation.

Partition 10 observations for 3-fold cross-validation. Notice that c contains three repetitions of training and test data.

rng(0,"twister") % For reproducibility
c = cvpartition(10,KFold=3)

c = 
K-fold cross validation partition
    NumObservations: 10
        NumTestSets: 3
          TrainSize: [7 6 7]
           TestSize: [3 4 3]
           IsCustom: 0
          IsGrouped: 0
       IsStratified: 0


  Properties, Methods

Identify the training set observations for each repetition of training and test data. Observations that correspond to 1s are in the corresponding training set.

data = training(c,"all")

data = 10×3 logical array

   0   1   1
   0   1   1
   1   0   1
   1   1   0
   1   1   0
   1   0   1
   1   1   0
   1   0   1
   0   1   1
   1   0   1

Visualize the results. All observations except the first, second, and ninth are in the first training set. All observations except the third, sixth, eighth, and tenth are in the second training set. All observations except the fourth, fifth, and seventh are in the third training set.

h = heatmap(double(data),ColorbarVisible="off");
sorty(h,["1","2","3"],"ascend")
xlabel("Repetition")
ylabel("Observation")
title("Training Set Observations")

Input Arguments

c — Validation partition

cvpartition object

Validation partition, specified as a cvpartition object. The validation partition type of c, c.Type, is 'kfold', 'holdout', 'leaveout', or 'resubstitution'.

i — Repetition indices

positive integer vector

Repetition indices, specified as a positive integer vector. For each element in i, the software finds the observations in the training set of the corresponding repetition.

Example: 2

Example: [1 3 5]

Data Types: single | double

Output Arguments

idx — Indices for training set observations

logical matrix

Indices for training set observations, returned as a logical matrix. Each row corresponds to an observation, and each column corresponds to a repetition.

A value of 1 (true) indicates that the corresponding observation is in the training set. A value of 0 (false) indicates that the corresponding observation is in the test set.

Version History

Introduced in R2008a

R2023b: Retrieve indices for multiple training or test sets simultaneously

For both custom and noncustom cvpartition objects, you can return multiple training set indices or multiple test set indices simultaneously by using the training or test function, respectively. For example, for a 5-fold cross-validation cvpartition object c:

• The syntax test(c,[1 3 5]) returns a logical array with 3 columns, where each 1 (true) entry indicates a test set observation. The three columns correspond to the first, third, and fifth repetitions.

• The syntax training(c,"all") returns a logical array with 5 columns, where each 1 (true) entry indicates a training set observation.