risk.validation.accuracyRatio

Return accuracy ratio value

Since R2025a

collapse all in page

    Syntax

    arValue = risk.validation.accuracyRatio(Score,BinaryResponse)
    arValue = risk.validation.accuracyRatio(Score,BinaryResponse,SortDirection=sortdir)
    [arValue,Output] = risk.validation.accuracyRatio(___)

    Description

    arValue = risk.validation.accuracyRatio(Score,BinaryResponse) returns the accuracy ratio, where Score contains numeric values that represent quantities such as rankings or predictions, probability of default (PD), or loss given default (LGD) estimates. For example, in credit scoring models, the values in Score can represent individual credit scores or other credit data. BinaryResponse specifies the target state of each value in Score.

    example

    arValue = risk.validation.accuracyRatio(Score,BinaryResponse,SortDirection=sortdir) specifies the sorting direction of the unique values in Score.

    [arValue,Output] = risk.validation.accuracyRatio(___) also returns a structure Output, that contains a table of metrics with columns Thresholds, TruePositiveRate, and RateOfPositivePredictions.

    Examples

    collapse all

    Open Live Script

    Compute the accuracy ratio of credit scores by using the accuracyRatio function with a credit validation data set contained in creditValidationData.mat. This data set includes a table, ScorecardValidationData, that contains credit scores and their corresponding default status.

    Load the credit validation data and display the table.

    load CreditValidationData.mat
head(ScorecardValidationData)
        CreditScore      PD       Default
    ___________    _______    _______

      579.86       0.14182       0   
      563.65       0.17143       0   
      549.52       0.20106       0   
      546.25       0.20845       0   
      485.34       0.37991       0   
      482.07       0.39065       0   
      579.86       0.14182       1   
      451.73         0.494       0

    Extract the variables CreditScore and Default from the table ScorecardValidationData. Use Default as the BinaryResponse input argument.

    Scores = ScorecardValidationData.CreditScore;
BinaryResponse = ScorecardValidationData.Default;

    Compute the accuracy ratio by using the accuracyRatio function with the fully qualified namespace risk.validation. For credit models, you can sort the scores from low scores to high scores by setting the SortDirection name-value argument to "ascending". This setting ensures that the function sorts the scores from higher risk individuals to lower risk individuals.

    [arValue,Output] = risk.validation.accuracyRatio(Scores,BinaryResponse,SortDirection="ascending")
    arValue = 
0.2156

    Output = struct with fields:
    AccuracyRatio: 0.2156
          Metrics: [107×3 table]

    The output argument arValue contains the accuracy ratio. You can display the metrics Threshold, TruePositiveRate, and RateOfPositivePredictions contained in the table Output.Metrics. For credit scoring models, TruePositiveRate corresponds to the proportion of defaulters.

    head(Output.Metrics)
        Threshold    TruePositiveRate    RateOfPositivePredictions
    _________    ________________    _________________________

     408.99                 0                       0         
     408.99          0.071429                0.033333         
     410.12          0.079365                0.038889         
     430.66          0.087302                0.041667         
     435.52          0.087302                0.047222         
     436.65           0.10317                0.055556         
     439.33           0.11905                0.061111         
     440.45           0.13492                0.066667

    Input Arguments

    collapse all

    Score values, specified as a numeric vector, which contains values that indicate quantities such as rankings or predictions, PD, or LGD estimates.

    Binary response, specified as a numeric or logical vector, that contains values of 1 (true) or 0 (false). The binary response represents the target state for each value in Score. For example, you can use the binary response to represent a discretized LGD target, where ones indicate a high LGD value.

    Sorting direction of the unique values in Score, specified as "descending" or "ascending". If Score contains credit scores, where low values commonly correspond to higher risk individuals, you can set the sorting direction to "ascending". This setting ensures that TruePositiveRate represents the proportion of defaulters. If Score contains PD values, where higher values correspond to higher risk, sorting the values in descending order is common practice.

    Output Arguments

    collapse all

    Accuracy ratio corresponding to the values in Score, returned as a numeric scalar.

    Output metrics, returned as a structure containing the following fields:

    • arValue — Accuracy ratio

    • Metrics — Table with columns:

      • Thresholds — Unique score values sorted according to the value of sortdir.

      • TruePositiveRate — True positive rate values corresponding to the unique scores in the Thresholds column. For credit scoring models, this column represents the proportion of defaulters.

      • RateOfPositivePredictions — Rate of positive predictions values corresponding to the unique scores in the Threshold column. This column corresponds to the proportion of borrowers.

      Metrics contains the data you need for making the capital accuracy profile (CAP), which plots TruePositiveRate as a function of RateOfPositivePredictions.

    Version History

    Introduced in R2025a

    See Also

    | |