padv.Assessment

Define formal assessments for task inputs and outputs

Description

This object requires CI/CD Automation for Simulink Check.

A padv.Assessment object represents a formal assessment for a task. You can use assessments to evaluate the compliance of task inputs and outputs against specific standards or custom criteria that you define and manage in the padv.Assessment object. When you add an assessment to a task and run the task, the padv.Assessment object can analyze the task inputs and outputs to generate assessment results, which determine the compliance status and task status and results. You can add assessments to a task by using the addAssessments method of padv.Task. Each assessment can specify evaluation criteria and executable actions to measure task results against defined objectives. The results of these assessments can influence the task status, triggering actions such as task failure if the task inputs, outputs, or task results match specific criteria. The build system automatically runs task assessments after you run the task itself.

Alternatively, you can define a task assessment by overriding the assess method of the task. For more information, see padv.Task.

Starting in R2024a, when you point to the task status, you can see a breakdown of the individual assessments for a task, the assessment results, and the impact of those assessment results on the overall task status. The assessment information provides the specific objectives associated with the failures, warnings, and passing results that you see in the Details column.

Creation

Description

padv.Assessment(Id,Objective=Objective) creates an assessment with the specified identifier and objective. Each assessment that you add to a task must have a unique Id.

padv.Assessment(___,Name=Value) sets properties using one or more name-value arguments.

For example, padv.Assessment("A1",Objective="Each requirement has at least one test.",Action=@assessRequirementsTraceability) creates an assessment that describes the objective of the assessment and uses the function assessRequirementsTraceability to perform the assessment.

Input Arguments

Id — Unique identifier for assessment

string

Unique identifier for assessment, specified as a string.

Each assessment that you add to a task must have a unique Id.

Example: "A1"

Data Types: string

Objective — Purpose of assessment

string

Purpose of the assessment, specified as a string.

You can use the Objective to describe what the assessment evaluates and expects.

Example: "Each requirement has at least one test."

Data Types: string

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: padv.Assessment("A1",Objective="Each requirement has at least one test.",Action=@assessRequirementsTraceability)

Action — Action that assessment performs

function handle

Action that assessment performs, specified as a function handle.

The Action defines the behavior of the assessment. When you run a task, the build system automatically runs the assessments associated with the task.

The function you reference must use this function signature:

function assessmentResult = functionName(assessment,inputArtifacts,taskResult)

You create and access assessment results by using a padv.AssessmentResult object.

Example: @assessRequirementsTraceability

Data Types: function_handle

RecommendedActions — Recommended action

padv.RecommendedAction object

Recommended action, specified as a padv.RecommendedAction object.

The padv.RecommendedAction object provides instructions that can help you fix the issues identified in the assessment results.

Example: padv.RecommendedAction(Instruction="Link requirement to test.")

Properties

Id — Unique identifier for assessment

string

Unique identifier for assessment, specified as a string.

Example: "A1"

Data Types: string

Objective — Purpose of assessment

string

Purpose of the assessment, specified as a string.

You can use the Objective to describe what the assessment evaluates and expects.

Example: "Each requirement has at least one test."

Data Types: string

Action — Action that assessment performs

function handle

Action that assessment performs, specified as a function handle.

The Action defines the behavior of the assessment. When you run a task, the build system automatically runs the assessments associated with the task.

The function you reference must use this function signature:

function assessmentResult = functionName(assessment,inputArtifacts,taskResult)

You create the assessment results by using a padv.AssessmentResult object.

Example: @assessRequirementsTraceability

Data Types: function_handle

RecommendedActions — Recommended action

padv.RecommendedAction object

Recommended action, specified as a padv.RecommendedAction object.

The padv.RecommendedAction object provides instructions that can help you fix the issues identified in the assessment results.

This property is read-only unless you specify the property value by using the addRecommendedActions function.

Example: padv.RecommendedAction(Instruction="Link requirement to test.")

Object Functions

Object Function	Description
addRecommendedActions	Add recommended actions addRecommendedActions(assessmentObj,recommendedActions)
run	Run the assessment. The run method runs on the assessment instance assessmentObj with task inputs inputArtifacts and task result taskResult. If you inherit from this class, make sure to use the same method signature. assessmentResult = run(assessmentObj,inputArtifacts,taskResult) Note You do not need to manually run this function. When you run a task using the Process Advisor app or the runprocess function, the build system automatically runs task assessments by using the run function.

Examples

Assess Quality of Task Results

You can define and manage formal assessments of your task inputs and outputs by using a padv.Assessment object. The padv.Assessment object specifies the assessment objectives, the assessment action that the assessment performs to evaluate the task and determine the task status, and recommended actions. You can associate an assessment with a specific task by using the addAssessments method and the build system automatically performs your assessments when you run the task.

Open the Process Advisor example project.

processAdvisorExampleStart

The model AHRS_Voter opens with the Process Advisor pane to the left of the Simulink^® canvas.

In the Process Advisor pane, click the Edit process model button to open the processmodel.m file for the project.

Replace the contents of the processmodel.m file with the following example code. The code defines:

• T — A custom task that uses the action function myTaskAction to create a task output file output.txt.

• A1 — An assessment that uses the action function assessOutputIsTxt to check if that output file is a text file.

• A2 — An assessment that uses the action function assessOutputFileContent to check if the output file is empty.

The assessment action functions summarize the results of the assessment by using padv.AssessmentResult objects.

function processmodel(pm)
    % Define process model for project

    arguments
        pm padv.ProcessModel
    end

    % --- Assessments ---
    A1 = padv.Assessment("A1",Objective="Output file must be a TXT file.",Action=@assessOutputIsTxt);
    A2 = padv.Assessment("A2",Objective="Output file should not be empty.",Action=@assessOutputFileContent);

    % --- Task ---
    T = padv.Task("MyTask",Action=@myTaskAction,OutputDirectory=fullfile("$PROJECTROOT$"),...
        Assessments=[A1,A2]);

    % Add the task to the process model
    pm.addTask(T);

end

function taskResult = myTaskAction(~)
    % Create a file named "output.txt"
    filename = "output.txt";
    fileID = fopen(filename,'w');
    fclose(fileID);
    % Create task result and specify file as task output
    taskResult = padv.TaskResult;
    taskResult.Status = padv.TaskStatus.Pass;
    taskResult.OutputPaths = fullfile(pwd,filename);
end

function res = assessOutputIsTxt(assessment, inputs, taskResult)
    try
        % Get first output artifact and its file address
        outputFile = taskResult.OutputArtifacts(1);
        fileAddress = outputFile.ArtifactAddress.getFileAddress();
        
        % Check if file has .txt extension
        [~, ~, ext] = fileparts(fileAddress);
        if strcmp(ext, ".txt")
            res = padv.AssessmentResult(assessment.Id, Status="Compliant");
            res.Summary = "Output file is TXT.";
            res.addCompliantArtifacts(outputFile);
        else
            res = padv.AssessmentResult(assessment.Id, Status="Warning");
            res.Summary = "Output file is not TXT.";
            res.addWarningArtifacts(outputFile);
        end
    catch
        res = padv.AssessmentResult(assessment.Id, Status="NonCompliant");
        res.Summary = "Output file is inaccessible or missing.";
    end
end

function res = assessOutputFileContent(assessment, inputs, taskResult)
    try
        % Get first output artifact and its file info
        outputFile = taskResult.OutputArtifacts(1);
        fileAddress = outputFile.ArtifactAddress.getFileAddress();
        fileID = fopen(fileAddress, 'r');
        fileContent = fread(fileID);
        fclose(fileID);
        % Check if file is empty
        if isempty(fileContent)
            res = padv.AssessmentResult(assessment.Id, Status="Warning");
            res.Summary = "Output file is empty.";
            res.addWarningArtifacts(outputFile);
        else
            res = padv.AssessmentResult(assessment.Id, Status="Compliant");
            res.Summary = "Output file is not empty.";
        end
    catch
        res = padv.AssessmentResult(assessment.Id, Status="NonCompliant");
        res.Summary = "Output file is inaccessible or missing.";
    end
end

In Process Advisor, run the task by clicking the Run All button.

In the Tasks column, the task status shows that the task passed. But the Details column shows one warning and one passing result.

Point to the task status to the left of the task name. Starting in R2024a, the Assessments section shows that the passing result comes from the A1 assessment because the task successfully generated a text file. The warning result comes from the A2 assessment because the task generates an empty text file.

View a breakdown of the assessments, assessment results, and the compliant, warning, and non-compliant artifacts by clicking Assessments in the task status pop-up. If you add recommended actions to your assessments by using the addRecommendedActions function, you can view those recommended actions in the Assessments dialog.

By default, non-compliant assessments cause the task to fail. But suppose that you want the warning assessment result to cause the task to fail. You can change the task definition in the process model to make the task fail on any warning results from the assessments by using the task property FailTaskOn.

    % --- Task ---
    T = padv.Task("MyTask",Action=@myTaskAction,OutputDirectory=fullfile("$PROJECTROOT$"),...
        Assessments=[A1,A2],FailTaskOn="AnyWarning");

Run the task in Process Advisor by clicking Refresh Tasks and clicking the run button for the task.

In the Tasks column, the task status now shows that the task failed. If you point to the task status, the Assessments section now indicates that warnings and non-compliant assessments cause the task to fail.