How can I include the "blue dot" when taking pictures of an app for a test report.

The App Testing Framework allows Matlab to "press" GUI buttons to test that they have the right effect, and it provides a helpful blue dot so that you can see where the virtual mouse clicks occur, while it happens. It would be handy if I could take pictures of this behaviour and log them in the error report, as an additional check that pressing the "Plot" button causes a graph to appear. Unfortunately, I know two methods that almost do this, but not quite. Both these are used with testCase.log() or similar constructions.

1. Using the FigureDiagnostic class (e.g. FigureDiagnostic(app.UIFigure, Formats = "png") ) correctly takes a picture of the app, but it does not include the blue dot. This makes it a bit harder to see at a glance what the test was attempting to do.
2. Using the ScreenshotDiagnostic class does include the blue dot in its image (see clipping above), but the screenshot includes the entire screen (including all monitors), not just the app. It would be inconvenient to turn off my extra monitors every time I run tests.

Is there a way to combine the advantage of both these methods, please?

I have thought of expanding the gesture functions ( testCase.press(app.Switch_1) etc.) to also change the colour of the control (although switches don't have a changeable colour); or to superimpose a label object with a symbol indicating the interaction. But they may come with all sorts of problems of their own, and I hope there is an easier way.

[Richard Quist]

I don't know if this will work any better than FigureDiagnostic, but you can try using the exportapp function if you have R2020b or later.

HTH

Answer 1

[Vidhi Agarwal]

Hi David,

I understand you want to capture the blue dot interaction with the virtual mouse for the test report. Combining the advantages of both the “FigureDiagnostic” and “ScreenshotDiagnostic” classes to capture the app's state with the blue dot indicating the virtual mouse clicks, without capturing the entire screen setup, is indeed a nuanced requirement.  

However, based on your needs, a creative approach that might serve as a workaround is to take a full screenshot including the blue dot and then cropping it to the area of your app window. Here's an approach you can achieve the functionality:

• First, use the “ScreenshotDiagnostic” to capture the entire screen, ensuring you include the blue dot interaction.
• You can programmatically determine the position and size of your app window using MATLAB properties. For instance, “app.UIFigure” has properties such as “Position” that you can use to get its dimensions and location on the screen.

appPosition = app.UIFigure.Position;

• With the screenshot taken and the app's position known, you can use MATLAB's image processing capabilities to crop the screenshot to just the area of your app window.

fullScreenshot = imread('screenshot.png'); % Read the screenshot taken in Step 1 
croppedImage = imcrop(fullScreenshot, [appPosition(1), appPosition(2), appPosition(3), appPosition(4)]); 
imwrite(croppedImage, 'croppedScreenshot.png'); % Save the cropped image

• Now, you can use the cropped image in your error report or logging mechanism.

As you've considered altering UI elements or adding indicators, this could be a less intrusive approach if implemented carefully:

• Temporary UI Changes: Temporarily change a property of the UI element (like background color) right before the action and reset it afterward. This requires careful timing and might not always be visually captured depending on when the screenshot is taken.
• Overlay Elements: Programmatically overlay a temporary graphical element (like a circle or arrow) on the UI to indicate the action point. This could be a bit more visually consistent but requires additional coding to manage these overlays.

For better understanding of the “exportapp”, “FigureDiagnostic” and “ScreenshotDiagnostic” you can refer to the following documentation:

• https://www.mathworks.com/help/matlab/ref/exportapp.html
• https://www.mathworks.com/help/matlab/ref/matlab.unittest.diagnostics.figurediagnostic-class.html
• https://www.mathworks.com/help/matlab/ref/matlab.unittest.diagnostics.screenshotdiagnostic-class.html

[David Szwer]

Thank you. Unfortunately I have moved to a completely different project, so I'm unable to test this any time soon, but it seems like the only solution.