TestIteration timeout in TestManager

Hello,

I am running a testManager having few testSuites, and each testSuite have a testCase with few TestIterations.

When running the TestManager a single TestIteration takes usually about 10min to complete, but in some cases the solver(my personal one) moves to smaller step size and the time for computation goes to 24+h per testIteartion. I do not want to spend 24h for a single step/testIteartion, but rather to skip it if it takes longer then 10min.

Is there a way to add a timeout in the TestManager/TestSuite for the testIteartions? If it takes more then 10min to just move to the next testIteartion with a message.

I was looking for this solution https://www.mathworks.com/help/matlab/ref/matlab.unittest.constraints.eventually-class.html but I am not sure I can add tic-tok inside the testManager.

I was also thinking to add timeout inside the TestSequence, but as simulation time stays at a given time step, while my solver is doing the computation for that time step, that is not working either.

Thank you for your help!

Answer 1

[Ishan]

I do not think matlab.unittest.constraints.Eventually is the right tool here. It waits for a condition to become true, but it does not interrupt a simulation that is blocked inside a long-running solver computation.

Similarly, a timeout implemented in a Test Sequence is unlikely to help if simulation time is not advancing. If the solver is spending a long wall-clock time inside a single timestep, the Test Sequence logic will not execute until control returns to the simulation engine.

As far as I can tell, Simulink Test does not provide a built-in wall-clock timeout for individual Test Iterations. The configured stop time is based on simulation time rather than elapsed execution time.

In this situation, the most reliable solution is usually to implement the timeout in the custom solver itself. The solver can periodically check elapsed wall-clock time and terminate gracefully if a predefined limit is exceeded. The iteration can then be marked as failed or errored, and the Test Manager can continue to the next iteration.

If the solver is implemented in C/C++ or inside an S-Function, the same approach can be implemented there. External MATLAB timers or Test Sequence logic generally cannot preempt a solver while it is executing a long computation.

Therefore, I would recommend:

1. Add a wall-clock timeout/checkpoint inside the custom solver.
2. Return a controlled error or diagnostic when the timeout is reached.
3. Configure the test run so that a failed iteration does not stop the remaining iterations.
4. Use the test results to identify which iterations exceeded the allowed runtime.

This is typically the most robust way to handle cases where simulation time is effectively stalled because the solver is spending excessive time on a single step.

[Kiril]

Hello Again,

Thank you for the answer !

I just saw that there is a function called Create Custom PlugIn ?

https://www.mathworks.com/help/matlab/matlab_prog/create-custom-plugin.html

Do you think I can add a plugin in the TestSuite, and there implement the tic tok mechaism ?

Where should the plug in be added? In the cration script of the TestManager ? or during the execution of the testSuite ?

[Ishan]

Hello Kiril,

I do not think a custom matlab.unittest plugin will solve this case. Those plugins are added to a matlab.unittest.TestRunner, not directly to a Simulink Test Manager TestSuite or TestIteration.

Also, even if you use a plugin, it cannot interrupt the simulation while the solver is busy inside one long computation step. The plugin code only runs when control returns to the test framework.

So I would still recommend putting the wall-clock timeout/check inside the custom solver itself, and returning a controlled error/status when the 10-minute limit is reached. Then Test Manager can continue with the next iteration.