Problem of using DDPG agent in external mode

1 view (last 30 days)
mehmet bozuyla
mehmet bozuyla on 7 Dec 2020
Answered: Emmanouil Tzorakoleftherakis on 11 Dec 2020
I am working on Reinforcement Learning to control coupled-tank system. I created a simulink environment for Deep Deterministic Policy Gradient (DDPG) agent. When I run the model in normal mode it works well. But when I choose the external mode it gives the below error. I would be happy you to help in this problem. Best regards
MATLAB System block 'rlwatertankControl/Coupled-Tank Simulation/RL Agent1/AgentWrapper' error occurred when invoking 'getSampleTime' method of 'AgentWrapper'. The error was thrown from '
'C:\Program Files\Polyspace\R2019b\toolbox\rl\rl\simulink\libs\AgentWrapper.m' at line 150
'C:\Program Files\Polyspace\R2019b\toolbox\rl\rl\simulink\libs\AgentWrapper.m' at line 210
'C:\Program Files\Polyspace\R2019b\toolbox\rl\rl\simulink\libs\AgentWrapper.m' at line 265
'C:\Program Files\Polyspace\R2019b\toolbox\rl\rl\simulink\libs\agentmaskinit.m' at line 11
'C:\Program Files\Polyspace\R2019b\toolbox\rtw\rtw\tlc_c.m' at line 199
'C:\Program Files\Polyspace\R2019b\toolbox\coder\simulinkcoder_core\+coder\+internal\ModelBuilder.m' at line 886
'C:\Program Files\Polyspace\R2019b\toolbox\coder\simulinkcoder_core\+coder\+internal\ModelBuilder.m' at line 218
'C:\Program Files\Polyspace\R2019b\toolbox\coder\simulinkcoder_core\+coder\+internal\@ModelCodegenMgr\make_rtw.m' at line 8
'C:\Program Files\Polyspace\R2019b\toolbox\coder\simulinkcoder_core\make_rtw.m' at line 20
'C:\Program Files\Polyspace\R2019b\toolbox\simulink\simulink\private\build_target.p' at line 0
'C:\Program Files\Polyspace\R2019b\toolbox\simulink\simulink\private\build_target.p' at line 0
'C:\Program Files\Polyspace\R2019b\toolbox\simulink\simulink\private\build_standalone_rtw_target.p' at line 0
'C:\Program Files\Polyspace\R2019b\toolbox\simulink\simulink\private\slbuild_private.p' at line 0
'C:\Program Files\Polyspace\R2019b\toolbox\simulink\simulink\private\slbuild_private.p' at line 0
'C:\Program Files\Polyspace\R2019b\toolbox\simulink\simulink\private\on_target_one_click.p' at line 0
'C:\Program Files\Polyspace\R2019b\toolbox\simulink\simulink\sl.m' at line 15
'C:\Program Files\Polyspace\R2019b\toolbox\coder\sltoolstrip_base_hw\m\+coder\+internal\+toolstrip\HardwareBoardHandler.m' at line 119
'C:\Program Files\Polyspace\R2019b\toolbox\coder\sltoolstrip_base_hw\m\+coder\+internal\+toolstrip\HardwareBoardHandler.m' at line 129
'C:\Program Files\Polyspace\R2019b\toolbox\coder\simulinkcoder\toolstripHW\m\+coder\+internal\+toolstripHW\CustomHardwareBoardHandler.m' at line 107
'C:\Program Files\Polyspace\R2019b\toolbox\coder\sltoolstrip_base_hw\m\+coder\+internal\+toolstrip\HardwareBoardContext.m' at line 84
'C:\Program Files\Polyspace\R2019b\toolbox\coder\sltoolstrip_base_hw\m\+coder\+internal\+toolstrip\HardwareBoardContext.m' at line 242'.
The RL Agent block only supports Normal and Accelerator simulation modes.
  2 Comments
Anirban
Anirban on 8 Dec 2020
Edited: Anirban on 8 Dec 2020
The paths in the message seem not right to me. They refer to C:\Program Files\Polyspace\R2019b\ whereas the paths should be C:\Program Files\MATLAB\R2019b\.
Do you happen to have a Polyspace installation and did you explicitly addpath folders from the Polyspace installation in your MATLAB? If so, you can check your path variable, then rmpath those folders and see if the errors go away. The steps to integrate Polyspace and MATLAB are given in Integrate Polyspace with MATLAB and Simulink, and if you follow these steps, there should not be a problem.
mehmet bozuyla
mehmet bozuyla on 8 Dec 2020
I tried the method which you mentioned on another computer and similarly got the following error again.
Error in 'rlwatertankControl/Coupled-Tank Simulation/RL Agent1': Initialization commands cannot be evaluated.
Caused by:
MATLAB System block 'rlwatertankControl/Coupled-Tank Simulation/RL Agent1/AgentWrapper' error occurred when invoking 'getSampleTime' method of 'AgentWrapper'. The error was thrown from '
'C:\Program Files\MATLAB\R2019b\toolbox\rl\rl\simulink\libs\AgentWrapper.m' at line 150
'C:\Program Files\MATLAB\R2019b\toolbox\rl\rl\simulink\libs\AgentWrapper.m' at line 210
'C:\Program Files\MATLAB\R2019b\toolbox\rl\rl\simulink\libs\AgentWrapper.m' at line 265
'C:\Program Files\MATLAB\R2019b\toolbox\rl\rl\simulink\libs\agentmaskinit.m' at line 11
'C:\Program Files\MATLAB\R2019b\toolbox\rtw\rtw\tlc_c.m' at line 199
'C:\Program Files\MATLAB\R2019b\toolbox\coder\simulinkcoder_core\+coder\+internal\ModelBuilder.m' at line 886
'C:\Program Files\MATLAB\R2019b\toolbox\coder\simulinkcoder_core\+coder\+internal\ModelBuilder.m' at line 218
'C:\Program Files\MATLAB\R2019b\toolbox\coder\simulinkcoder_core\+coder\+internal\@ModelCodegenMgr\make_rtw.m' at line 8
'C:\Program Files\MATLAB\R2019b\toolbox\coder\simulinkcoder_core\make_rtw.m' at line 20
'C:\Program Files\MATLAB\R2019b\toolbox\simulink\simulink\private\build_target.p' at line 0
'C:\Program Files\MATLAB\R2019b\toolbox\simulink\simulink\private\build_target.p' at line 0
'C:\Program Files\MATLAB\R2019b\toolbox\simulink\simulink\private\build_standalone_rtw_target.p' at line 0
'C:\Program Files\MATLAB\R2019b\toolbox\simulink\simulink\private\slbuild_private.p' at line 0
'C:\Program Files\MATLAB\R2019b\toolbox\simulink\simulink\private\slbuild_private.p' at line 0
'C:\Program Files\MATLAB\R2019b\toolbox\simulink\simulink\private\on_target_one_click.p' at line 0
'C:\Program Files\MATLAB\R2019b\toolbox\simulink\simulink\sl.m' at line 15
'C:\Program Files\MATLAB\R2019b\toolbox\coder\sltoolstrip_base_hw\m\+coder\+internal\+toolstrip\HardwareBoardHandler.m' at line 119
'C:\Program Files\MATLAB\R2019b\toolbox\coder\sltoolstrip_base_hw\m\+coder\+internal\+toolstrip\HardwareBoardHandler.m' at line 129
'C:\Program Files\MATLAB\R2019b\toolbox\coder\simulinkcoder\toolstripHW\m\+coder\+internal\+toolstripHW\CustomHardwareBoardHandler.m' at line 107
'C:\Program Files\MATLAB\R2019b\toolbox\coder\sltoolstrip_base_hw\m\+coder\+internal\+toolstrip\HardwareBoardContext.m' at line 84
'C:\Program Files\MATLAB\R2019b\toolbox\coder\sltoolstrip_base_hw\m\+coder\+internal\+toolstrip\HardwareBoardContext.m' at line 242'.
The RL Agent block only supports Normal and Accelerator simulation modes.

Sign in to comment.

Sign in to answer this question.

Answers (1)

Emmanouil Tzorakoleftherakis
The root of the error is likely on Reinforcement Learning Toolbox side, not Polyspace. As the last error line mentions,
"The RL Agent block only supports Normal and Accelerator simulation modes."

Categories

Find more on Reinforcement Learning in Help Center and File Exchange

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!