Connect a Physical Model in Simulink to Unreal Engine | AUV Deep Dive, Part 6

Hello, this video picks up where we left off with our Unreal Engine 4 environment. You can also jump in here if you have an existing 3D scene. The very first thing I want to do is make sure that my submarine CAD model has been imported to the scene. Of course, it will still be possible to migrate or import such models after we complete the Simulink coupling but it is convenient to do as much as possible while we can quickly switch between Unreal Engine 4 projects without going through MATLAB.

When we're happy, we can go to the plug ins and enable the MATLAB plug in. This plug in will appear in this list after you perform the setup steps for in my case, the vehicle dynamics blocks the interface for Unreal Engine 4. Notice that this says it requires a restart. Now if we tried to open that project directly from Unreal Engine, there might be some errors that's normal as it's now coupled with Simulink.

At this point, we can retreat to the safety of the Simulink Canvas. Here we have the top of a model containing the physical system model and control systems model for our underwater vehicle. On the right hand side there is a nice system set up for us. Inside we find the transformative rotation for the underwater vehicle. My colleagues have provided transform in meters and rotation in both radians and degrees.

This is perfect to get started with our visualization, we need to complete the connection between Simulink in Unreal Engine 4 which we started by enabling the plug in on the Unreal side. From Simulink we can open up the library browser. Scroll down a vehicle dynamics block set, open up vehicle scenarios, send 3D core, and then get the simulation 3D scenes configuration block.

This block represents the connection to Unreal Engine on the Simulink side. Opening up the block parameters, we see scene source is set to default scenes. These are targeted at the automotive industry and not very applicable to underwater simulations. Instead we can set this Unreal Editor then we will point the project to our underwater scenes that you project file which was created by Unreal Engine.

Now if we click Apply and open an Unreal Editor, we should be greed with a scene from earlier. Looks good, we need to finalize this connection on the Unreal Engine side. We're going to go to blueprints, open level blueprint, we can go to file and repair this blueprint to the simulation 3D level script actor. Now this level and all of the undercuts the hood stuff on the Unreal Engine side will inherit the necessary functionality and the plug in develop by MathWarks.

All the things in this list are generally geared towards the automotive industry or UAV's instead, we want to simulate a submersible. To do this, we're going to go over to the few options on the content pane and make sure engine content and plug in content are checkbox as well as developers content then in the content browser on the left hand side we're going to scroll down until we see math or simulation content.

We'll open up MathWorks simulation C+ plus classes, MathWorks simulation, and then public. In here we have the base classes, I'll right click Same 3D actor and create a blueprint class based on this, we can name this submarine and put it in the content folder. Creating this blueprint class will open up this screen. On the left hand side, we can add a component specifically, a static mesh called AV body. On the right hand side with the static mesh selected, we can specify that this VAV combined model.

When we drag this actor into the scene it looks like a submarine. Now we're going to set up our camera and LiDAR which in this case will simulate sonar. Search for camera and find the 3D camera drag this into the scene. Additionally we're on a search for LiDAR and drag the same 3D LiDAR into the scene.

Finally, to create that isometric view we'll take a sphere and drag it into the scene. So we're going to assemble these in the world outliner. First I'm going to drag the camera, the LiDAR, and the sphere onto the submarine. For the sphere, I'll need to check movable here to let it know that it can move around the environment during the simulation.

Next I will attach the camera to the sphere then I can create that isometric view by going to the sphere, going up to the transform, going to the rotation selecting world this will lock the rotation of the sphere and the camera that is attached to it. Once we have all the actors in our scene, it's important to hide them so that you don't see a large sphere or rotating light are during your simulation. To do this, simply click the object in the world outliner and search for hidden in the details. We can check the actor hidden in game here to hide the sphere easily.

Hiding the LiDAR is a bit more complicated. After selecting the LiDAR and the world outliner will open up the level blueprint. This is the finished product. To get there, we right click the Canvas search for actor hidden in game. With the LiDAR are selected in the world outliner it should automatically appear as a target. We can now specify that the new hidden value is true and connect it to the event began play. This will hide the liter as soon as the scene begins simulating.

Moving back to the Simulink Canvas we can set up our actor transform set block. This is located in the same folder we found the simulation 3D scene configuration block in a library browser. In this case, the rotation can be fed directly in. However, the translation needs to be transformed slightly this block is included in the model that I'll provide you. Here we just need to invert the z-axis.

Next, we need to supply a scale. In this case scale can just be a constant block. We need to specify scale in terms of x, y, and z dimensions and then feed it in. At this time, we can also set up the camera. This is the simulation 3D camera get block is also part of the vehicle dynamics blocks set.

The final block we need is in the automated driving toolbox, simulation 3D, simulation 3D LiDAR. To display the image from the camera we're going to use a video viewer blog which I can just search for on the Canvas or find in the computer vision toolbox sinks blocks library.

To visualize the light we'll need a special block I've put one together using a MATLAB function block. We need to tell each of these blocks which actors or elements of our Unreal Engine simulation they're going to get data from. To do this, we use tags. The properties here indicate that the tag name here is Simulink actor 1. Instead I'm going to call this sub 1.

With respect to the camera and the LiDAR, the tag name is set by a formula. That formula is sensor type and then sensor identification vocation number. In this case, our camera sensor identification number is one and for our LiDAR the sensor identification number is two. This will increase for every new sensor we add, for example, copying and pasting a camera result in a new sensor identification number 3. We can now set the parameters for the camera and a LiDAR. For more information on these parameters please refer to the documentation for each of these blocks.

Now we need to go to Unreal Engine and make sure the tags are defined appropriately. So bring our scene back up and click on our submarine here. In Simulink remember we tagged this as sub 1. So I'm going to add a tag and identify that as sub 1. For the camera the tag is camera 1.

We also need to apply the same tag to the LiDAR. First will select a LiDAR in the world outliner. Then we'll search details for tag. Notice there's a lot of things that come up, however we're looking for the actor tag, in here will add an array and we'll call this LiDAR 2 according to the formula.

Now everything should be set up. So we can move back to simulate and try to run our scene. Now that we see initializing in the bottom left hand corner, we can go back to Unreal Editor and hit Play. After a brief delay we should see the submersible start moving in the 3D environment. On the right hand side, we can see the output from the LiDAR are providing a 3D point cloud of the seabed including all of the rocks.

There are a few options we have to try to improve the performance of this scene. We could replace the ocean surface, for underwater simulations this isn't critical. Second we should build the scene, this will back in the lighting and improve the performance overall as Unreal Engine does some optimisations. The third and final option is to package the scene in an executable. To do this will navigate the final, package project, and select your operating system. In my case windows and windows 64 bit.

Then we can move back to the Simulink Canvas, select the simulation 3D scene configuration block and switch the scene source from editor to Unreal executable. Then we would specify the file path to the Unreal executable we just packaged. This concludes the tutorial on how to set up and put together our Simulink and 3D environment simulation.