How do display visuals of one player on multiple monitors, while using Unreal Engine and MATLAB Simulink to start the program?

I am using MATLAB R2021b and Unreal Engine. How can I split the visuals of a single person driving in a driving simulator, which operates on Unreal Engine, and display them on two screens?

[Nishan Nekoo]

Hi Kaitlyn, can you clarify what you mean by splitting the visuals?

Do you mean mirror the same display on two different screens, or do you mean to show some parts of the scene in one window, and other parts on a different window?

What are you trying to accomplish by "splitting the visuals"?

[Kaitlyn Woolf]

Hi Nishan,

I wish to show some parts of the scene in one window and other parts on a different window.

I want to take the real-time scenes of a single driver and display them across multiple (two) monitor screens.

The purpose of this is that eventually I want to use multiple projectors to project the scene on a curved projection screen, and this seems like it is a good initial step, considering it does not involve edgeblending and etc.

[Kaitlyn Woolf]

Excuse me

I wish to show some parts of the scene on one monitor and other parts on a different monitor.

Answer 1

[MathWorks Support Team]

To display visuals of a single person driving on multiple monitors using MATLAB R2021b and Unreal Engine you can use one of the following two workflows:

Using the Simulink 3D Animation Toolbox:

You can do this by placing two "Simulation 3D Camera" blocks in your Simulink model and adjust the parameters so that they cover the parts of the scene that you desire. To visualize the camera images that are output by the "Image" port of that block, use a "Video Viewer" or a "To Video Display" block. This will open the view from each camera in a separate window.

Using the Image Processing Toolbox and the Image Acquisition Toolbox:

You can do this by following the steps below:

1. Set up the driving simulation in Unreal Engine and ensure it is working correctly with a single screen.
2. Use the MATLAB Simulink Integration with Unreal Engine to capture the rendered frames. You can use the "From Video Device" block in the "Image Acquisition Toolbox" library to capture the frames as an input signal. 
3. Display each stream on its respective screen using the "imshow" function. Use the "ScreenNumber" Name-Value pair to specify the monitor where you want each stream to be displayed (0 represents the primary monitor and higher numbers represent additional monitors). 

The following code will display the driving simulation visual on two screens: 

Please note that this code example assumes two screens connected to the primary computer, and that each half needs to be displayed on a separate screen.

% Capture frames from Unreal Engine using Simulink
videoInput = Simulink.Signal;
videoInput.DataType = 'uint8';
videoInput.SampleTime = -1;
videoInput.Complexity = 'real';
videoInput.Name = 'VideoInput';

% Split the frames into two halves
frameWidth = size(videoInput, 2);
halfWidth = frameWidth / 2;
leftHalf = videoInput(:, 1:halfWidth);
rightHalf = videoInput(:, halfWidth+1:end);

% Display each half on its respective screen
figure('Position', [0 0 halfWidth size(videoInput, 1)]);
imshow(leftHalf, 'InitialMagnification', 'fit', 'Border', 'tight', 'ScreenNumber', 1);

figure('Position', [halfWidth 0 halfWidth size(videoInput, 1)]);
imshow(rightHalf, 'InitialMagnification', 'fit', 'Border', 'tight', 'ScreenNumber', 2);

For a different monitor configuration or to split the visuals in a different way, the code can be modified accordingly.

Answer 2

[Nishan Nekoo]

Kaitlyn, one way to do this might be to place two Simulation 3D Camera blocks in your Simulink model and adjust the parameters so that they cover the parts of the scene that you desire.

To visualize the camera images that are output by the Image port of that block, use a Video Viewer or To Video Display block. This will open the view from each camera in a separate window.