CarSim Unreal Engine Plugin Example using VS Connect

Mechanical Simulation 755 Phoenix Drive, Ann Arbor MI, 48108, USA Phone: 734 668-2930 Fax: 734 668-2877 Email: info@carsim.com Technical Memo carsim.com CarSim Unreal Engine Plugin Example using VS Connect Before You Start... 1 CarSim Setup... 2 Unreal Engine Setup... 3 Unreal Project Configuration... 4 Moving Object Vehicle Setting... 6 Simulink Model Setting... 7 VS Connect Setting... 9 How to Run the Example... 11 This memo describes an example simulation setup with a CarSim vehicle model running within an Unreal Engine environment and co-simulation with a Simulink model using. VS Connect is used for asynchronous data synchronization between the Unreal and Simulink environments. The example contains two vehicles; one is a Vehicle Sim moving object and the other is the ego vehicle. The ego vehicle follows the moving object and applies brake force as necessary to avoid colliding with the leading vehicle (the moving object). The vehicles run inside the Unreal Engine simulation environment and the brake controller logic is implemented in the Simulink model. The vehicle location data and the distance between the two vehicles is sent from Unreal Engine to the Simulink, and the required braking force is sent from Simulink to Unreal. Before You Start Before starting, there are a few requirements. 1. CarSim 2018.1 or later must be installed on your computer along with a valid license to enable it. 2. Unreal Engine version 4.19 or later must be installed from www.unrealengine.com 3. The CarSim Vehicle Dynamics Plugin for Unreal version 2018.1 or later must be installed from Epic Games market place. www.unrealengine.com/marketplace/carsimvehicle-dynamics 4. MATLAB 2010b or newer must be installed, including Simulink. 1 / 13 June 2018

CarSim Setup CarSim 2018.1 includes an example simulation run named *Unreal Engine 4 Plugin, Lead Vehicle in the category {External Control: COM, FMU, MATLAB, Python, VB} of the Run Control screen (Figure 1). Figure 1. CarSim example Dataset for Unreal Plugin. The vehicle in this example uses C-Class Hatchback with no shape. The animation shape is omitted because the vehicle shape is defined in the Unreal project. The braking dataset is set to Control with pedal force. In the Procedure, the road used is matched with the road geometry in the Unreal Engine project. Since the brake is controlled inside the Simulink model, the brake is set to 0. The variable speed from 20 to 25 kph is defined inside the Procedure. This speed is used by the moving object vehicle which is defined in. To use this example with Unreal Engine, the simfile needs be generated. The dataset named UE4 simfile generator is selected from Models: Transfer to Local Windows Directory. UE4 simfile generator contains the information shown in Figure 2. 2 / 13

Figure 2. UE4 simfile generator dataset screen. The working directory contains a period,.. When there is a period,., in this field, CarSim will create the simfile in the current database directory. 64-bit solver is selected, and simfile ue4simfile.sim is added in the yellow field to set the simfile file name. Go back to the Run Control screen and press the Generate File for this Run button. Verify that the file ue4simfile.sim is created in the CarSim database. Unreal Engine Setup This example assumes you have Unreal Engine 4.19.x installed in your PC, and that the CarSim Vehicle Dynamics plugin is installed in Unreal. The plugin can be downloaded from Unreal Engine Marketplace. For more information, refer to How to Setup the CarSim Plugin section in Unreal Engine CarSim Plugin Tutorial. You might need to manually enable the plugin after it is installed. A link to download the example project used here can be found on the plugin Marketplace page. This page can be found by searching the Marketplace for CarSim and click CarSim Vehicle Dynamics plugin icon. The link to the Example Project zip file is found at the bottom of the Technical Details section. Download the zip file and unzip into Unreal Project folder (e.g. C:\Users\<your user name>\documents\unreal Projects). Note You can save the Project to any directory. However, if it is saved in the Unreal Project folder, it will automatically show up in the Unreal Project Browser. 1. Launch Unreal Engine. 2. In the Unreal Project Browser, select Example_UE4_Project.uproject. There are two vehicles in this Unreal Engine project. One is an ego vehicle, and the other is a VehicleSim moving object. The ego vehicle follows the moving object, and applies the brakes when necessary. 3 / 13

Unreal Project Configuration To use the CarSim scenario/procedure described above with the Unreal example project, you must perform these configuration tasks: The simfile you created in the previous section must be specified inside the Unreal Engine project. The VS Connect object names for the two vehicles (ego vehicle, and moving object) must be set to link to the named objects in the Simulink setup. The ego vehicle must be instructed to use the CarSim driver model. The ego vehicle must be instructed to start at the location/orientation specified by the CarSim dataset (not the location where the shape file is placed in the Unreal Editor). The following steps describe how apply these settings. Figure 3. UE4 CarSim Example Project. 1. Left click to highlight MSC_DemoVehicle (Figure 3) from the World Outliner and scroll to the bottom of the Details panel and left-click to highlight the component named CarSimMovement (Inherited). This will display the properties of the CarSimMovement component. Specify the full path name of the 4 / 13

simfile you generated in the previous section (ue4simfile.sim) as the Vs Config File property. Tip To simplify filling in the VS Config File property, try this: In the CarSim browser open the Help About CarSim dialog and click on the underlined Working Directory: link. This will open the database folder with Windows File Explorer where you will find your simfile. Hold down shift on the keyboard and right-click the file, then select Copy as path from the pop-up menu. This copies the full path name of the file to the system clipboard. Now go back to the Unreal Editor and paste the path into the Vs Config File filed (by pressing Ctrl-V, or right click and select Paste ). 2. Click the checkbox on Use Vehicle Sim Start Position and Use Vehicle Sim Driver. 3. It is a good idea to click the checkbox on Auto Reset to Start on Vs Solver Crash so the vehicle will automatically reset to the original position if the CarSim Solver encounters an error. Figure 4. VS Connect communication settings. 4. Still highlighting MSC_DemoVehicle from the World Outliner, left click to select the VsConnect component (Figure 4)from the component list in the Details tab. 5 / 13

5. Enter VS Vehicle 0 on VS Connect Object Name. Note that this field is case-sensitive in this example. Click the checkbox on Auto Register with VS Connect. Moving Object Vehicle Setting Next, you need to set the VS Connect Object Name of the moving object lead vehicle. 1. Go to the World Outliner tab and left click to highlight A_MovingObjectVehicle (Figure 5). Go to the Details tab and left click to highlight VsConnect. 2. Enter Moving_Car as the VS Connect Object Name. Ensure that the Auto Register with VsConnect option is enabled. Figure 5. Moving Object Vehicle Settings. 6 / 13

Simulink Model Setting In the previous section, you have specified which CarSim simfile to use in Unreal Engine and what names of the objects are used for VS Connect. In this section, the Simulink model settings are discussed. In this example, the locations of the vehicles and the distance between the ego vehicle and the moving object are obtained from Unreal Engine via VS Connect and determine when and how much to apply the brake force and pass that information to CarSim. The example Simulink file is in: {CarSim Database} > Extensions > Simulink > vs_connect> And the filename is: vs_connect_moving_object_2018_1_matlab_r2010b.mdl. It is very important that the MATLAB Current Folder is pointing to this vs_connect folder especially if you are using older version of MATLAB. Instead of double-clicking to open *.mdl file, the recommended method is: 1. Open MATLAB using the Windows Start Menu. 2. Point the MATLAB Current Folder to the vs_connect folder within your CarSim database (Figure 6). 3. Double-click vs_connect_moving_object_2018_1_matlab_r2010b.mdl file to open the Simulink model (Figure 6). Figure 6. Set MATLAB Current Folder to vs_connect. When you open the Simulink model (Figure 7), you will find vs_connect_sf2 block. If you double-click to open this block (Figure 8), you notice that S-function parameters is set to moving_object_config.json. This *_config.json file controls both the data that will be communicated to/from Unreal Engine and the CarSim solver, and the input/output ports that are available on the S-function block in the Simulink model. The next section discusses about the *_config.json file settings. 7 / 13

Figure 7. The Simulink model with vs_connect_sf2 block. 8 / 13

Figure 8. VS Connect S-Function2 Block. In the Simulink model, there are several outputs from vs_connect_sf2 block such as the vehicle location (Figure 7) and the moving car location. For the brake control, the vehicle velocity and the distance to moving object (from the ego vehicle) are used. Once the brake force is calculated, it passes the information back to vs_connect_sf2 block. Note that the distance and the velocity threshold values are hardcoded in the Simulink model. If the users want to copy this model and use it as their own, an adjustment may be needed. When the link between the VS Connect S-function in Simulink and the VS Connect server in the Unreal plugin has been established, you will see the number 3 displayed in. For more information of the connection status, please refer to VS Connect API documentation. VS Connect Setting In the first two sections, you have generated the simfile in CarSim and then specified which simfile to use in Unreal Engine. To co-simulate Unreal Engine/CarSim and the Simulink model, VS Connect S-Function block is used in the Simulink model. The S-Function block uses *_config.json file which defines the inputs/outputs of S-Function. Additional details about the configuration and operation of the VS Connect S-function block can be found in the VS Connect Simulink Block tech memo (vs_connect_s_function.pdf). In this example, the file called moving_object_config.json is used as discussed in the previous section. This configuration file can be found in the CarSim database: 9 / 13

CarSim Database} > Extensions > Simulink > vs_connect When you open moving_object_config.json file, you will see the remoteip and remoteobject on the third and fourth lines of the file. remoteobject is set to VS Vehicle 0 (Figure 9), which should match with the name give in Step 5 of Unreal Project Configuration. Figure 9. Inside moving_object_config.json file. You will see outgoingcontracts and incomingcontracts sections. You can add the solver variables in these sections. In this example, there is only one solver import variable for outgoingcontracts, IMP_FBK_PDL. If there are more than one import variables, you can use a curly bracket, { }, and add more variables. Inside outgoingcontracts, you again find remoteobject which is set to VS Vehicle 0. The "remoteobject": "VS Vehicle 0" on the fourth line acts as a default. Therefore, "remoteobject": "VS Vehicle 0" inside outgoingcontracts is redundant and unnecessary. 10 / 13

"datatype": "FLOAT" and "datasize": 64 are also the default settings. updateperiodms indicates the minimum update period (time between updates). For example, if this is set to 8, the minimum time between updates 8 milliseconds, which is about 120 fps (1/120 = 0.00833). Note that even though you can enter numbers smaller than 8 milliseconds (for example, 1 millisecond), the actual update rate will be limited by the simulation rate in the Unreal Engine simulation, which in turn may be limited by the refresh rate of the monitor, such as 17ms (60 Hz), 8ms (120 Hz), etc. How to Run the Example Assuming all the settings are done by going through previous sections, follow the steps below to run the example. 1. Open CarSim. If you have not generated the simfile, push the Generate Files for this Run button to generate it. 2. If MATLAB/Simulink is open, close it. 3. Open Example_UE4_Project in Unreal Engine 4.19.x. 4. Manually open vs_connect_moving_object_2018_1_matlab_r2010b.mdl Do not open this Simulink model from CarSim, open it manually in MATLAB as described in the Simulink Model Setting section above. 5. Go back to Unreal Engine, and click Window > Developer Tools > Output log to open the output log tab (Figure 10). 11 / 13

Figure 10. How to open output log in Unreal Engine. 6. Press Play button in Unreal Engine (Figure 11). Figure 11. Unreal Engine play button. 7. Immediately after you pushed the Unreal Engine play button, press the Simulink green play button. Figure 12. Simulink model play button. 12 / 13

8. You should see some output values display in the Simulink model, and see VS Connect message in Unreal Engine Output Log. The ego vehicle should apply the brake when it approaches to the moving object vehicle too close. Tip: The Unreal Editor output log fills up quickly with lots of messages you may not be interested in. Try typing CarSim or VS Connect into the Output Log window s Filters field to limit the messages to only those related to the CarSim plugin, or VS Connect, respectively. 9. You probably notice that the animation looks somewhat slow in Unreal Engine. By default, the Unreal Editor slows down the frame rate dramatically when it is not the active/focused application. Click the Viewport in Unreal Engine while the animation is running. This should make the animation smooth. Tip: The Play In Editor (PIE) mode of the Unreal Editor (when the simulation is run within the editor) captures the mouse and hides the mouse cursor. To release the mouse and allow the simulation to continue, press Shift-F1. You can also press Alt-Tab to switch to another application (e.g. Simulink), which will release the mouse. 10. Click on the Unreal Editor viewport and press Escape to stop the simulation. If you see two vehicles but you don t see any values in the Simulink model or you do not see the brake is applied, stop the Simulink model and restart it. If you see license error messages in the Unreal output log, it means that you did not have the CarSim browser (or CSLM.exe) running with the Solver License enabled when the Unreal Editor was launched. Close the Unreal Editor, ensure that CarSim or CSLM is running, then restart the Unreal Editor. 13 / 13