DELTAGEN STELLAR. DISTRIBUTED RENDERING - CLUSTER SETUP Administration Guide

Transcription

1 DELTAGEN STELLAR DISTRIBUTED RENDERING - CLUSTER SETUP Administration Guide

2 CONTENTS Prerequisites 2 Overview 2 Setup 2 Set up Artifacts 3 Cluster Setup 3 Cluster Startup 3 Start and Shutdown Order 3 Remote Node 3 Gateway Service 4 4 Client Setup 4 Reference 4 Remote Node 4 Synopsis 4 4 Synopsis 5 5 Batch Renderer 6 Executing the Batch Renderer 6 6 Parallel distributed Batch Rendering 6 The batch renderer can also connect to an existing cluster to accelerate the rendering. For this a cluster of render_node must be started first (see previous section to see how to configure and start a cluster). Then the batch renderer can connect to the cluster the following way: Distributed STELLAR in a multi cell cluster environment 8 Definition 8 Setting up a multi cell cluster environment for Distributed STELLAR 8 Setting up a set of connected Remote Nodes in each of the cell networks 9 9 Setting up a Bridge Broker Node for one of the cell networks 9 9 Setting up other Broker Nodes for all other cell networks 9 9 Connect the Distributed Stellar application to the bridge Broker Node 10 Synopsis DELTAGEN Stellar - Distributed Rendering - Cluster Setup Administration Guide 1

3 PREREQUISITES Distributed Stellar binaries can be executed on the following systems: Operating systems Supported Versions Additional requirements Microsoft Windows Windows 7 64bit version Windows 10 Visual Studio 2015 Redistributable installed Patch KB installed 64bit version Visual Studio 2015 Redistributable installed Linux CentOS/RedHat Enterprise Linux bit version package libgomp or later Suse Linux Enterprise System 11 SP3 64bit version package libgomp r or later OVERVIEW SETUP Gateway Service allows client applications like DELTAGEN to connect to a remote cluster where the Worker Nodes are invisible from outside networks. Connections are established via a head node that is visible from the outside. The service forwards rendering requests to Remote Node services. It may act as a Bridge Node for establishing a cluster of Remote Node services. As a DELTAGEN user you can configure interactive rendering such that your view is also shown in a web-browser on a separate computer and screen. The gateway therefore can send streams to stream clients. It only acts as a streamer to the browser if configured to do so. Remote Node is a service for distributed computation on Cluster Nodes. It turns a cluster node into a distributed computer that processes tasks that operate on distributed data as specified by the DStellar client. Remote Node services set up a P2P network in order to form a cluster. A Remote Node must establish an initial connection to one other Remote Node or Gateway Service to join. We call this initial entry Bridge Node for this service. Its end-point needs to be specified when starting a Remote Node. The startup order of the services is however arbitrary as connections to the Bridge Node are retried. DELTAGEN Stellar - Distributed Rendering - Cluster Setup Administration Guide 2

4 Set up Artifacts The following binaries necessary for the setup can be found in the folder DELTAGEN_Distributed_ Stellar, located in the installer source folder. CloudGate <win64/linux> RemoteNodes <win64/linux> RenderExecutables BatchRenderer <win64/linux> GatewayService binaries and dependencies RemoteNode binaries and dependencies Plug-ins for the distribution system BatchRenderer binaries and dependencies Cluster Setup 1. EXTRACT THE REMOTENODES ARCHIVE TO A FILE SYSTEM ACCESSIBLE FROM THE CLUSTER MACHINES 2. EXTRACT THE BATCHRENDERER ARCHIVE TO A FILE SYSTEM ACCESSIBLE FROM A CLUSTER HEAD NODE 3. EXTRACT THE CLOUDGATE ARCHIVE TO A FILE SYSTEM ACCESSIBLE FROM A CLUSTER HEAD NODE 4. EXTRACT THE RENDER_EXECUTABLES ARCHIVE TO A FILE SYSTEM ACCESSIBLE FROM A CLUSTER HEAD NODE The first two folders contain two executables (remote_node, cloudgate_server) and their dependencies for 64-bit Linux or Windows operating systems. The executables can be started from a file system without further installation. The render_executables folder contains zip archives and.md5 hashes for plug-ins. They do not have to be extracted; they will be automatically deployed to the Worker Nodes as needed by the distribution system. Both, Stellar and DStellar require to have a temporary folder where their dependencies are stored. 1. CREATE A TEMPORARY DIRECTORY 2. SET THE ENVIRONMENT VARIABLE TMPDIR TO POINT IT Example: export $TMPDIR= ~/temp It is important to note that each of the Nodes (Head Node and Worker Nodes) must have their own Temp folder and its TMPDIR variable set to it. It is strongly recommended to have the Temp folder in the local disk space of each node rather than on a shared file system since it will degrade the performance the in later case. CLUSTER STARTUP Start and shutdown order The starting process follows the given order: The distributed rendering system is started on a cluster by launching first one Remote Node as Bridge Node. The Remote Node is started on all remaining Worker Nodes which should connect to the Bridge Node. The Cloudgate Server is started and connects to the Bridge Node. For a clean shutdown of the rendering service, all the started Remote Node processes and the Cloudegate Server process have to be terminated by sending them a SIGTERM signal. Remote node 1. START A FIRST REMOTE_NODE AS BRIDGE NODE Example: remote_node -l START REMOTE_NODE ON ALL REMAINING WORKER NODES AND SPECIFY THE BRIDGE NODE AS ENTRY FOR JOINING THE CLUSTER Example: remote_node -l 9000 h <bridge_node hostname or IP address> p 9000 DELTAGEN Stellar - Distributed Rendering - Cluster Setup Administration Guide 3

5 Option -l 9000 Configures the port remote_node listens on for connections from other remote_node services and the cloudgate_server. -p 9000 Configures the port for initial connection to the cluster - use the Bridge Node port. -h <Bridge Node host name or Configures the host for initial connection to the cluster - use the IP address> Bridge Node host name. Gateway service START THE GATEWAY SERVICE ON A HEAD NODE AND CONNECT IT TO THE BRIDGE NODE Example: Option cloudgate_server -s l 0 -h <Bridge Node host name or IP address> -p v 0 -e <task executable path> -I <host name of gateway node external interface> -s 5000 Specifies the service port for connections from DStellar client applications and streaming clients. -l 0 Configures the port cloudgate_server listens on for connections from other remote_node services. If set to 0 a free port is automatically chosen. -v 0/1 Configures the service to use IPv4 for the service port if set to 0. If set to 1 it activates IPv6. -e <task executable path> Configures the location of plug-ins for the distribution system. Set it to the task_executables folder containing render_executablelinux64-release.zip/.md5. -h <Bridge Node host name> Configures the host for initial connection to the cluster - use the Bridge Node host name. -p 9000 Configures the port for initial connection to the cluster - use the -I <host name of gateway node external interface> Bridge Node port. Configures the interface where the cloudgate can be reached from the outside. Client setup See the 3DEXCITE DELTAGEN Installation Guide. REFERENCE Remote node Remote Node is a service turning a machine into a distributed computer, used by Stellar clients for distributed rendering. START REMOTE_NODE ON WORKER NODES IN THE CLUSTER Synopsis remote_node [OPTION]... Program options --help Produces a help message. -v [--version ] Displays the version information. -l [ --localport ] Is the local listening port. -h [ --host ] Is the hostname of the Bridge Node used to connect to the cluster. -p [ --port ] Is the port of the Bridge Node used to connect to the cluster. Default is 0. This does not connect to bridge node. DELTAGEN Stellar - Distributed Rendering - Cluster Setup Administration Guide 4

6 Advanced options -i [ --interface ] Is the local network interface to listen on. -w [--worker ] Is the flag telling if the node will execute work package (true by default). -m [ --monitoringport ] Is the monitoring port (limited feature). Default is 0. -L [ --logging_config_path ] Is the path of the config file for the logging. Default is logging.conf. -d [ --dist_config_path ] arg Is the path of the config file for distribution service. Default is node_config.json. Gateway service NOTE: The system is using the default network interface. Only if multiple network interfaces are provided should you specify the correct one. Otherwise there is no need to specify these options. This is a service allowing Stellar clients to connect to a remote cluster via one host and wellknown service port. START CLOUDGATE_SERVER ON A HEAD NODE OF THE CLUSTER This can be set up for example on a node that is visible from the outside and is able to resolve the remote_node services running on Worker Nodes. Synopsis cloudgate_server [OPTION]... Program options --help Produces a help message. -v [ --version ] Display the version information. -l [ --localport ] Is the cluster-internal listening port. -s [ --websocketport ] Is the service (web-socket server) listening port. -I [ --external_interface ] Defines the interface where the cloudgate can be reached from the outside - usually set to the machine hostname. Default is -hostname. -e [ --execpath ] Is the directory the executables are located in. Default value is./ -i [ --interface ] Is the cluster-internal network interface to listen for connections from remote_node services. -h [ --host ] Is the host name of the Bridge Node used to connect to the cluster. -p [ --port ] Is the port of the Bridge Node used to connect to the cluster. The default is 0. Advanced options -L [ --logging_config_path ] Is the path of the config file for the logging. Default is logging.conf. -a [--activemonitoringproxy ] If set to true, then monitor agents can use the cloud gate server as a proxy to connect to the cluster. Default is false. -m [ --monitoringport ] Is the monitoring port. The default is 0. -c [ --dist_config_path ] Is the path of the config file for distribution service. -n [ --websocketpingtimeout ] Sets the time in [ms] when a ping is timed out. The default is o [ --websocketconnectiontimeout ] Sets the time in [ms] after a connection is considered being dead. The default is w [ --websocketuseipv6 ] Sets the use of IPv6 protocol. The default is 1. -x [ --websocketmaxmessagesize ] Is the maximum sendable message size in [bytes]. Default is d [ --docroot ] Is hosted documents root folder - contains a stream client web-application. Default is./docroot. NOTE: The options - L, - m, -c, -n, -0, -x, and -d are optional settings and currently not needed to be specified. DELTAGEN Stellar - Distributed Rendering - Cluster Setup Administration Guide 5

7 BATCH RENDERER DELTAGEN allows exporting batch jobs and ships a batch renderer command-line program for executing such jobs. This batch renderer can easily be run under the control of a job scheduler or a render manager. It can use a cluster of Worker Nodes to speed-up a job by employing parallel distributed rendering. Using the batch renderer as a parallel program allows accelerating both: the rendering of single images and image sequences. This requires connecting the batch renderer to a set of Worker Nodes running the remote node service. Alternatively the batch renderer can be run alone without additional workers. This mode may be advantageous if cluster utilization is high and not enough Nodes are available at the same time for parallel distributed rendering. By splitting a job at frame boundaries and scheduling each frame sub-job using the batch renderer in non-distributed mode, the rendering of image sequences may still get accelerated. Executing the batch renderer The batch renderer can be executed as a stand-alone console application on a single machine in the following way: Option batch_renderer b <batch job file> -e <task executable path> -r <output directory> -b <batch job file> Specifies the path of the batch job file that was produced by DELTAGEN Render Export. -e <task executable path> Configures the location of plug-ins for the distribution system. Set it to the task_executables folder containing render_executablelinux64-release.zip/.md5. -r < output directory > Specifies the path where the output images will be stored. Once started the batch renderer reads the content of the batch job file, renders the different job, and stores the resulting images in the specified directory. Parallel distributed batch rendering The batch renderer can also connect to an existing cluster to accelerate the rendering. For this a cluster of render_node must be started first (see previous section to see how to configure and start a cluster). Then the batch renderer can connect to the cluster the following way: batch_renderer b <batch job file> h <bridge_node hostname or IP address> p e <task executable path> -r <output directory> Option -b <batch job file> Specifies the path of the batch job file that was produced by DELTAGEN Render Export. -e <task executable path> Configures the location of plug-ins for the distribution system. Set it to the task_executables folder containing render_executablelinux64-release.zip/.md5. -r < output directory > Specifies the path where the output images will be stored. -p 9000 Configures the port for initial connection to the cluster - use the Bridge Node port. -h <Bridge Node host name Configures the host for initial connection to the cluster - use the or IP address> Bridge Node host name. NOTE: The options -L, -i, -l, -h, -p, -c, -w, -s, -f, -x, -y, -t, -m, -o and -r are optional settings and can be omitted. DELTAGEN Stellar - Distributed Rendering - Cluster Setup Administration Guide 6

8 Reference batch renderer Synopsis batch_renderer [OPTION] Program options --help Produces a help message. -v [--version ] Displays the version information. -b [ --batchjobpath ] Is the path of the batch job file. -e [ --execpath ] Is the directory the task executable plug-ins are located in. Default is current directory.. -h [ --host ] Is the host name of the bridge node to connect to when using a cluster. -p [ --port ] Is the port of the bridge node to connect to when using a cluster. Default is 0. -w [ --worker ] Is the worker flag. If set to 1 this node participates in rendering, otherwise rendering is only done on connected Worker Nodes. Default is 0. -M [ --meta_info_output ] Is the path (including file name) of the job meta information file. If specified status for every frame of the job is written to the file. -o [ --time_limit ] Is the time limit (in minutes) for executing the batch job. The job is aborted after the specified time elapses. Default is 1. Advanced options -s [ --start_frame ] Sets the start frame of the batch rendering. Default is 0. -f [ --end_frame ] Sets the end frame of the batch rendering. Default is x [ --override_image_ Sets the override image resolution(x) of the batch rendering. resolution_x ] Default is 1. -y [ --override_image_ Sets the override image resolution(y) of the batch rendering. resolution_y ] Default is 1. -t [ --override_iterations ] Sets overriding iterations (target quality) of batch rendering. Default is 1. -r [ --rendered_job_file_path ] Is the output folder where the rendered images will be stored. Default is./. -L [ --logging_config_path ] Is the path of the config file for the logging. Default is logging. conf. -i [ --interface ] Is the local network interface name or IP address to listen on for connections from Worker Nodes in a cluster. -l [ --localport ] Is the local listening port for connections from Worker Nodes in a cluster. A free port is automatically chosen if set to 0. Default is 0. -c [ --dist_config_path ] Is the path of the distribution service configuration file. -m [--monitoringport ] Is the monitoring port. Default is 0. -I [ --initial_interval_ Sets the initial intermediate image update interval duration in ms. duration_ms ] Default is F [--final_interval_duration_ms ] Sets the initial intermediate image update interval duration in ms. Default is T [--nb_render_threads ] Is the number of threads used by renderer. When set to 0 Stellar will use all the resources ( by default).when set to < 0,Stellar will use all threads minus numthreads. When set to > 0, it will use numthreads threads. This parameter does not influence Worker Nodes. See remote_node command line help to configure this on Worker Nodes. DELTAGEN Stellar - Distributed Rendering - Cluster Setup Administration Guide 7

9 DISTRIBUTED STELLAR IN A MULTI CELL CLUSTER ENVIRONMENT Definition A Broker Node is a Bridge Node forwarding tasks and data from an Originator Network to a Substitute Network and task results from the Substitute Network to the Originator Network. An Originator Network is a network connecting one or more Broker Nodes and the application. A Substitute Network is network connecting Worker Nodes and a Broker Node, running on cluster cells. The Broker Node runs on the head node of the cell, connecting the Substitute Network to the Originator Network. Broker Nodes, similarly to Gateway Services, allow Distributed Stellar applications to connect to a remote cluster, where the Worker Nodes are invisible from outside networks. While a Gateway Service is used in situation, where the network between the application and the Gateway Service is potentially having low bandwidth and high latency, Broker Nodes require a fast dedicated network to connect to the application. In turn, the Distributed Stellar application is able to connect to multiple Broker Nodes at once. This allows the application to use the computational power of multiple, not interconnected cluster networks (cells, Substitute Networks), which are connected by one Originator Network. Setting up a multi cell cluster environment for Distributed STELLAR This section describes how to setup Distributed STELLAR on the following example network: In this simple setup there are two small cells with a Head Node each. The Application Node cannot directly access any of the Cluster Nodes. There are example IP addresses next to each node. The Head Nodes have two IP addresses, one identifying the network adapter connected to the cell, the other one identifying the network adapter connectable from the Application Node. DELTAGEN Stellar - Distributed Rendering - Cluster Setup Administration Guide 8

10 Setting up a set of connected Remote Nodes in each of the cell networks 1. START BRIDGE REMOTE NODES ON EACH CELL CLUSTER, ON CLUSTER NODE 1 AND CLUSTER NODE 3 EXECUTE: remote_node l LAUNCH REMOTE NODES ON THE OTHER CELL CLUSTER NODES, CONNECTING TO THE BRIDGE REMOTE NODE ON CLUSTER NODE 2 AND CLUSTER NODE 4: remote_node l 0 h p l 9000 Configures the port Remote Node listens on the network for connections from other Remote Node, Broker Node services or the application. h Configures the host for initial connection to the network cluster, in this case we try to connect to the Bridge Remote Node. -p 9000 Configures the port for initial connection to the network cluster, we specify the bridge Remote Node listening port here. -l 0 No connections expected here, let remote node use an automatic port. Setting up a Bridge Broker Node for one of the cell networks ON CELL HEAD NODE 1 SET: broker_node l 9001 i H P 9000 I L 0 -l 9001 Configures the port Broker Node listens on the Originator Network for connections from other Broker Node services and the application. i The IP of the network adapter used to connect the Broker Node to the Originator Network. -P 9000 Configures the port for initial connection to the cluster on the Substitute Network side - the Bridge Node port is used. -H Configures the host for initial connection to the Substitute Networks cluster, Cluster Node 1 s IP is used. I The IP of the network adapter used to connect the Broker Node to the Substitute Network. -L 0 Configures the port Broker Node listens on the Substitute Network for connections from remote node services. 0 means automatic. Setting up other Broker Nodes for all other cell networks ON CELL HEAD NODE 2 SET: broker_node l 0 i h p 9001 H P 9000 I L 0 -h Configures the host for initial connection to the Originator Networks cluster. We use the Bridge Broker Node s IP address. p 9001 Configures port for initial connection to cluster on Originator Network side; the Bridge Broker Node port on the Originator side 9001 is used. -l 0 We do not expect any incoming connections from the originator side here, so just let the Broker Node use an automatic port. i The IP of the network adapter used to connect the Broker Node to the Originator Network. -P 9000 Configures the port for initial connection to the cluster on the Substitute Network side - the Bridge Node port is used. -H Configures the host for initial connection to the Substitute Networks cluster, Cluster Node 1 s IP is used. DELTAGEN Stellar - Distributed Rendering - Cluster Setup Administration Guide 9

11 I The IP of the network adapter used to connect the Broker Node to the Substitute Network. -L 0 Configures the port Broker Node listens on the Substitute Network for connections from remote node services. 0 means automatic. Connect the Distributed Stellar application to the bridge Broker Node At this point the cluster is ready to receive connections and execute tasks from any Distributed STELLAR application. The application has to connect to the Bridge Broker Node the same way as you would connect it to a Remote Node, using as IP and 9001 as port. See the 3DEXCITE DELTAGEN Installation Guide for details on how to do that with DELTAGEN. Broker Node reference Synopsis broker_node [OPTION]... Program options --help Produces a help message. -v [--version ] Displays the version information. Originator interface options -l [ --originator_port ] Port where the Originator listen on. -i [ --originator_interface ] Network interface where the Originator listen on. -h [ --originator_bridge_host ] Hostname of the bridge node the Originator connects to. -p [ --originator_bridge_port ] Port of the bridge node the Originator connects to. Default is Substitute interface options 0 (we do not connect to a bridge node). -L [ --substitute_port ] Port where the Substitute listen on. -I [ --substitute_interface ] Network interface where the Substitute listen on. -H [ --substitute_bridge_host ] Hostname of the bridge node the Substitute connects to. -P [ --substitute_bridge_port ] Port of the bridge node the Substitute connects to. Default is Advanced options 0 (we do not connect to a bridge node). -w [ --worker ] Flag telling if the Broker Node will execute work package (false by default). -m [ --originator_monitoringport] Monitoring port for Originator. Default value is 0. -M [ --substitute_monitoringport] Monitoring port for Substitute. Default value is 0. -o [ --logging_config_path ] Path of the config file for the logging. Default value is logging.conf. -c [ --originator_config_path ] Path of the config file for theoriginator distribution service. Default value is originator_interface_config.json. -C [ --substitute_config_path ] Path of the config file for the Substitute distribution service. Default value is substitute_interface_config.json. Our 3DEXPERIENCE platform powers our brand applications, serving 12 industries, and provides a rich portfolio of industry solution experiences. Dassault Systèmes, the 3DEXPERIENCE Company, provides business and people with virtual universes to imagine sustainable innovations. Its world-leading solutions transform the way products are designed, produced, and supported. Dassault Systèmes collaborative solutions foster social innovation, expanding possibilities for the virtual world to improve the real world. The group brings value to over 170,000 customers of all sizes in all industries in more than 140 countries. For more information, visit NOTE: -w is not a recommended option, especially for lowperformance workstations, as it could influence the performance of DELTAGEN in general Dassault Systèmes. All rights reserved. 3DEXPERIENCE, the Compass icon and the 3DS logo, CATIA, SOLIDWORKS, ENOVIA, DELMIA, SIMULIA, GEOVIA, EXALEAD, 3D VIA, BIOVIA, NETVIBES, and 3DEXCITE are commercial trademarks or registered trademarks of Dassault Systèmes or its subsidiaries in the U.S. and/or other countries. All other trademarks are owned by their respective owners. Use of any Dassault Systèmes or its subsidiaries trademarks is subject to their express written approval. Americas Dassault Systèmes 175 Wyman Street Waltham, Massachusetts USA Europe/Middle East/Africa Dassault Systèmes 10, rue Marcel Dassault CS Vélizy-Villacoublay Cedex France Asia-Pacific Dassault Systèmes K.K. ThinkPark Tower Osaki, Shinagawa-ku, Tokyo Japan