The RANCH RenderFarm User Guide Part II the RANCH for Maxwell Render

Transcription

1 The RANCH RenderFarm User Guide Part II the RANCH for Maxwell Render March 3, 2015 Welcome to the RANCH automated rendering service, the super-powerful - and affordable - supercomputer for all your Maxwell projects! This document contains information specific to the use of Maxwell Render on the RANCH. Before reading it, we highly recommend you read Part I first - the General Guide - which includes everything not specifically related to Maxwell Render. The total time of a project, which determines its cost, includes: - the distribution of the project across the network + the preparation / voxelization phase + the render time itself that you specify in the submit project form + and the merging of all the individual MXIs produced by the render nodes into the final MXI Important: MXI merging changes since version 2.7 Maxwell Render 2.7 introduced MXI compression. The size of the MXI file produced by a project is smaller, so it will take less time to download the final MXI, even though the compression ratio can vary considerably with each scene, depending on many parameters (number of channels, definition, number of emitters, use of Multilight, etc.). Compressed MXIs typically take a bit more time to merge than their non-compressed equivalent in low/medium definition, but less time in high definition. The drawback of MXI compression is that it is no longer possible to accurately predict the merging time of a project based on the size of the MXI, since: - the compression ratio is never the same from one project to the next. - compressed MXIs of the same size do not necessarily take the same time to merge. - the final merged MXI is heavier than individual MXIs. So a 500 MB compressed MXI file may be equivalent to a 1 GB uncompressed MXI, or a 4 GB uncompressed one... the merging time will be 4 times longer in the second case, even though the MXI compressed size is about the same. General rule to keep in mind: a complete merging on the RANCH Runner generally takes between 2 and 20 minutes, for typical MXI sizes (from a few MBs up to 2 GB). Tip: to keep the size of the MXI low, do not use Multilight unless you absolutely need it. If that is the case, avoid Color Multilight as it can produce really _huge_ MXIs. 1

2 1) Maxwell Render on the RANCH: specific features In addition to the general features described in Part I of the RUG, the RANCH offers specific features for unbiased renderers in general, and Maxwell Render in particular: - Maxwell 3.x and Maxwell 2.x cooperative projects are supported - Ability to resume projects to improve image quality - Ability to target a cooperative sampling level (SL) - You get back the MXIs files and the bitmaps - Ability to render several cameras within a single project (MultiCam mode) - A denoising pass is automatically applied to cooperative renders - The waiting list displays in real time an approximation of the current global Sampling level (SL) reached while rendering a cooperative project. The first SL display is available 5 to 6 minutes after the beginning of the project. - Real time image previews let you check your project visually while it is being rendered. 2) Important information about scene preparation It is VERY important that you read carefully this section before submitting a scene. It will help you to avoid common mistakes. Please verify before sending the scene that you have saved it with all the correct parameters. It is your responsibility to provide the scene with every parameter correctly set. The scene will be rendered exactly as you send it. Before sending your scene to the farm, reload it on your computer and begin a render from Maxwell Render (NOT from Maxwell Studio) at full resolution for a few minutes, just to check that everything is all right and that your scene does not cause a crash or other potential problem. Some errors triangles without material for instance - are not caught when rendering from MS, but they crash MR. Do not use accentuated / non-alphanumeric characters, spaces, apostrophes and dots in the filenames of your project (scene, textures, objects, etc.). To avoid any possible parsing problems on the RANCH Runner, please use only letters and numbers. You can also use the _ sign (underscore) to replace spaces. You can use externally referenced.mxs scenes (introduced in Maxwell 2.6) in your projects if all these scenes have the 'xref_' prefix (e.g. 'xref_myproduct.mxs'). Only the main scene file can have a standard name (e.g. 'MyScene.mxs'). All the referenced.mxs files must have the 'xref_' prefix, or the server won't know which is the main scene and the project will be rejected or considered as an animation (with as many frames as there are.mxs files - see page 6). 2

3 3) Scene preparation for a still image project Cooperative mode To prepare your still image projects, you must use RANCHecker, a free standalone application developped by the RANCH. This easy to use tool gathers all the assets needed by your scenes and prepares them as verified, ready-to-render.vum project archives. RANCHecker is also able to benchmark your system and give you useful cost/time estimations for your projects. You can download RANCHecker on our RANCH for Maxwell Render web site (PC and Mac versions are available). Since version 2.2, RANCHecker is also able to generate a MultiCam project (one project with several cameras to render). For more details on MultiCam projects, please read Appendix B of this manual dedicated to the MultiCam mode. 3

4 4) Resuming a cooperative project! This exclusive RANCH feature lets you resume a previously rendered cooperative project. Let s say you just rendered a project called MyHouse.vum for two hours. You download your files, check the image and the MXI but wish you had specified a render time of four hours, because the Sampling Level is not high enough and the image is still too noisy. Fortunately, the RANCH is able to resume a cooperative render! If you resend a project with the exact same name (in our example MyHouse.vum ) and - obviously - the exact same X / Y resolution, the RANCH will be able to automatically retrieve the data from its projects cache and resume the render from the state it was in at the end of the last render. Other things to know about resuming: - All the data from a cooperative project is kept in the cache until the temporary ftp account for the project expires. Thus if you try to resume an old project (more than 5 days old), no data will be available and the project will be considered as a new project. - If you want to resume a project, you must send it from the same RANCH user account as the previous one. - Each resumed project, when finished, will itself be considered as a new project by the RANCH. It will have its own new ftp directory and you will be able to resume it! It means that you can improve the original project progressively, through successive resume projects! - When using the resume feature, the you receive will confirm that the current project is indeed a resumed project. - Do not worry if, looking at the RANCH queue during the second run, you see that the displayed SL is not higher than the one displayed during the first run: the MXIs of the first and second run are simply merged at the end to produce the final MXI. Another good surprise for the end: when resuming a project, the RANCH will only use the data from its cache, and will not attempt to decompress the.vum file you send. It means that you don t need to resend a big project file (the true scene) each time. Just create a small text file (it must not be empty though, its size must be > 0), rename it to the exact name of the original project - in our example it would be MyHouse.vum - and send it! If the same name is found in the RANCH cache, only the previous data will be retrieved, so it does not matter that the.vum file you send is not a real.vum file. Of course, if no data is available in the cache for a project of this name, it will simply be rejected as an invalid.vum file. 4

5 5) Scene preparation for an animation project Frame mode The preparation is different for an animation. Here are the steps to follow: 1) Create your animation in the host application (3DS Max, C4D, etc.). In the Maxwell plugin, choose the Write MXS option. The Maxwell plugin will then generate as many MXS files as there are frames in the animation. Copy all these MXS scenes in a new directory that you will use to store all your project s files. Let s call it DIRA in our example (but you can give it the name you want, with only letters and numbers no spaces nor special characters). 2) Now, you need to gather all the external files needed by your project (textures, etc.). To do this, open one of the generated MXS scenes in Maxwell Studio, and use the Pack n Go function to write all the external files found to another directory (NOT your project directory with all the MXS). Let s call this second directory DIRB. 3) Copy all the files in DIRB to DIRA, except the lone MXS scene which has been created in DIRB by the Pack n Go function. After this operation, all the necessary files are in DIRA (all the MXS files + all required external files, textures, etc.). 4) Compress all the files in DIRA in one archive with the.7z format or the.zip format (see 5-A for details) depending on the compacter you use. Please do not use another format. 5) Rename the compressed archive you just created with the extension.vum. For instance, if your archive is MyAnim.zip, rename it to MyAnim.vum. It is essential for the automated renderfarm to detect the file as a Maxwell project. The RANCH knows that a project is a still image if it finds only one main MXS scene in the.vum file. It will then allocate the same scene to all the render nodes in cooperative mode. If it finds several main MXS scenes in the archive (that is, not 'xref_' files - see page 3), it will process the project as an animation (one MXS scene / frame per node). So, if you want to send a cooperative still image project and not an animation, please make sure that there is only ONE main.mxs file in your.vum archive. Please also note that you cannot upload a file with a size > 2 GB to the RANCH. If you send an animation project composed of many.mxs scenes, this limit can be reached. In that case, you will have to split your big project into several smaller projects, each with a size < 2 GB. For 3ds Max, Maya and Cinema 4D users You can send your 3ds Max / Maya / Cinema 4D + Maxwell animation directly as a 3ds Max, Maya or C4D project, instead of having to send a large number of big.mxs files. To do this, please consult our web sites and PDF user guides dedicated to these applications. 5

6 6) Choosing a strategy for rendering You have two possibilities: choosing a maximum render time limit (in minutes) or a maximum Sampling Level (SL), depending on your preferences. If you are on a deadline, you will probably want to get the best quality image in a given time. For instance if you have around two hours you can enter 120 in the Render phase time limit field and 30 in the Sampling Level field. The render phase will then be interrupted after 2 hours, or after a sampling level of 30 is reached (which is not likely ). Another strategy: you want to get to a sampling level of N (for instance N=15), which you consider necessary and sufficient for your needs. In that case you could enter 120 minutes (the maximum time authorized) and 15 in the SL field. The render will then stop when the desired SL is reached. You must always use this strategy when you render an animation, to ensure that all frames are of the same quality. If you are not too sure, for instance you would like a SL of 16 but it is not mandatory, and in any case you do not want to exceed a 1 hour render phase time, you would enter 60 and 16. In that case the render phase will be stopped as soon as the 60 minutes limit is reached, OR a SL of 16 is reached. Please note that the total time of a project, which determines its total cost, includes several factors (the example below is for a cooperative project): - The time needed to send the scene to all the nodes. This is quick, generally less than one minute with most scenes, and several minutes with complex scenes. - The preparation time (the voxelization can take a while on complex scenes). - The render time (the longer part obviously, which you specify the length in the Submit Project form). - The time needed to gather and merge all the MXIs into the final MXI. It depends on the MXI sizes (see next page). - The time to create your temporary ftp account and transfert the files (typically less than one minute). We recommend that you use RANCHecker to better estimate the total cost of your project. 6

7 IMPORTANT: MXI complexity and processing time The final MXI file is obtained by gathering and merging the cooperative MXI files from all the RANCH Runner nodes. The complexity of a MXI file depends on the resolution of the project, the use or not of the Multilight option, the number of light emitters and channels. The gathering time (copying the MXIs from all the nodes across the network) and the merging time (the fusion of all these MXIs into the final MXI) depend on the complexity of each MXI. With large MXIs, a huge amount of data must be transferred back and forth across the network, and the time required for the gathering + merging phase can be quite long. That being said, the RANCH Runner offers the highest level of merging performance available today. The merging time on the RANCH Runner typically takes between 2 and 20 minutes depending on the project complexity. Warning: Merging has changed since version 2.7 of Maxwell Render. Please make sure you have read the details on page 1 of this guide. If you check the Multilight option when saving your scene, the MXI sizes will be much bigger, and processing time will increase accordingly. Please be very careful when using the Multilight option, and be aware that it can considerably increase the total processing time of the project, even if the render time itself is small. If you absolutely need to use the Multilight features of Maxwell Render, try to avoid Color Multilight, as it can generate huge MXIs that can eat a lot of RAM and take a very long time to merge. 7

8 7) Files produced by the render * If your project is a still image / cooperative render You will find in the ftp directory of your project: - your original.vum file. - the final MXI merged after the cooperative render. - bitmap layers you specified in your scene (alpha, IDmaterial, IDmesh, etc.) - a final image of your project (with the suffix _final ). - a denoised version of the final image (with the suffix _denoised ). The final image file is automatically filtered at the end of the RANCH process in an attempt to reduce the amount of digital noise in the picture. Note 1: the effects of the denoising are subtle. It will never replace a longer render time, and it will never transform a supergrainy image in a perfect image, but it can help to clean the final picture by removing some of the noise. The denoising pass is very fast and done at no extra cost for the customer. Note 2: on some images the denoiser may fail. In that case the _denoised image will not be present in the directory. As it is an automated process, the denoising phase cannot be as refined as if you had denoised the image manually, and in some rare circumstances it won t work. Please consider this feature as a bonus free of charge, not as a guaranteed result * If your project is an animation You will find in your directory: - your original.vum file. - all the frames of your animation, saved in the bitmap format you chose in the scene. If for any reason the bitmap output format is not identified, the 32-bit TIF format will be used. - bitmap layers you specified in your scene (alpha, IDmaterial, IDmesh, etc.) 8

9 Appendix A: the real time previews Cooperative project preview (still image) The RANCH lets you check visually your cooperative project while it is being rendered, on a web page specific to your project in progress. To access this page, click on the button in the waiting/priority list. Below is an example of a preview image. Notes: Scene by Felix de Montesquiou - The first preview is available 5 to 6 minutes after the beginning of the project. - The preview does not show all the subtleties of the final image. - The preview image has a maximum definition of 1400 pixels (wide or high). - The preview is annotated (project name, current SL, render time, real image definition). - The preview is generated with each SL increase. - You can click on the REFRESH button to update the preview. 9

10 Animation preview The RANCH offers you a very handy feature for checking visually your animation project when it is being rendered. It displays 256-pixel wide thumbnails of every tenth frame rendered (frames 0, 10, 20, etc.) on a web page specific to your inprogress project. To access this page, you just have to click on the button which appears when your project is being rendered (and of course if there is something to preview: if each frame of your animation takes 30 minutes to render, obviously there will be nothing to see during the first 30 minutes :) Below is an example of what you will be able to see when you click on Preview (the preview image is always around 1900 pixels wide, its height depends on the number of thumbnails). Notes: - At the end of the render, the animation preview image is copied in your project's directory. 10

11 Appendix B: the MultiCam mode In some projects, you may have to render several views of the same scene. The MultiCam mode (integrated in RANCHecker >= 2.2) lets you select the cameras you want to render: and then writes this information in the.vum archive. When the project is processed, all the cameras you checked will be rendered one after the other, so you won t need to send as many projects as you have views to render. If there is only camera in the project, the above requester won t appear when you prepare your project. Things to know about the MultiCam mode: - if you check only one camera to render, the X/Y definition used when rendering will be the one you enter in the Submit Project form on the RANCH web site - if you check at least two cameras to render, the X/Y definition used for a camera when rendering will be the definition saved for this camera in the.mxs scene - each camera is rendered in its own subdirectory in the project directory - each camera can have a different definition (width/height) - if you cancel your project while it is rendering, the whole project will be canceled, not only the current camera. - the waiting list will display CAM 2/4 for instance if the currently rendered camera is the second camera out of four. - the list of selected cameras is displayed in the sent when the project starts - the SL reached by each rendered camera is displayed in the end project - the preview (see appendix A) works with each camera successively - resuming a MultiCam project is not possible, even if only one camera is selected 11