Code architecture and organisation

Transcription

1 RenderDoc Code architecture and organisation This document covers RenderDoc at a high level, giving you an idea of how the UI is separated from the rest of the code and generally how the capture & replay method flows. I ll go over the three main flows or organisations of the code (capture, local replaying and remote replaying - which isn t too different). At each stage I ll point out the files that correspond to the systems I m talking about so you can match up and get a better idea of where the different bits of code sit. Since it s more complex and harder to intuit, this will mostly focus on the core renderdoc library rather than anything much on the renderdocui application.

2 Capturing structure (1) We start with the capture flow. The application marked in green is whatever the target application is - it could be a game or program, anything that s using a 3D API that renderdoc supports. If you look at the arrow marked with (1), we start off with the application loading the renderdoc library (dll on windows). If the application is launched through the UI directly as indicated in the diagram, the method varies by OS - e.g. on Linux we just launch with LD_PRELOAD= and any other environment variables set. On Windows process is launched suspended, and several threads are injected into it to first load the dll, then call out to set any capture options and such. This is handled inside OS-specific code as an implementation of Process::InjectIntoProcess or Process::CreateAndInjectIntoProcess - See renderdoc/os/win32/win32_process.cpp or renderdoc/os/linux/linux_process. cpp. It s also possible that the application voluntarily hooks itself by loading the renderdoc library without the UI intervening at all.

3 Capturing structure (2) (2) marks the API hooks that are inserted. From the library initialisation point - DllMain or attribute ((constructor)) - we set up any API-level hooking. See: renderdoc/hooks/*_libentry.cpp This will depend on what support is built into the version of the renderdoc library, but e.g. for GL we ll hook various dll-exported functions and in particular wgl/glxgetprocaddress so that we can return our own function pointers. For D3D11 we ll hook D3D11CreateDevice and DXGICreateSwapChain and other entry points like that. Each API has its own hook implementation, see: renderdoc/hooks/*_hooks.cpp These are set up immediately and must be set up before the application calls into them, as otherwise it s too late - this is the reason that when attaching to a process right now you need to do it before the process has invoked any API calls (because these hooks are only attached then).

4 Capturing structure (3) (3) is optional and will depend on the API - e.g. D3D11 uses wrapped API objects heavily to wrap all of the COM interfaces returned to the application code. There are a few bits of common utility code that aid in this. See: renderdoc/common/wrapped_pool.h and renderdoc/core/resource_manager.h When the API entry points like D3D11CreateDevice are called into the hooked version creates an instance of our implementation of ID3D11Device which is returned to the user, and in turn it creates our own implementation of ID3D11Texture2D, etc etc. More details of the D3D back-end s implementation details are available on the github wiki.

5 Capturing structure (4) (4) is the meat of the capturing system. Through whatever means make most sense for the API, it s hooked into the application and is now sitting as a man-in-the-middle between the application and the real API. I call this layer the driver. See: renderdoc/drivers/* All that s required of the driver implementation is that it produces an RDC capture file in the appropriate format at the end. Details are in the next slide but the important part is that it s made up of a series of chunks. A typical implementation such as D3D will do a couple of things: 1) It will track live resources and organise API access that relates to these resources separately into bundles of chunks, e.g. data upload or parameter setting on a texture, render target creation, etc. 2) It will also track dependencies between textures such as copies. Doing this it will be able to discard bundles once the resources are released, and it can keep track of which resources are referenced when a given frame is captured, and only include those bundles in the final file. The usual organisation is friendly to the replay side - it first writes out heavyweight chunks (6) that contain resource creation, initialisation and any other pre-frame actions needed to set up for the initial state of the frame. Other things like resource setting and draws from previous frames are never captured and so omitted from the log. This initial stream of chunks takes up most of the log, and then at the end of the log the API events from the frame capture (7) itself live. This is convenient for the replay side as then after initial loading only the memory for the last part of the log needs to be kept in persistent memory.

6 Capturing structure (5) The RDC capture file (5) is fairly loosely defined. It has a small header that contains things like version, length information. Beyond that, everything in the file is stored as a series of chunks - each chunk has an integer type, a length, and optionally a callstack. See: renderdoc/serialise/serialise.* We define a few chunks that must always be present at the start of a RDC file as standard - there s a chunk that includes the capture thumbnail (provided by the API driver implementation) which can be easily read out without much processing e.g. to provide file previews. Then follows a chunk that defines which API the log contains as well as an API-defined set of initialisation parameters. These chunks are used so that renderdoc can identify and do things with logs even when it has no API implementation to process the rest of the log. See: renderdoc/core/core.cpp and specifically RenderDoc::OpenWriteSerialiser and RenderDoc::FillInitParams

7 Local replay structure (1) Now we look at the replay side of the picture. As I mentioned, we won t be focussing on the UI code much, but instead looking at how the code flows through the main library. (1a) the C# UI uses P/Invoke to call into the mostly-c interface on the replay layer. See: renderdocui/interop (1b) renderdoccmd is a small C++ utility app that does a few different things and is mostly used for testing functionality, but it links directly against the renderdoc library and so calls straight into the C interface. See: renderdoccmd/renderdoccmd.cpp

8 Local replay structure (2) The inventively-named replay layer (2) presents a high-level interface to whatever UI or program is built on top of it - it will open a log, replay to given point, do analysis like give me the histogram of values for this texture or display this texture, within the range 5-10, on this window handle. See: renderdoc/replay/renderdoc.h and renderdoc/replay/replay_renderer.h The replay layer is actually fairly slim and most of its work is passed along elsewhere. It can feel like an unnecessary set of duplication at times, but it removes a lot of the drudgery of preparing things for passing across the external interface as well as doing some API-agnostic bookkeeping. When the replay layer is asked to load a log, it figures out which API is used in the log and depending on what functionality is built into the library it will call out to a replay driver creator that s registered for that API. See: renderdoc/core/core.h struct DriverRegistration / enum RDCDriver for the principle and renderdoc/driver/d3d11/d3d11_replay.cpp for an implementation This creator is responsible for taking the log + initialisation parameters and returning an implementation of the API interface that sits between the replay layer and the actual driver implementation in the above diagram. The replay layer then talks to this API interface to actually do all concrete operations on the log. See: renderdoc/replay/replay_driver.h for the interface and renderdoc/driver/d3d11/d3d11_replay.h for an implementation

9 Local replay structure (3) The API driver that we first saw in the capture flow also appears here (3) in the replay flow. Much of the same code is executed to go through the serialisation in reverse to read out of the API chunks the parameters, data, etc. Exactly what the driver does with this data is implementation-dependent but usually it will read out the initial data and chunks once, then keep the frame chunks in memory for repeated access. This part of the code has a lot of overlap with the driver implementation for the capture side, and is mostly responsible for keeping the replay from diverging, replaying any subsets of the frame that are requested, and returning things like texture metadata and other information gleaned directly from the log. For D3D11 see: renderdoc/driver/d3d11/d3d11_device* and renderdoc/driver/d3d11/d3d11_context*, although there are other files involved. More info in the D3D implementation details in the github wiki.

10 Local replay structure (4) In replay mode, an additional set of code in the driver implementation gets used (4) for doing analysis and debugging beyond the scope of capturing. Things like calculating histograms, displaying meshes, fetching post-transform data out of shaders etc isn t used at all during capture and is only done on replay. Again this code is used to implement the other functions in the API interface which the replay layer will call into. For D3D11 see renderdoc/driver/d3d11/d3d11_analyse.cpp and renderdoc/driver/d3d11/d3d11_debug.cpp, as well as all of the shader disassembly and debugging code in renderdoc/driver/d3d11/shaders/*. More info in the D3D implementation details in the github wiki.

11 Remote replay structure (3) When a log uses an API that doesn t have support built into the local library version, the UI is responsible for querying for this and asking the user how they d like to proceed. If the user chooses to replay the log via a remote host, then the UI will invoke the replay layer (2) asking it to replay on a remote host instead of loading the log locally. When this happens, a bridge class is used to implement the API interface, so that as far as the replay layer and above are concerned nothing is different to a locallysupported log from this point on. See: renderdoc/core/replay_proxy.cpp

12 Remote replay structure (4) The remote bridge will connect to the network host (launching the replay program on the remote host is beyond the scope of renderdoc, although this can be supported by plugins. For this explanation we assume that the replay program is already running). The network host is essentially running the renderdoc library as-is, but instead of exposing the replay layer it listens on a network port for a connection that will transfer a log and it will immediately try to load it locally. The remote host if it supports the API only has to provide a subset of the API interface functionality - that functionality that requires knowledge of the specifics of the API and replaying. See: the distinction between IRemoteDriver and IReplayDriver in renderdoc/replay/replay_driver.h - note an IReplayDriver is a superset of IRemoteDriver functionality. This remote replay then behaves as before in the local replay case, providing this subset of functionality and serialising across the network to the remote bridge.

13 Remote replay structure (5) So that the remote bridge can implement the whole of the API interface (3) it also creates a local proxy. This local proxy is one of the locally supported APIs the implements all of the API interface functionality. Although this local proxy doesn t support the API from the log, all of that functionality is implemented remotely across the network link, and so it is left to do things like render texture data to windows, perform any generic analysis like producing texture histograms, displaying textures with a given range or channel masking, etc. Above the remote bridge, this split and network link is invisible (beyond the inevitable network latency and throughput limitations), as the remote bridge acts just like a catch-all API driver that implements any API that isn t supported locally.

14 The end If you have any questions feel free to ask! Contact details, source, everything is linked from github: