Written above
: 1. This description is only for the renderdoc opengl es scenario, not for vukan, but in fact it should be almost the same. If necessary, I will consider adding it later.
2. This article uses text + pictures + code to describe. If you just want to understand the principle process, you can ignore the code part .
Since the renderdoc code is quite complex and messy, the code part is mainly to assist students who are interested in reading the source code to grasp the main points of the code. logic.
3. renderdoc related nouns
4. When capturing a frame,
1. Start timing: When the previous frame is swapbuffer, the specific logic is done in the StartFrameCapture() interface;
2. End timing: When the current frame is swapbuffer, the specific logic is done in EndFrameCapture() interface;
that is, the end of the previous frame is the beginning of this frame, as shown in the figure:
what
- Resource manager, as the name suggests, is used by renderdoc to manage gl resources. It has built a state machine on top of opengles to record the usage of gl resources.
- The gl resource is mainly described by three structures in the renderdoc resource manager.
GLResource: records the resource type and the id in the opengl es state machine, such as the texture value generated by glGenTextures;
ResourceId: records the id of the resource in the resource manager. ,
GLResourceRecord: In the form of chunks, it records the relevant gl interface calls for operating the resource, especially creation, binding, attribute setting, data upload, such as glGenTextures, glGenBuffers, etc. One glXXX is recorded in one chunk. - gl resource:
The following can be described as resources in renderdoc, which basically covers all gl resources. It is somewhat similar to the idea that everything in Linux is a file, hahaha.
enum GLNamespace
{
eResUnknown = 0,
eResSpecial,
eResTexture,
eResSampler,
eResFramebuffer,
eResRenderbuffer,
eResBuffer,
eResVertexArray,
eResShader,
eResProgram,
eResProgramPipe,
eResFeedback,
eResQuery,
eResSync,
eResExternalMemory,
eResExternalSemaphore,
};
why
- I built a state machine on opengles, which can play the following roles in
stream capture scenarios:
(1) Performance and memory optimization: During BackgroundCapturing, for each gl resource, record its data update time (postpone mechanism) and whether the current frame is used (frame reference mechanism), whether it is dirty (dirty mechanism),
when performing real capture ActiveCapturing, you can only serialize the resource-related chunks saved in the current frame;
replay scenario:
(1) New and old resource mapping: For from The resource ID of the opengl state machine is saved during stream capture and serialization. During
replay, a corresponding resource needs to be created. Since the ID is not guaranteed to be once, a mapping needs to be done,
that is: originating GLResource <------ > live GLResource
(2) Performance optimization:
When capturing a frame, some resources upload data to the GPU side before the start of the frame,
so the GPU side memory of the resource needs to be processed before each replay frame starts. Initialization, if you have to find the data from the disk and then disk–>cpu–>gpu every time, it will consume too much performance,
so you can make a record in the resource manager: live resource/origing resource <–> initialContents,
initialContents is placed on the cpu side. For memory, it will be ok if you do cpu-data->gpu before each replay.
Neither the resource manager itself nor the various mechanisms implemented in it such as frame reference, dirty, persistent, postpone, etc. are necessary. They are optimizations for performance and memory. They can also be used when capturing streams. It is simple after hooking
. Roughly recording all gl calls and resources will of course bring about various lags and excessive memory usage in both Background Capturing and Active Capturing.
frame reference mechanism
- what
marks a resource in a frame as being used - Why
optimize memory and performance: When capturing streams, resources that are not marked in the frame do not need to be saved. - Related interfaces
template <typename Configuration>
void ResourceManager<Configuration>::MarkResourceFrameReferenced(ResourceId id, FrameRefType refType)
//EndFrameCapture 时，序列化frame reference 资源
template <typename Configuration>
void ResourceManager<Configuration>::InsertReferencedChunks(WriteSerialiser &ser)
dirty mechanism
- What
uses MarkDirtyResource() to mark the resource as dirty, which is used when the gpu-side content of the resource is changed;
different from the frame reference resource, MarkResourceFrameReferenced() is used to mark the frame as using the resource. - Why
BackgroundCapturing scene, every time the application updates data to the resource, renderdoc does not need to be saved every time, because in fact, when capturing the frame, only the last content of the resource before the start of this frame is enough.
For example, for a buffer:
//BackgroundCapturing
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, buffName);
//frame1
glBufferData(GL_ELEMENT_ARRAY_BUFFER, data_size_in_bytes, data1, GL_STATIC_DRAW);
//frame2
glBufferData(GL_ELEMENT_ARRAY_BUFFER, data_size_in_bytes, data2, GL_STATIC_DRAW);
//framen
glBufferData(GL_ELEMENT_ARRAY_BUFFER, data_size_in_bytes, dataN, GL_STATIC_DRAW);
//ActiveCapturing
//frame capture
//对于当前要抓的这一帧，若用到了buffName,那么我们需要保存的就只是dataN的内容，
//所以对于BackgroundCapturing时更新的数据data1、data2等，只要每次更新数据时做个dirty标记就行，
//表示这块资源dirty，可是没有保存，咱们在抓帧时把他保存下来就行
- Related interfaces
void MarkDirtyResource(ResourceId id);
void MarkDirtyResource(GLResource res)；
//reference和dirty一起标记了
void MarkDirtyWithWriteReference(GLResource res)
void MarkVAOReferenced(GLResource res, FrameRefType ref, bool allowFake0 = false);
void MarkFBOReferenced(GLResource res, FrameRefType ref);
void MarkFBODirtyWithWriteReference(GLResourceRecord *record);
- manual
- When BackgroundCapturing, for the resource MarkDirtyResource,
- Before the actual frame capture starts, that is, inside StartFrameCapture(), make a gpu-side copy of the dirty resource and a cpu-side mapping (because the gpu-side resource content needs to be accessed through the cpu-side handle)
- At the end of the frame capture, that is, inside EndFrameCapture(), if these resources are referenced in the frame, copy the previously backed up resources from the gpu side to the cpu, and serialize and save the dirty resources.
Related
interfaceActiveCapturing
/*****************StartFrameCapture**********************/
//遍历所有标记为dirty的资源，创建map resource, 即origin resource <----> map resource,
//gpu侧，origin resource ---data copy---> map resource
template <typename Configuration>
void ResourceManager<Configuration>::PrepareInitialContents()
/*****************EndFrameCapture**********************/
//对每块dirty resource , origin resource ---->取出 map resource,
//map resource ---gpu data----> cpu,
//cpu 侧对数据序列化，写入rdc文件
template <typename Configuration>
void ResourceManager<Configuration>::InsertInitialContentsChunks(WriteSerialiser &ser)
//序列化保存所有需要初始化的dirty resource，frame reference resource
template <typename Configuration>
void ResourceManager<Configuration>::Serialise_InitialContentsNeeded(WriteSerialiser &ser)
replay
//创建map resource, origin resource <-----> map resource ,
//rdc ---data--> cpu---data---> gpu, gpu侧这边为map resource
//map resourece 在cpu侧以InitContents形式与 origin resource 建立映射
template <typename SerialiserType>
bool GLResourceManager::Serialise_InitialState(SerialiserType &ser, ResourceId id,
GLResourceRecord *record,
const GLInitialContents *initial)
//读出所有需要初始化的resource id,
//若此时该origin resource 有对应的live resource, 且还没有创建初始化内容，
//则做一遍类似上面的事情:
//创建map resource, origin resource <-----> map resource ,
// gpu侧 live resource --data copy---> gpu侧map resource
//map resourece 在cpu侧以InitContents形式与 origin resource 建立映射
template <typename Configuration>
void ResourceManager<Configuration>::CreateInitialContents(ReadSerialiser &ser)
template <typename Configuration>
//每次重放开始时，执行 gpu 侧map resource ---data copy ---> gpu侧 live resource
void ResourceManager<Configuration>::ApplyInitialContents()
share
//获取一块 resource的初始化内容
InitialContentData GetInitialContents(ResourceId id);
//保存一块resource 的初始化内容
void SetInitialContents(ResourceId id, InitialContentData contents);
Regarding the specific process of stream capture, you are welcome to learn more about the process of stream capture through renderdoc ~
-
Judgment mechanism
: There is a gl interface for data update, and the resource is marked dirty. -
Data saving timing and process:
When StartFrameCapture, the data is copied to the gpu side;
When EndFramecapture, the data is copied from the gpu to the cpu and serialized and saved; -
Timing of data recovery
Then, the data recovery during replay is of course the reverse of the above process.
-
Sequence diagram
, I didn’t want to draw this diagram originally, it’s too time-consuming, but the renderdoc code is too messy and complicated, so I might as well draw it so that I can quickly understand the code flow through this diagram next time.
Capture the stream
replay
- Relevant call flow
capture flow
StartFrameCapture()
-->
PrepareInitialContents();
-->
Prepare_InitialState(GLResource res)
-->
//当前上下文里边做资源在gpu侧的拷贝
ContextPrepare_InitialState()
-->
创建初始化资源存到这，还没有回读与序列化
m_InitialContents[id].data = contents;
===========================================
EndFrameCapture()
-->
InsertInitialContentsChunks()
-->
Serialise_InitialState()
-->
template <typename SerialiserType>
bool GLResourceManager::Serialise_InitialState(SerialiserType &ser,
将数据从gpu侧回读并做序列化保存
replay
//数据加载到gpu侧，创建InitialContents
Serialise_InitialState()
-->
//若有live resource，且没有创建InitialContents
//gpu侧拷贝live resource 数据
CreateInitialContents()
-->
//执行gpu侧数据拷贝
Create_InitialState()
-->
//每次重放开始时，gpu侧，map resource --data copy--> live resource
ApplyInitialContents()
-->
//执行gpu侧数据拷贝
Apply_InitialState()
persistent mechanism
-
Persistent resources
are resources that have been resident since they were created. For gles, they are texture and buffer resources that have been updated after creation and have not been destroyed after 3 seconds; that is,
for texture and buffer resources, if it exceeds 3 seconds, When the resource is updated again, which also indicates that the resource has not been destroyed during this period, it is considered to be a postpone resource, that is, a resident resource that is not created within the frame; -
The persistent rule
operates 3 seconds from the last update (write) time. -
Related interfaces and data structures
//texture、buffer资源，更新速度没那么快（大于PERSISTENT_RESOURCE_AGE，即3s外），或者前面没有被引用
bool HasPersistentAge(ResourceId id);
//texture、buffer资源
virtual bool IsResourceTrackedForPersistency(const WrappedResourceType &res) {
return false; }
//StartFrameCapture时，对于persistent resource，不需要在prepareInitialContents()时
//先拷贝一份资源出来， 而是可以延迟到写rdc时，
//原因： 因为read only资源？反正内容不会被改变？
//场景：texture、buffer资源，更新速度没那么快（大于PERSISTENT_RESOURCE_AGE，即3s外），或者前面没有被引用
std::unordered_set<ResourceId> m_PostponedResourceIDs;
//资源被引用（写）的时间，只记录最后那次
rdcarray<ResourceRefTimes> m_ResourceRefTimes;
postpone mechanism
-
Why
is this mechanism necessary? Is it a must or an optimization?
ans: Optimization item, reduce the number of resources in PrepareInitialContents,
because these resources may not be used in the current frame to be captured;
what is optimized?
ans: performance, memory -
postpone resources
for persistent resources
// During initial resources preparation, persistent resources are
// postponed until serializing to RDC file.
std::unordered_set m_PostponedResourceIDs;
-
postpone rule:
If it is a persistent resource, postpone processing is performed, and PrepareInitialContents is not done until markFrameRef,
instead of doing it at StartFrameCapture. -
When using scene
StartFrameCapture, for buffer and texture resources, if it has been more than 3s since the last update, it is also considered that for the general development process, the current frame will not be updated again. It is considered to be a postpone resource, and it will be postponed
to EndFrameCapture processing,
will the frame capture process process it? -
Related interfaces and data structures
bool IsResourcePostponed(ResourceId id);
//返回true情况：
// texture、buffer资源，更新速度没那么快（大于PERSISTENT_RESOURCE_AGE，即3s外），或者前面没有被引用 ,即 对于texture、buffer资源，若超过3s，资源还在，就认为是postpone资源
bool ShouldPostpone(ResourceId id);
template <typename Configuration>
//EndFrameCapture()时， 处理postpone 资源，前面PrepareInitialContents没有处理嘛
//所以需要留到这才处理
void Prepare_InitialStateIfPostponed(ResourceId id, bool midframe);
//gles里边没啥逻辑，应该没有skiped资源，vulkan才有，先忽略
void SkipOrPostponeOrPrepare_InitialState(ResourceId id, FrameRefType refType);
std::unordered_set<ResourceId> m_PostponedResourceIDs;
- Resource serialization
is no different from the normal serialization process.
EndFrameCapture()
-->
InsertInitialContentsChunks()
-->
Prepare_InitialStateIfPostponed()
-->
Prepare_InitialState()
-->
Serialise_InitialState()
- The recovery of resources
is no different from the normal recovery process.
Serialise_InitialState()
-->
//从rdc读出数据，创建副本，数据加载到gpu侧副本
Serialise_InitialState()
-->
//gpu侧从副本拷贝
ApplyInitialContents();
- Why
performance optimization, for buffer, texture, some situations that can not be processed will not be processed.
Opengl currently should not have a resource that can be skipped, because there is no eFrameRef_CompleteWriteAndDiscard scene
only vulkan has, this can be ignored for now. - Related interfaces
//opengl 目前来说应该是没有一个资源能被skip，因为没有一个eFrameRef_CompleteWriteAndDiscard场景
//vulkan才有，这个可以先不管
template <typename Configuration>
inline bool ResourceManager<Configuration>::ShouldSkip(ResourceId id)
template <typename Configuration>
inline bool ResourceManager<Configuration>::HasSkippableAge(ResourceId id)
//应该是一直为空
// During initial resources preparation, resources that are completely written
// over are skipped
std::unordered_set<ResourceId> m_SkippedResourceIDs;
- skip rules
- scenes to be used
replacement mechanism
//TODO
- why
- rule
- scenes to be used
- Related interfaces
Related data structures
- FrameRefType
is used to mark the use of resources
enum FrameRefType
{
// Initial state, no reads or writes
eFrameRef_None = 0,
//标记有对资源做了写操作
eFrameRef_PartialWrite = 1,
//标记有对资源做了写操作, 并且后面没有读操作了
eFrameRef_CompleteWrite = 2,
//标记对资源做了写操作
eFrameRef_Read = 3,
//每次重放前都需要重置该资源，
//因为使用顺序是read-->write, 可能后面write后又回到前面了嘛
//先对资源进行了读，又对它进行了写
eFrameRef_ReadBeforeWrite = 4,
//使用顺序是write --> read
//先对资源进行了写，又对他进行了读
eFrameRef_WriteBeforeRead = 5,
//目前没用
eFrameRef_CompleteWriteAndDiscard = 6,
eFrameRef_Unknown = 1000000000,
};
//用来算新状态的，即老状态+新状态 结果
FrameRefType ComposeFrameRefs(FrameRefType first, FrameRefType second)
The above resource read and write status records are mainly used when starting to capture frames, specifically StartFrameCapture()–>ClearReferencedResources(), to determine whether to copy the marked resources. For details on StartFrameCapture(), please refer to the renderdoc flow capture process .
template <typename Configuration>
void ResourceManager<Configuration>::ClearReferencedResources()
{
SCOPED_LOCK_OPTIONAL(m_Lock, m_Capturing);
for(auto it = m_FrameReferencedResources.begin(); it != m_FrameReferencedResources.end(); ++it)
{
RecordType *record = GetResourceRecord(it->first);
if(record)
{
//这块资源在前面是有被写过的，要标记为dirty，保留
//可能在后面的帧中有用到时，
//需要在重放的一开始就去初始化它
if(IncludesWrite(it->second))
MarkDirtyResource(it->first);
record->Delete(this);
}
}
m_FrameReferencedResources.clear();
}
-
ResourceId
, as the name suggests, marks the ID of a resource. -
ResourceRecord and GLResourceRecord
store the relevant chunks that use the resource. For chunks, refer to the renderdoc flow capture process ;
with context as the boundary (different contexts may share resources), for a resource, it is necessary to record the process of its use,
ques: What is it like? The resource operation interface needs to be recorded in the resource's own record?
ans: The resource initialization process, such as texture creation and binding, uploading data from the CPU side to the gup side, copying data from the gpu side, resource attribute configuration, etc., are all stored in ResourceRecord through chunks.
The purpose is to facilitate the retrieval of these initialization interfaces to initialize resources in the replay scene when replay starts. -
As the name suggests, GLResourceManager
is used to manage resources. The hero of this article, all resource operation and management logic is here; capture
stream: record which resources are currently available and which ones are marked as dirty,
replay: manage the original resource Id and For the mapping of live resource Id,
the main marking operations for resources are dirty and frameReflence.
When capturing streams, these two types of resources mainly need to be processed. -
GLInitialContents
stores a piece of resource initialization data -
InitialContentDataOrChunk
-
ResourceRefTimes
marks the time when the resource was last frame referenced.
If it is more than 3 seconds since it was last marked when capturing the frame, it is considered a persistent resource. -
TextureData
is used to describe a texture in renderdoc. -
ResourceDescription
describes the chunks id required for initialization of this resource. -
FBOCache
records a fbo and how many resources are attached to it. -
GLNamespace
marks resource types, such as buffer, texture, shader, program, etc. -
GLContextTLSData
gl context thread share data? Not sure yet
TIPS
- VERBOSE_DIRTY_RESOURCES
, as the name suggests, is a switch used to debug the dirty mechanism. When turned on, the relevant debug log of the mechanism can be printed. - The relationship between GLResource and GLResourceRecord is that
GLResource describes the resource,
and GLResourceRecord records the relevant gl interfaces used for the resource, that is, chunks.
FAQ
- For a texture mark dirty, after a drawcall ends, the texture is deleted. How to deal with the renderdoc?
ANS: It seems that it cannot be handled at present. WrappedOpenGL::glDeleteTextures deletes the resource record.
However, I feel that general development code will not be written like this. Either create them uniformly during initialization and destroy them later,
or create and destroy them within the frame.