In-depth UE - UObject (13) type system - reflection practice

introduction

The previous chapter summarized the last few knowledge points of the type system. In order to avoid saying that it is purely theoretical knowledge, in this article we will talk about some examples of using reflection.

Get type object

If you want to get all the classes defined in the program, the convenient method is:

TArray<UObject*> result;
GetObjectsOfClass(UClass::StaticClass(), result);   //获取所有的class和interface
GetObjectsOfClass(UEnum::StaticClass(), result);   //获取所有的enum
GetObjectsOfClass(UScriptStruct::StaticClass(), result);   //获取所有的struct

GetObjectsOfClass is a very convenient method that has been written in UE4, which can get all objects belonging to a certain UClass*. Therefore, if you use UClass::StaticClass() itself, you can obtain all classes defined in the program. It is worth noting that the interface in UE4 has a matching UInterface object to store metadata information. Its type is also represented by UClass*, so the interface will also be obtained. According to the previous article, enum will generate UEnum, and struct will generate UScriptStruct, so change the parameters to UEnum::StaticClass() to get all UEnum* objects, UScriptStruct::StaticClass() is all UScriptStruct*, and finally you can Type information is obtained through reflection based on these type objects.

And if you want to find a certain type of object based on a name exactly, you can use another method in UE4:

template< class T > 
inline T* FindObject( UObject* Outer, const TCHAR* Name, bool ExactClass=false )
{
    return (T*)StaticFindObject( T::StaticClass(), Outer, Name, ExactClass );
}

UClass* classObj=FindObject<UClass>(ANY_PACKAGE,"MyClass");   //获得表示MyClass的UClass*

This way you can easily get different types of objects. The internal principles of FindObject will be discussed in the next chapter on memory management.

Iterate over fields

After obtaining a type object, you can use various methods to traverse and find the internal fields. For this purpose, UE4 provides a convenient iterator TFieldIterator<T>, through which the traversed fields can be filtered.

const UStruct* structClass; //任何复合类型都可以
//遍历属性
for (TFieldIterator<UProperty> i(structClass); i; ++i)
{
    UProperty* prop=*i; 
}
//遍历函数
for (TFieldIterator<UFunction> i(structClass); i; ++i)
{
    UFunction* func=*i; 
    //遍历函数的参数
    for (TFieldIterator<UProperty> i(func); i; ++i)
    {
        UProperty* param=*i; 
        if( param->PropertyFlags & CPF_ReturnParm ) //这是返回值
        {

        }
    }
}
//遍历接口
const UClass* classObj; //只有UClass才有接口
for (const FImplementedInterface& ii : classObj->Interfaces)
{
    UClass* interfaceClass = ii.Class;
}

By passing UFunction to the template parameter T, you can get all the functions under the type. Through this, you can also traverse to get the parameter list under UFunction. Of course, TFieldIterator can also pass other parameters to control whether it contains fields of the base class, whether it contains obsolete fields, and whether it contains fields in the interface. The internal implementation of TFieldIterator is actually quite simple. One is to obtain Super through SuperStruct, the other is to obtain the implemented interface through Interfaces, and the third is to use Field->Next to traverse fields. The information data is complete, and iterative traversal is simple.

Traversing the fields of an enumeration is also simple:

const UEnum* enumClass;
for (int i = 0; i < enumClass->NumEnums(); ++i)
{
    FName name = enumClass->GetNameByIndex(i);
    int value = enumClass->GetValueByIndex(i);
}

There is also traversal of metadata:

#if WITH_METADATA
const UObject* obj;//可以是任何对象，但一般是UField才有值
UMetaData* metaData = obj->GetOutermost()->GetMetaData();
TMap<FName, FString>* keyValues = metaData->GetMapForObject(obj);
if (keyValues != nullptr&&keyValues->Num() > 0)
{
    for (const auto& i : *keyValues)
    {
        FName key=i.Key;
        FString value=i.Value;
    }
}
#endif

Of course, if you want to search accurately, there are corresponding methods.

//查找属性
UProperty* UStruct::FindPropertyByName(FName InName) const
{
    for (UProperty* Property = PropertyLink; Property != NULL; Property = Property->PropertyLinkNext)
    {
        if (Property->GetFName() == InName)
        {
            return Property;
        }
    }
    return NULL;
}

//查找函数
UFunction* UClass::FindFunctionByName(FName InName, EIncludeSuperFlag::Type IncludeSuper) const;

View inheritance

After getting the type object, you can also traverse to view its inheritance relationship. Traverse the inheritance chain:

const UStruct* structClass; //结构和类
TArray<FString> classNames;
classNames.Add(structClass->GetName());
UStruct* superClass = structClass->GetSuperStruct();
while (superClass)
{
    classNames.Add(superClass->GetName());
    superClass = superClass->GetSuperStruct();
}
FString str= FString::Join(classNames, TEXT("->")); //会输出MyClass->UObject

On the other hand, if you want to get all subclasses under a class, you can do this:

const UClass* classObj; //结构和类
TArray<UClass*> result;
GetDerivedClasses(classObj, result, false);
//函数原型是
void GetDerivedClasses(UClass* ClassToLookFor, TArray<UClass *>& Results, bool bRecursive);

GetDerivedClasses is also a method written in UE4. The HashMa method (TMap<UClass*, TSet<UClass*>> ClassToChildListMap) is used internally to save the mapping from classes to subclass lists.

So how to get all subclasses that implement a certain interface? Well, there is no good way, because it may not be used much, so this mapping relationship is not saved. We can only violently traverse it:

TArray<UObject*> result;
GetObjectsOfClass(UClass::StaticClass(), result);

TArray<UClass*> classes;
for (UObject* obj : result)
{
    UClass* classObj = Cast<UClass>(obj);
    if (classObj->ImplementsInterface(interfaceClass))//判断实现了某个接口
    {
        classes.Add(classObj);
    }
}

Get set attribute value

With UProperty, you can easily obtain its value through reflection:

template<typename ValueType>
ValueType* UProperty::ContainerPtrToValuePtr(void* ContainerPtr, int32 ArrayIndex = 0) const
{
    return (ValueType*)ContainerVoidPtrToValuePtrInternal(ContainerPtr, ArrayIndex);
}
template<typename ValueType>
ValueType* UProperty::ContainerPtrToValuePtr(UObject* ContainerPtr, int32 ArrayIndex = 0) const
{
    return (ValueType*)ContainerVoidPtrToValuePtrInternal(ContainerPtr, ArrayIndex);
}

void* UProperty::ContainerVoidPtrToValuePtrInternal(void* ContainerPtr, int32 ArrayIndex) const
{
    //check...
    return (uint8*)ContainerPtr + Offset_Internal + ElementSize * ArrayIndex;
}
void* UProperty::ContainerUObjectPtrToValuePtrInternal(UObject* ContainerPtr, int32 ArrayIndex) const
{
    //check...
    return (uint8*)ContainerPtr + Offset_Internal + ElementSize * ArrayIndex;
}
//获取对象或结构里的属性值地址，需要自己转换成具体类型
void* propertyValuePtr = property->ContainerPtrToValuePtr<void*>(object);
//包含对象引用的属性可以获得对象
UObject* subObject = objectProperty->GetObjectPropertyValue_InContainer(object);

UE4 specially added void* and UObject* overloads to obtain the attribute values ​​​​in the structure and object respectively. There are actually more check codes inside that I ignored. In UE4, external structures or object values ​​are called Containers, which is very reasonable. Isn’t the external thing wrapping the attributes a container? Another thing that can be seen is that obtaining the attribute value is actually very simple. Offset_Internal is the memory offset value of the attribute passed in during STRUCT_OFFSET() at the beginning. ElementSize is the memory size of the element. You can get the element value in the array through the ArrayIndex array index (such as the attribute of a fixed array such as int values[10]).

Also because what is obtained is the pointer address where the attribute value is stored, you can actually *propertyValuePtr=xxx; to set the value conveniently. Of course, if you import settings from strings, UE4 also provides two methods to export and import:

//导出值
virtual void ExportTextItem( FString& ValueStr, const void* PropertyValue, const void* DefaultValue, UObject* Parent, int32 PortFlags, UObject* ExportRootScope = NULL ) const; 

//使用
FString outPropertyValueString;
property->ExportTextItem(outPropertyValueString, property->ContainerPtrToValuePtr<void*>(object), nullptr, (UObject*)object, PPF_None);

//导入值
const TCHAR* UProperty::ImportText( const TCHAR* Buffer, void* Data, int32 PortFlags, UObject* OwnerObject, FOutputDevice* ErrorText = (FOutputDevice*)GWarn ) const;
//使用
FString valueStr;
prop->ImportText(*valueStr, prop->ContainerPtrToValuePtr<void*>(obj), PPF_None, obj);

ExportTextItem and ImportText are actually methods called when we select a property in the Details panel in the editor and perform Ctrl+C Copy and Ctrl+V Paste. UE4 actually serializes them into strings and passes them.

Reflection call function

Now that we can get UFunction, of course we can call them. Although this part of the mechanism is about blueprints, it’s not a bad idea to list them out in advance. The simplest way to call a UFunction method by name on a UObject is probably:

//方法原型
int32 UMyClass::Func(float param1); 

UFUNCTION(BlueprintCallable)
int32 InvokeFunction(UObject* obj, FName functionName,float param1)
{
    struct MyClass_Func_Parms   //定义一个结构用来包装参数和返回值，就像在gen.cpp里那样
    {
        float param1;
        int32 ReturnValue;
    };
    UFunction* func = obj->FindFunctionChecked(functionName);
    MyClass_Func_Parms params;
    params.param1=param1;
    obj->ProcessEvent(func, &params);
    return params.ReturnValue;
}
//使用
int r=InvokeFunction(obj,"Func",123.f);

Special note is that we need to define a memory to wrap and store parameters and return values, just like when registering a function. There is also such a piece of memory in gen.cpp. The Offset in the UProperty that represents the parameter is actually for this piece of memory. Therefore, in order to correctly retrieve the value based on the Offset, the field memory layout of this memory must be strictly consistent with the function registration time! Therefore, the order of field declaration must be consistent with that in gen.cpp. That is, in the order of parameters first and then the return value.

ProcessEvent is also a very convenient function pre-defined in UE4, which will automatically handle blueprint VM problems internally. Of course, a lower-level method can also be:

//调用1
obj->ProcessEvent(func, &params);
//调用2
FFrame frame(nullptr, func, &params, nullptr, func->Children);
obj->CallFunction(frame, &params + func->ReturnValueOffset, func);
//调用3
FFrame frame(nullptr, func, &params, nullptr, func->Children);
func->Invoke(obj, frame, &params + func->ReturnValueOffset);

Calling 123 is actually almost equivalent. To call the static function without obj, you can use calling 3. We know that the functions and events written in the blueprint will eventually be compiled into the generated UFunction object, so this method can directly call the member functions and custom events in the blueprint.

Of course, we have also seen that the above method also has inconveniences. We must manually define a parameter structure and a fixed function prototype. This is a method I wrote myself to call a function through reflection:

template<typename... TReturns, typename... TArgs>
void InvokeFunction(UClass* objClass, UObject* obj, UFunction* func, TTuple<TReturns...>& outParams, TArgs&&... args)
{
    objClass = obj != nullptr ? obj->GetClass() : objClass;
    UObject* context = obj != nullptr ? obj : objClass;
    uint8* outPramsBuffer = (uint8*)&outParams;

    if (func->HasAnyFunctionFlags(FUNC_Native)) //quick path for c++ functions
    {
        TTuple<TArgs..., TReturns...> params(Forward<TArgs>(args)..., TReturns()...);
        context->ProcessEvent(func, &params);
        //copy back out params
        for (TFieldIterator<UProperty> i(func); i; ++i)
        {
            UProperty* prop = *i;
            if (prop->PropertyFlags & CPF_OutParm)
            {
                void* propBuffer = prop->ContainerPtrToValuePtr<void*>(&params);
                prop->CopyCompleteValue(outPramsBuffer, propBuffer);
                outPramsBuffer += prop->GetSize();
            }
        }
        return;
    }

    TTuple<TArgs...> inParams(Forward<TArgs>(args)...);
    void* funcPramsBuffer = (uint8*)FMemory_Alloca(func->ParmsSize);
    uint8* inPramsBuffer = (uint8*)&inParams;

    for (TFieldIterator<UProperty> i(func); i; ++i)
    {
        UProperty* prop = *i;
        if (prop->GetFName().ToString().StartsWith("__"))
        {
            //ignore private param like __WolrdContext of function in blueprint funcion library
            continue;
        }
        void* propBuffer = prop->ContainerPtrToValuePtr<void*>(funcPramsBuffer);
        if (prop->PropertyFlags & CPF_OutParm)
        {
            prop->CopyCompleteValue(propBuffer, outPramsBuffer);
            outPramsBuffer += prop->GetSize();
        }
        else if (prop->PropertyFlags & CPF_Parm)
        {
            prop->CopyCompleteValue(propBuffer, inPramsBuffer);
            inPramsBuffer += prop->GetSize();
        }
    }

    context->ProcessEvent(func, funcPramsBuffer);   //call function
    outPramsBuffer = (uint8*)&outParams;    //reset to begin

    //copy back out params
    for (TFieldIterator<UProperty> i(func); i; ++i)
    {
        UProperty* prop = *i;
        if (prop->PropertyFlags & CPF_OutParm)
        {
            void* propBuffer = prop->ContainerPtrToValuePtr<void*>(funcPramsBuffer);
            prop->CopyCompleteValue(outPramsBuffer, propBuffer);
            outPramsBuffer += prop->GetSize();
        }
    }
}

Wow, so complicated! It's not my fault, but this function handles various situations.

1. The template class TTuple within UE4 itself is used to automatically package multiple parameters, so that there is no need to manually define the parameter structure. Multiple return values ​​are also returned using TTuple.
2. You can call member functions and static functions defined in C++, and also support calling member functions, events, and functions in the blueprint library. Because the functions in the blueprint support multiple input and multiple output values, after the blueprint function is compiled, it often has more parameters than in C++, and the temporary variable in the function is often regarded as a UProperty. So how to push parameters to the blueprint VM will require some extra steps than in C++. In the above function, I generated a piece of FMemory_Alloca(func->ParmsSize);memory on the stack as parameter memory when the function is run. Of course, this memory must be initialized from the function parameter set before the reflection call, and the value must be copied back to the return parameter after the reflection call.
3. When calling a static function, because UObject* is not actually needed, nullptr can be passed, so the corresponding UClass* can be used as the context calling object.
4. This function handles the most complex situation. For the situation with no parameters and no return value, readers can use variable parameter template partial specialization and overloading to define different versions. It's also relatively easy.
5. We also see that you need to provide UClass*, UObject* and UFunction* in the function parameters. These parameters are found using a similar method to FindFunctionByName above. We can also continue to add some convenience methods on top to expose it to the blueprint.

If you still want to call the blueprint function library by just providing the function name and parameters, just like calling the blueprint function library in the blueprint, you can do this:

template<typename... TReturns, typename... TArgs>
static void InvokeFunctionByName(FName functionName,TTuple<TReturns...>& outParams,TArgs&&... args)
{
    /*
    错误！在PIE模式下，有可能会获得SKEL_XXX_C:Func这个对象，里面的Script为空
    UFunction* func = FindObject<UFunction>(ANY_PACKAGE, *functionName.ToString()); 
    */
    UFunction* func = (UFunction*)StaticFindObjectFast(UFunction::StaticClass(), nullptr, functionName, false, true, RF_Transient); //exclude SKEL_XXX_C:Func
    InvokeFunction<TReturns...>(func->GetOuterUClass(), nullptr, func, outParams,Forward<TArgs>(args)...);
}

It is worth noting how UFunction* is searched. We cannot simply use FindObject to search, because the blueprint will generate two classes after compilation and before Cook: BPMyClass_C and SKEL_BPMyClass_C. The latter also has a UFunction* object with the same name, but the Script inside is empty, so it cannot be entered. Blueprint VM call. It is also observed that the ObjectFlags of this UFunction* object contains the RF_Transient flag (meaning not to save), so you can use this to filter out this object to find the correct UFunction* object.

If you wrap these functions yourself and make them BlueprintCallable, you can call functions through function names in both C++ and BP. I wish you good luck!

Runtime modification type

Let us continue to broaden our thinking. We have explained in detail the construction process of various types of objects before, and in the end, they are often called to UE4CodeGen_Private. Now that we already know the logic of its operation, we can also imitate it! We can also dynamically modify the type or even register the type during the middle of the game running after the regular type system registration process is executed, because after all, the UE4 editor is just a special game! This method is somewhat similar to C#'s emit method, which uses code to generate code and then compiles it. These methods are all feasible in theory. I will provide some ideas and usage. Friends who are interested can implement it by themselves. It would be too long to post the code.

1. Modifying the MetaData information of UField can actually change the display information of the field in the editor. You can view the fields in MetaData yourself in ObjectMacros.h.
2. Dynamically modifying the various corresponding Flags data of UField, such as PropertyFlags, StructFlags, ClassFlags, etc., can achieve the effect of dynamically changing its display behavior in the editor.
3. By dynamically adding and deleting the Names field in the UEnum object, you can dynamically add and delete enumeration items to the enum.
4. By dynamically adding reflective attribute fields to a structure or class, you can create structures with variable fields within Blueprints. Of course, the premise is to reserve memory for storing properties in the structure, so that the UProperty's Offset has a value to point to. Now that I think about it, I don't know what it can be used for.
5. Just like attributes, in fact, if you refer to the right process, you can also dynamically expose functions to the blueprint. Sometimes this can work wonders for some kind of encryption protection.
6. You can dynamically register new structures and dynamically construct the corresponding UScriptStruct.
7. It is actually possible to dynamically register new classes, but the construction of UClass is a little more troublesome, but it is not that troublesome. If there is a need, you can naturally implement a process yourself according to the process in the source code.
8. Even more, in fact, it is actually feasible to use code to dynamically create blueprint nodes and fill in blueprint VM instructions to some extent. But after thinking about it, it seems that such a major surgery is generally not necessary.

Summarize

In this article, we explain the application of type system reflection. In fact, these application knowledge must be based on a relatively deep understanding of the structural meaning of the type system. When you see a field in the editor and want to modify it, you should be able to think that there must be a corresponding field in C++. Even if it is hidden, you will also think of using reflection to get it.

The next article will start to explain UObject memory management. After learning memory management, you will have a better sense of control of whichever object you want in your heart. At the same time, you will also have a sense of how the object should be maintained and when it will be released. Now, we have a deeper understanding of why we were released.