An epoch-making code hot update solution huatuo source code flow analysis

foreword

The day before yesterday, I heard that a hot update solution called huatuo was born. It performs hot update on user code based on IL2CPP. I thought it would implement a whole set of virtual machine runtimes like some hot update solutions, but after reading the source code, I The expression was greatly shocked, and at the same time admired the author's brain circuit, which is simply wonderful.

Hot Update Basics

We need to first understand the two scripting backends of Mono and IL2Cpp

The first is Mono, the structure is quite simple, that is, Mono VM interprets and executes IL code

C++Then there is IL2CPP, which is relatively complicated. It needs to translate IL into C++ code, and then use the IL2CPP virtual machine runtime to execute our translated code after being compiled by a local compiler C++. The IL2CPP runtime has done quite a lot. Including but not limited to supported C#reflection, GC, generics, virtual functions, Interface, thread management, etc., so that C# can really run on the IL2CPP virtual machine

Aside from the IOS and some host platforms’ prohibition of JIT policies, C++IL2CPP itself is AOT due to language restrictions, so C#operations like LoadAssembly, Emit, and Expression, which were originally supported by JIT, are not supported by IL2CPP. It is also the reason why we need various code hot update solutions

huatuo

Introduction

huatuo is a nearly perfect c# hot update solution with complete features, zero cost, high performance and low memory .

huatuo provides a very complete cross-platform CLR runtime, which can efficiently execute in AOT+interpreter hybrid mode not only on the Android platform, but also on IOS, Consoles and other platforms that limit JIT.

huatuo features:

• Features complete. A nearly complete implementation of the ECMA-335 specification , all features are supported except those described in "Limitations and Caveats" below.
• Zero learning and use costs. huatuo is a complete CLR runtime, hot update code works seamlessly with AOT code. No need to write any special code, no code generation, and no special restrictions. The script class and the AOT class are in the same runtime, even codes such as reflection and multi-threading (volatile, ThreadStatic, Task, async) can work normally.
• Execute efficiently. Implemented an extremely efficient register interpreter, all indicators are significantly better than other hot update schemes. performance test report
• Memory efficient. The classes defined in the hot update script occupy the same memory space as ordinary c# classes, which is far superior to other hot update solutions. memory usage report
• Native support for repairing some AOT codes. No additional development and runtime overhead.

More specifically, huatuo has done the following:

• Implemented an efficient metadata (dll) parsing library
• Modified the metadata management module of il2cpp to realize the dynamic registration of metadata
• Implemented a compiler from an IL instruction set to a custom register instruction set
• Implemented an efficient register interpreter
• A large number of instinct functions are provided to improve the performance of the interpreter
• Provide hotfix AOT support (in progress)

installation and use

https://focus-creative-games.github.io/huatuo/start_up/

infrastructure

The basic principle of huatuo is very simple. For AOT code, use IL2CPP to execute, and for non-AOT code (such as the dll we provide), it uses huatuo to interpret and execute

Run the flowchart

First look at a large flow chart, including huatuo's Init, LoadAssembly and Execute processes, where the left side is the IL2CPP layer call, and the right side is the huatuo layer call

Run the flowchart

Init

Called by Unity Native, executed bool Runtime::Init, and then called all the way tovoid ModuleManager::Initialize

C++

void InterpreterModule::Initialize()
{
    
    
	for (size_t i = 0; ; i++)
	{
    
    
		NativeCallMethod& method = g_callStub[i];
		if (!method.signature)
		{
    
    
			break;
		}
		s_calls.insert_or_assign(method.signature, method);
	}
	for (size_t i = 0; ; i++)
	{
    
    
		NativeInvokeMethod& method = g_invokeStub[i];
		if (!method.signature)
		{
    
    
			break;
		}
		s_invokes.insert_or_assign(method.signature, method);
	}
}

Among them, g_callStub and g_invokeStub are the instinct functions provided by huatuo

C++

NativeCallMethod huatuo::interpreter::g_callStub[] = 
{
    
    
	{
    
    "v", (Il2CppMethodPointer)__Native2ManagedCall_v, (Il2CppMethodPointer)__Native2ManagedCall_AdjustorThunk_v, __Managed2NativeCall_v},
	{
    
    "vi", (Il2CppMethodPointer)__Native2ManagedCall_vi, (Il2CppMethodPointer)__Native2ManagedCall_AdjustorThunk_vi, __Managed2NativeCall_vi},
	{
    
    "vf", (Il2CppMethodPointer)__Native2ManagedCall_vf, (Il2CppMethodPointer)__Native2ManagedCall_AdjustorThunk_vf, __Managed2NativeCall_vf},
	{
    
    "vii", (Il2CppMethodPointer)__Native2ManagedCall_vii, (Il2CppMethodPointer)__Native2ManagedCall_AdjustorThunk_vii, __Managed2NativeCall_vii},
	{
    
    "vfi", (Il2CppMethodPointer)__Native2ManagedCall_vfi, (Il2CppMethodPointer)__Native2ManagedCall_AdjustorThunk_vfi, __Managed2NativeCall_vfi},
	....
}
 
NativeInvokeMethod huatuo::interpreter::g_invokeStub[] = 
{
    
    
	{
    
    "v", __Invoke_instance_v, __Invoke_static_v},
	{
    
    "vi", __Invoke_instance_vi, __Invoke_static_vi},
	{
    
    "vf", __Invoke_instance_vf, __Invoke_static_vf},
	{
    
    "vii", __Invoke_instance_vii, __Invoke_static_vii},
	{
    
    "vfi", __Invoke_instance_vfi, __Invoke_static_vfi},
	{
    
    "vif", __Invoke_instance_vif, __Invoke_static_vif},
	{
    
    "vff", __Invoke_instance_vff, __Invoke_static_vff},
    ...
}

Here we take the commonly used function __Invoke_instance_v as an example. We can ignore some data related to stack frame construction, but we need to pay attention to it. Interpreter::ExecuteThis is the starting point of huatuo's formal interpretation and execution. We will continue to mention it later

C++

static void* __Invoke_static_v(Il2CppMethodPointer methodPtr, const MethodInfo* method, void* __this, void** __args)
{
    
    
    StackObject args[1] = {
    
     };
    ConvertInvokeArgs(args, method, __args);
    StackObject* ret = nullptr;
     Important 
    Interpreter::Execute(method, args, ret);
     Important 
    return nullptr;
}

LoadAssemblyLoadAssembly

Also initiated by Unity Native, the starting point is the IL2CPP layer const Il2CppAssembly* Assembly::Load, and then it will be called to the huatuo layer Il2CppAssembly* Assembly::Createto parse the Dll and generate the Image, Assembly and metadata information needed by IL2CPP and huatuo. The analysis of Assembly on huatuo is implemented by the author. It is quite hard-core. It can be considered as a one-to-one reproduction of the CLR's analysis behavior of Dll. In this process, of course, some explanations are required on 很多元数据类型都是直接使用的IL2CPP中的（比如Il2CppTypeDefinition，Il2CppMethodDefinition等），这也是为了与IL2CPP直接Hook联调打下基础huatuo Execute the operation, so some huatuo-specific metadata will also be generated (such as TbMethod, MethodBody, ilcodedata pointer, etc., many of which also wrap the IL2CPP metadata type)

The more important thing is the structure of the Method metadata. You can refer to the Method Table of the CLR, which is almost the same.

CLR的Method Table

Execute

Let's take the official demonstration Demo as an example. When the function LoadDll.csof the script is called by the LoadGameDll function APP.csof the script APP.Main, the stack is as follows

C++

 	GameAssembly.dll!il2cpp::vm::SetupMethodsLocked(Il2CppClass * klass, const il2cpp::os::FastAutoLock & lock) 行 1192	C++
 	GameAssembly.dll!il2cpp::vm::Class::SetupMethods(Il2CppClass * klass) 行 1279	C++
 	GameAssembly.dll!huatuo::metadata::GetMethodInfoFromMethodDef(const Il2CppType * type, const Il2CppMethodDefinition * methodDef) 行 572	C++
 	GameAssembly.dll!huatuo::metadata::GetMethodInfo(const Il2CppType * containerType, const Il2CppMethodDefinition * methodDef, const Il2CppGenericInst * instantiation, const Il2CppGenericContext * genericContext) 行 1076	C++
 	GameAssembly.dll!huatuo::metadata::ReadMethodInfoFromToken(huatuo::metadata::Image & image, const Il2CppGenericContainer * klassGenericContainer, const Il2CppGenericContainer * methodGenericContainer, const Il2CppGenericContext * genericContext, Il2CppGenericInst * genericInst, huatuo::metadata::TableType tableType, unsigned int rowIndex) 行 1135	C++
 	GameAssembly.dll!huatuo::metadata::Image::GetMethodInfoFromToken(unsigned int token, const Il2CppGenericContainer * klassGenericContainer, const Il2CppGenericContainer * methodGenericContainer, const Il2CppGenericContext * genericContext) 行 1195	C++
 	GameAssembly.dll!huatuo::transform::HiTransform::Transform(huatuo::metadata::Image * image, const MethodInfo * methodInfo, huatuo::metadata::MethodBody & body, huatuo::interpreter::InterpMethodInfo & result) 行 2405	C++
 	GameAssembly.dll!huatuo::interpreter::InterpreterModule::GetInterpMethodInfo(huatuo::metadata::Image * image, const MethodInfo * methodInfo) 行 332	C++
 	GameAssembly.dll!huatuo::interpreter::Interpreter::Execute(const MethodInfo * methodInfo, huatuo::interpreter::StackObject * args, huatuo::interpreter::StackObject * ret) 行 888	C++
>	GameAssembly.dll!__Invoke_static_i(void(*)() methodPtr, const MethodInfo * method, void * __this, void * * __args) 行 22127	C++
 	GameAssembly.dll!il2cpp::vm::Runtime::Invoke(const MethodInfo * method, void * obj, void * * params, Il2CppException * * exc) 行 575	C++
 	GameAssembly.dll!il2cpp::vm::InvokeConvertThis(const MethodInfo * method, void * thisArg, void * * convertedParameters, Il2CppException * * exception) 行 684	C++
 	GameAssembly.dll!il2cpp::vm::Runtime::InvokeConvertArgs(const MethodInfo * method, void * thisArg, Il2CppObject * * parameters, int paramCount, Il2CppException * * exception) 行 778	C++
 	GameAssembly.dll!il2cpp::vm::Runtime::InvokeArray(const MethodInfo * method, void * obj, Il2CppArray * params, Il2CppException * * exc) 行 594	C++
 	GameAssembly.dll!il2cpp::icalls::mscorlib::System::Reflection::MonoMethod::InternalInvoke(Il2CppReflectionMethod * method, Il2CppObject * thisPtr, Il2CppArray * params, Il2CppException * * exc) 行 240	C++
 	GameAssembly.dll!MonoMethod_InternalInvoke_mFF7E631020CDD3B1CB47F993ED05B4028FC40F7E(MonoMethod_t * __this, Il2CppObject * ___obj0, ObjectU5BU5D_tC1F4EE0DB0B7300255F5FD4AF64FE4C585CF5ADE * ___parameters1, Exception_t * * ___exc2, const MethodInfo * method) 行 39012	C++
 	GameAssembly.dll!MonoMethod_Invoke_mD6E222F8DAB5483E6640B8E399A56B366635B923(MonoMethod_t * __this, Il2CppObject * ___obj0, int ___invokeAttr1, Binder_t2BEE27FD84737D1E79BC47FD67F6D3DD2F2DDA30 * ___binder2, ObjectU5BU5D_tC1F4EE0DB0B7300255F5FD4AF64FE4C585CF5ADE * ___parameters3, CultureInfo_t1B787142231DB79ABDCE0659823F908A040E9A98 * ___culture4, const MethodInfo * method) 行 39080	C++
 	GameAssembly.dll!VirtFuncInvoker5<Il2CppObject *,Il2CppObject *,int,Binder_t2BEE27FD84737D1E79BC47FD67F6D3DD2F2DDA30 *,ObjectU5BU5D_tC1F4EE0DB0B7300255F5FD4AF64FE4C585CF5ADE *,CultureInfo_t1B787142231DB79ABDCE0659823F908A040E9A98 *>::Invoke(unsigned short slot, Il2CppObject * obj, Il2CppObject * p1, int p2, Binder_t2BEE27FD84737D1E79BC47FD67F6D3DD2F2DDA30 * p3, ObjectU5BU5D_tC1F4EE0DB0B7300255F5FD4AF64FE4C585CF5ADE * p4, CultureInfo_t1B787142231DB79ABDCE0659823F908A040E9A98 * p5) 行 71	C++
 	GameAssembly.dll!MethodBase_Invoke_m5DA5E74F34F8FFA8133445BAE0266FD54F7D4EB3(MethodBase_t * __this, Il2CppObject * ___obj0, ObjectU5BU5D_tC1F4EE0DB0B7300255F5FD4AF64FE4C585CF5ADE * ___parameters1, const MethodInfo * method) 行 18289	C++
 	GameAssembly.dll!LoadDll_RunMain_mEDAF0764CCCFDE2F0B9801051CCD5FFDF0241B4C(LoadDll_tF4302664700CA4FCBC0471B8C95631AE6442BC68 * __this, const MethodInfo * method) 行 41550	C++
 	GameAssembly.dll!LoadDll_Start_m9DCFAB46D91AA07BA3DD00B2F19473B66E1E78DB(LoadDll_tF4302664700CA4FCBC0471B8C95631AE6442BC68 * __this, const MethodInfo * method) 行 41413	C++
 	GameAssembly.dll!RuntimeInvoker_TrueVoid_t700C6383A2A510C2CF4DD86DABD5CA9FF70ADAC5(void(*)() methodPointer, const MethodInfo * methodMetadata, void * obj, void * * args) 行 216780	C++
 	GameAssembly.dll!il2cpp::vm::Runtime::Invoke(const MethodInfo * method, void * obj, void * * params, Il2CppException * * exc) 行 575	C++
 	GameAssembly.dll!il2cpp_runtime_invoke(const MethodInfo * method, void * obj, void * * params, Il2CppException * * exc) 行 1118	C++
 	[外部代码]	
 	huatuo.exe!wWinMain(HINSTANCE__ * hInstance, HINSTANCE__ * hPrevInstance, wchar_t * lpCmdLine, int nShowCmd) 行 16	C++
 	[外部代码]	

In particular, it should be noted that il2cpp::vm::SetupMethodsLocked, 这个函数会将MethodInfo的methodPointer和invoker_method绑定好（我们初始化时构造的那些s_calls和s_invokes）is used to hook to huatuo for interpretation and execution

Function pointer binding in MethodInfo

methodPointer：

C++

Il2CppMethodPointer il2cpp::vm::MetadataCache::GetMethodPointer(const Il2CppImage* image, uint32_t token)
{
    
    
    uint32_t rid = GetTokenRowId(token);
    uint32_t table =  GetTokenType(token);
    if (rid == 0)
        return NULL;
 
    // === huatuo
    if (huatuo::metadata::IsInterpreterImage(image))
    {
    
    
        return huatuo::metadata::MetadataModule::GetMethodPointer(image, token);
    }
    // === huatuo
 
    IL2CPP_ASSERT(rid <= image->codeGenModule->methodPointerCount);
 
    return image->codeGenModule->methodPointers[rid - 1];
}

invoker_method：

C++

InvokerMethod il2cpp::vm::MetadataCache::GetMethodInvoker(const Il2CppImage* image, uint32_t token)
{
    
    
    uint32_t rid = GetTokenRowId(token);
    uint32_t table = GetTokenType(token);
    if (rid == 0)
        return NULL;
    // === huatuo
    if (huatuo::metadata::IsInterpreterImage(image))
    {
    
    
        return huatuo::metadata::MetadataModule::GetMethodInvoker(image, token);
    }
    // === huatuo
    int32_t index = image->codeGenModule->invokerIndices[rid - 1];
 
    if (index == kMethodIndexInvalid)
        return NULL;
 
    IL2CPP_ASSERT(index >= 0 && static_cast<uint32_t>(index) < s_Il2CppCodeRegistration->invokerPointersCount);
    return s_Il2CppCodeRegistration->invokerPointers[index];
}

IL2CPP Side Summary

In other cases, the stack may be different from the example here, but the core is the same:

1. Called by Unity Native
2. Execution il2cpp::vm::SetupMethodsLockedconstructs function pointers in MethodInfo (if needed)
3. huatuo::interpreter::Interpreter::Executeexplain execution
4. (Not necessarily) Call the IL2CPP function to complete the function realization. For some simple mathematical operations and logical huatuo custom implementations, if some functions IL2CPP has provided the corresponding interface, then call the IL2CPP interface

Interpreter::Execute

This part has been described in detail in the flowchart. This function is the core of huatuo interpretation and execution, and there are about 7k lines of switch invoke.

C++

void Interpreter::Execute(const MethodInfo* methodInfo, StackObject* args, StackObject* ret)
{
    
    
	INIT_CLASS(methodInfo->klass);
	MachineState& machine = InterpreterModule::GetCurrentThreadMachineState();
	InterpFrameGroup interpFrameGroup(machine);
	const InterpMethodInfo* imi;
	InterpFrame* frame;
	StackObject* localVarBase;
	byte* ipBase;
	byte* ip;
	PREPARE_NEW_FRAME(methodInfo, args, ret, false);
	// exception handler
	Il2CppException* curException = nullptr;
LoopStart:
	try
	{
    
    
		Execute Interpreter ... 
	}
	catch (Il2CppExceptionWrapper ex)
	{
    
    
		curException = ex.ex;
		PREPARE_EXCEPTION();
		FIND_NEXT_EX_HANDLER_OR_UNWIND();
	}
	catch (Il2CppException* ex)
	{
    
    
		curException = ex;
		PREPARE_EXCEPTION();
		FIND_NEXT_EX_HANDLER_OR_UNWIND();
	}
	return;
UnWindFail:
	IL2CPP_ASSERT(curException);
	interpFrameGroup.CleanUpFrames();
	il2cpp::vm::Exception::Raise(curException);
}

In PREPARE_NEW_FRAME, the MethodInfo that can be recognized by huatuo will be obtained first InterpreterModule::GetInterpMethodInfo. If there is already a cache, it will be returned directly without escaping.

C++

InterpMethodInfo* InterpreterModule::GetInterpMethodInfo(metadata::Image* image, const MethodInfo* methodInfo)
{
    
    
	il2cpp::os::FastAutoLock lock(&il2cpp::vm::g_MetadataLock);
	if (methodInfo->huatuoData)
	{
    
    
		return (InterpMethodInfo*)methodInfo->huatuoData;
	}
	metadata::MethodBody& originMethod = image->GetMethodBody(methodInfo->token);
	InterpMethodInfo* imi = new (IL2CPP_MALLOC_ZERO(sizeof(InterpMethodInfo))) InterpMethodInfo;
	transform::HiTransform::Transform(image, methodInfo, originMethod, *imi);
	il2cpp::os::Atomic::FullMemoryBarrier();
	const_cast<MethodInfo*>(methodInfo)->huatuoData = imi;
	return imi;
}

If there is no cache transform::HiTransform::Transform, translate by and cache the translated codes in il2cpp-class-internals::MethodInfo.huatuoData

other

Source Code Debug

https://focus-creative-games.github.io/huatuo/source_inspect/

Breakpoints can be easily traced to the script execution process after the package is exported

Summarize

We can see that, in fact huatuo自身并没有一个完整的虚拟机系统，而是借由Unity Native和IL2CPP本身驱动执行自己的一套简单的解释执行栈帧，并且由于其自身就是IL2CPP的拓展，所以跨域调用性能也很强劲（毕竟只是几次指针跳转和函数调用）, C++quite powerful performance can be obtained by means of pointer offsets and function calls

Why is it an epoch-making code hot update solution

Consider how the vast majority of commercial projects are currently suffering

• A project is more likely to mix multiple languages ​​for heat, Lua, C#, TS, and a framework must be implemented in each language. The reason is either that the performance of cross-domain calls is too ugly, or it can only be done in this way
• If you want to connect a third-party plug-in that may involve hot-updates, to be stable and usable, you should not run out of that one-week man-day, such as Protobuf, you have to specially modify its source code to adapt to the C# hot-update framework if you connect it to the hot-update C#layer The implementation of various redirections can only get a plug-in that is still trembling and must be protected as a baby and dare not be used casually, not to mention Lua, the lua-protobuf annotation tool alone is enough for people to drink a pot, and there is also It is necessary to modify and customize the source code according to the needs of the project. What is the crux of the problem? Because these hot update programs are separated from the CLR Runtime, the CLR does not know that there is such additional information, so it can only be artificially constructed and maintained.
• C#The high-level features are obedient and dare not use, for fear of alarming that adult
• Cross-domain calls and cross-domain inheritance, these two originally quite natural functions, require developers to pay attention to a lot of details, redirection, and code generation, otherwise it will be overwhelming GC and Caton
• After all, Lua and C# are things in different domains. p/invokeHow much room is there for developers to make a big difference after all kinds of things come down?
• Especially criticizing Lua by name, can this thing really be used as a game framework and business? Rewriting is faster than refactoring. Another brother took over and sent it directly.

And these are not a problem for huatuo, because he supports hot update from the bottom layer. Huatuo's expansion of IL2CPP makes Unity Native see that our hot update code and AOT code are the same, that is to say, even if we run IL2CPP backend , It can also be written as a PC or Android under the Mono backend.