Deep learning Lua-meta table

Metamethod

function	description
__add	Operator +
__sub	Operator-
__I have	Operator *
__ div	Operator /
__mod	Operator%
__unm	Operator-(negation)
__concat	Operator:
__eq	Operator ==
__lt	Operator<
__the	Operator<=
__call	When the function is called
__tostring	Convert to string
__index	Call an index
__newindex	Assign an index

Since those operators are similar in use, they will not be explained separately. Next, we will talk about the four meta methods of __call, __tostring, __index, and __newindex.

__call

__call allows table to be used as a function.

local mt = {}
--__call的第一参数是表自己
mt.__call = function(mytable,...)
    --输出所有参数
    for _,v in ipairs{...} do
        print(v)
    end
end

t = {}
setmetatable(t,mt)
--将t当作一个函数调用
t(1,2,3)

result:

1
2
3

__tostring

__tostring can modify the behavior of table conversion to string

local mt = {}
--参数是表自己
mt.__tostring = function(t)
    local s = "{"
    for i,v in ipairs(t) do
        if i > 1 then
            s = s..", "
        end
        s = s..v
    end
    s = s .."}"
    return s
end

t = {1,2,3}
--直接输出t
print(t)
--将t的元表设为mt
setmetatable(t,mt)
--输出t
print(t)

result:

table: 0x14e2050
{1, 2, 3}

__index

When calling a non-existent index of the table, the __index meta method of the meta table is used. Unlike the previous meta methods, __index can be a function or a table.
As a function:
Pass the table and index as parameters to the __index meta method, and return a return value

local mt = {}
--第一个参数是表自己，第二个参数是调用的索引
mt.__index = function(t,key)
    return "it is "..key
end

t = {1,2,3}
--输出未定义的key索引，输出为nil
print(t.key)
setmetatable(t,mt)
--设置元表后输出未定义的key索引，调用元表的__index函数，返回"it is key"输出
print(t.key)

result:

nil
it is key

As a table:
find the __index metamethod table, if there is the index, return the value corresponding to the index, otherwise return nil

local mt = {}
mt.__index = {key = "it is key"}

t = {1,2,3}
--输出未定义的key索引，输出为nil
print(t.key)
setmetatable(t,mt)
--输出表中未定义，但元表的__index中定义的key索引时，输出__index中的key索引值"it is key"
print(t.key)
--输出表中未定义，但元表的__index中也未定义的值时，输出为nil
print(t.key2)

result:

nil
it is key
nil

__newindex

When assigning a value to a non-existent index in the table, the __newindex metamethod in the meta table will be called
as
a function. When __newindex is a function, the table, index, and assignment values ​​in the assignment statement will be called as parameters. No changes to the table

local mt = {}
--第一个参数时表自己，第二个参数是索引，第三个参数是赋的值
mt.__newindex = function(t,index,value)
    print("index is "..index)
    print("value is "..value)
end

t = {key = "it is key"}
setmetatable(t,mt)
--输出表中已有索引key的值
print(t.key)
--为表中不存在的newKey索引赋值，调用了元表的__newIndex元方法，输出了参数信息
t.newKey = 10
--表中的newKey索引值还是空，上面看着是一个赋值操作，其实只是调用了__newIndex元方法，并没有对t中的元素进行改动
print(t.newKey)

result:

it is key
index is newKey
value is 10
nil

As the table
__newindex is a table, assigning a value to an index that does not exist in t will assign the index and value to the table pointed to by __newindex, without changing the original table.

local mt = {}
--将__newindex元方法设置为一个空表newTable
local newTable = {}
mt.__newindex = newTable
t = {}
setmetatable(t,mt)
print(t.newKey,newTable.newKey)
--对t中不存在的索引进行负值时，由于t的元表中的__newindex元方法指向了一个表，所以并没有对t中的索引进行赋值操作将，而是将__newindex所指向的newTable的newKey索引赋值为了"it is newKey"
t.newKey = "it is newKey"
print(t.newKey,newTable.newKey)

result:

nil nil
nil it is newKey

rawget and rawset

Sometimes when we want to directly change or get the values ​​in the table, we need rawget and rawset methods.
Rawget allows you to directly obtain the actual value of the index in the table, without passing through the __index meta method of the meta table.

local mt = {}
mt.__index = {key = "it is key"}
t = {}
setmetatable(t,mt)
print(t.key)
--通过rawget直接获取t中的key索引
print(rawget(t,"key"))

result:

it is key
nil

rawset allows you to directly assign values ​​to the indexes in the table without passing the __newindex meta method of the meta table.

local mt = {}
local newTable = {}
mt.__newindex = newTable
t = {}
setmetatable(t,mt)
print(t.newKey,newTable.newKey)
--通过rawset直接向t的newKey索引赋值
rawset(t,"newKey","it is newKey")
print(t.newKey,newTable.newKey)

result:

nil nil
it is newKey    nil

Use scenarios for meta tables

Metatable as table

Object-oriented programming can be implemented in Lua by setting metatables for the table.

Meta table as userdata

Object-oriented access to the structure in c in Lua can be achieved through userdata and metatables.

 

Weak reference table

 

Similar to scripting languages ​​such as python, Lua also uses automatic memory management (Garbage Collection), a program only needs to create objects without deleting objects. By using the garbage collection mechanism, Lua will automatically delete expired objects. The garbage collection mechanism can free programmers from low-level bugs such as memory leaks and invalid pointer references that often occur in the C language.

We know that Python's garbage collection mechanism uses a reference counting algorithm. When all the names pointing to an object are invalid (exceeding the lifetime or the programmer explicitly del), the memory occupied by the object will be reclaimed. But for circular references is a special case, the garbage collector usually can't recognize it, which will cause the reference counter on the object with circular references to never become zero, and there is no chance of being recycled.

An example of using circular references in python:

class main1:
    def __init__(self):
        print('The main1 constructor is calling...')
    def __del__(self):
        print('The main1 destructor is calling....')

class main2:
    def __init__(self, m3, m1):
        self.m1 = m1
        self.m3 = m3
        print('The main2 constructor is calling...')
    def __del__(self):
        print('The main2 destructor is calling....')

class main3:
    def __init__(self):
        self.m1  = main1()
        self.m2 = main2(self, self.m1)
        print('The main3 constructor is calling...')
    def __del__(self):
        print('The main3 destructor is calling....')

# test
main3()
       

The output is:

The main1 constructor is calling...
The main2 constructor is calling...
The main3 constructor is calling...

It can be seen that the destructor (__del__ function) is not called, and the circular reference causes a memory leak.

 

The garbage collector can only collect what it considers to be garbage, not what users consider to be garbage. For example, objects stored in global variables, even if the program will no longer use them, they are not garbage for Lua unless the user assigns these objects to nil so that they can be released. But sometimes, simply clearing the reference is not enough. For example, when an object is placed in an array, it cannot be recycled. This is because even if no other place is currently using it, the array still references it, unless the user tells it This reference to Lua should not hinder the recycling of this object, otherwise Lua would not know it.

There are key and value in the table, both of which can contain any type of object. Generally, the garbage collector will not reclaim an object that is a key or value in an accessible table. In other words, these keys and values ​​are strong references, and they will prevent the recycling of the referenced objects. In a weak reference table, key and value can be recycled.

A weak reference table (weak table) is used by users to tell Lua that a reference should not hinder the recovery of the object. The so-called weak reference is an object reference that will be ignored by the garbage collector. If the references of an object are weak references, the object will also be recycled, and the weak references themselves can be deleted in some form.

There are 3 types of weak reference tables:

1. Table with weak reference key;
2. Table with weak reference value;
3. Table with weak reference key and value at the same time;

The weak reference type of a table is determined by the __mode field in its metatable. The value of this field should be a string:
if it contains'k', then the key of this table is weakly quoted;
if it contains'v', then the value of this table is weakly quoted;

 

An example of weakly referencing a table, here is the use of the collectgarbage function to force a garbage collection:

a = {1,4, name='cq'}

setmetatable(a, {__mode='k'})

key = {}
a[key] = 'key1'

key = {}
a[key] = 'key2'

print("before GC")
for k, v in pairs(a) do
    print(k, '\t', v)
end

collectgarbage()

print("\nafter GC")
for k, v in pairs(a) do
    print(k, '\t', v)
end

Output:

before GC
1                       1
2                       4
table: 0x167ba70                        key1
name                    cq
table: 0x167bac0                        key2

after GC
1                       1
2                       4
name                    cq
table: 0x167bac0                        key2

In this example, the second sentence assignment key={} will overwrite the first key. When the collector runs, because there is no place to reference the first key, the first key is recycled, and the table The corresponding entry has also been deleted. As for the second key, the variable key still references it, so it has not been recycled.

Note that only objects in weak reference tables can be recycled, while "values" like numbers, strings, and booleans are not.

 

 

The memoize function is a practice of using space for time. For example, if an ordinary server receives a request, it must call loadstring on the code string, and then call the compiled function. However, loadstring is an expensive function. Some commands sent to the server have a high frequency, such as "close()". If loadstring is called every time such a command is received, it is better to let the server use an auxiliary The table records the results of all calls to loadstring.

Examples of memo functions:

local results = {}

setmetatable(results, {__mode='v'})

function mem_loadstring(s)

    local res = results[s]

    if res == nil then
        res=assert(loadstring(s))
        results[s]=res
    end

    return res
end 

local a = mem_loadstring("print 'hello'")
local b = mem_loadstring("print 'world'")

a = nil

collectgarbage()

for k,v in pairs(results) do
    print(k, '\t', v)
end

In the example, table results will gradually accumulate all commands received by the server and their compilation results. After a certain period of time, a lot of memory will be consumed. Weak reference table can just solve this problem. If the results table has a weak reference value, then every garbage collection will delete all unused compilation results during execution.

 

In the article Lua Metatable , it was mentioned how to implement a table with default values. If you want to set a default value for each table and don't want these default values ​​to persist, you can also use weak references to the table, as in the following example:

local defaults = {}

setmetatable(defaults, {__mode='k'})

local mt = {__index=function(t) return defaults[t] end}

function setDefault(t, d)
    defaults[t] = d
    setmetatable(t, mt)
end 


local a = {}
local b = {}

setDefault(a, "hello")
setDefault(b, "world")

print(a.key1)
print(b.key2) 

b = nil
collectgarbage()

for k,v in pairs(defaults) do
    print(k,'\t',v)
end

Application of weak references: memory leak detection in Lua

There are also two levels to detect memory leaks in Lua.

Level 1: Detect the leakage of lua objects in the lua layer and the leakage of c objects caused by the lua layer at the same time.

Level 2: Only detect the leakage of lua objects in the lua layer.

The memory leak detection that meets level 1 is relatively slightly complicated, but the principle is still relatively simple, that is, scan the lua state at two points in time and obtain a snapshot, then compare the two snapshots, and the extra memory reference in the latter one is The memory allocated at the time of the next card is more than that of the previous card. If these two time points are functionally logically consistent (for example, the previous time point is before entering a copy, and the next time point is after exiting a copy), then the extra memory reference is the leaked memory.

Here we will focus on the implementation of level 2. It is also very important to only detect the leakage of lua objects in the lua layer. The code scale is large, it is very difficult to detect memory leaks through keen eyesight and rigorous brain analysis, so you need certain tools to help yourself. Here you can use the "weak reference" provided by lua. "Weak reference" is a kind of reference. Its reference to an object will not affect the garbage collection of the object by the lua gc mechanism. That is to say, if an object has only weak references, then the object will Reclaimed by gc-in short, gc ignores weak references.

Therefore, we can establish a global memory leak monitoring weak reference table whose references to key values ​​and content values ​​are "weak references" (that is, the __mode element attribute value in the metatable is "kv"), and the objects we care about Place it in the table. Then wait for a certain moment when we think that this object should have been recycled, and check whether there is still this object in the table: if it does not exist, it means that the object has been recycled correctly; if it exists, it means that the object has not been recycled correctly. In short, the subject leaked.