[Note] Advanced python metaclass Metaclasses, implement ORM

[Note] Advanced python metaclass Metaclasses, implement ORM

table of Contents

1. Moto类

1.1. The class is also an object

1.2. Creating dynamic classes

1.3. Use type create a class

1.4 The use of type to create complex class

1.4.1. Type create a class with inheritance

1.4.2. Examples of the method is added

1.4.3. Adding static method

1.4.4. Adding class methods

1.4.5. Creating a more complete use of type classes

1.5 metaclass defines

1.6. __Metaclass__ property

1.7. Metaclasses

. 1.8 Update: __call__ Method metaclasses

Note: new meta class, init, call

1.9. Metaclasses usage scenarios 

2. metaclass implement ORM (Object Relational Mapping)

What 2.1. ORM is

2.2. ORM simple implementation of the function insert by the metaclass

---

 

1. Moto类

Yuan class is used to create a class of "things", Yuan class create a class, the class creates an instance of an object .

1.1. The class is also an object

In most programming languages, the class is used to describe how to generate a set of code segments of an object . This remains true in Python:

In the following ipython3 by class generates an object and outputs:

>>>class ObjectCreator(object):
…       pass
…
>>> my_object = ObjectCreator()
>>> print(my_object)
<__main__.ObjectCreator object at 0x8974f2c>

 

However, Python middle class is far more than that. Class is also a kind of object . Just use the keyword class, Python interpreter will automatically create an object at the time of execution.

The following code snippet:

class ObjectCreator(object):
    pass

python will automatically create an object in memory, the name is ObjectCreator.

This object (the class object ObjectCreator) has the ability to create objects (instances of objects) is. However, it is still essentially an object , it can do operations on objects:

• Assigned to a variable
• Copy it
• As it increases property
• Passed as function parameters

 

Note:

• globals ()  function returns the current location of all global variables in a dictionary type, including references to all global objects. That returned dictionary will contain python-defined variables, also contains its own definition of variables, it can be used directly.
• To get a class or module what is inside, you can use  the class .__ dict__  or  module .__ dict__ .
• Globals () returns a built-in module __builtins__, which contains other conventional print objects (__builtins __.__ dict__ visible).
• When using a variable or function names, first directly to the globals () returns a dictionary to find inside, and then can not find a dictionary built-in modules __builtins__ go.

 

 

1.2. Creating dynamic classes

Because the classes are objects, they can be created dynamically at run time, just like any other object. You can create class function, use the class keyword.

def choose_class(name):
    if name == 'foo':
        class Foo(object):
            pass
        return Foo     # 返回的是类，不是类的实例
    else:
        class Bar(object):
            pass
        return Bar
MyClass = choose_class('foo')
print(MyClass)  # 函数返回的是类，不是类的实例

print(MyClass())  # 你可以通过这个类创建类实例，也就是对象

# 输出：
# <class '__main__.choose_class.<locals>.Foo'>
# <__main__.choose_class.<locals>.Foo object at 0x7fa8b5ac2f98>

The transmission parameters are not the same, the returned reference is not the same class, and then led to the creation of an object instance is not the same. But this is not enough dynamic , because still need to write code for the entire class of their own. You can use to create a dynamic type class . ( Note : one type can be function-return type of an object, such as print (type (1)) will print <type 'int'>)
 

1.3. Use type create a class

There is also a completely different type of function, namely to create a dynamic class . Function type is actually a meta-class.

describe a class type is acceptable as an argument, and returns a class object .

type (class name, parent name (for the case of inheritance, can be empty) by a tuple consisting of, containing attributes dictionary (names and values))

 

The following are two ways to create a class and class Test1 Test2:

In [13]: class Test:
   ....:     num = 100
   ....:     num2 = 200
   ....:     

In [14]: Test2 = type("Test2", (), {"num":100, "num2":200})

Use help to test these two categories:

In [16]: help(Test)  # 用help查看Test类

Help on class Test in module __main__:

class Test(builtins.object)
 |  Data descriptors defined here:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)
 |  
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |  
 |  num = 100
 |  
 |  num2 = 200


In [18]: help(Test2)

Help on class Test2 in module __main__:

class Test2(builtins.object)
 |  Data descriptors defined here:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)
 |  
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |  
 |  num = 100
 |  
 |  num2 = 200

Obviously, the two are the same.

 

1.4 The use of type to create complex class

 

1.4.1. Type create a class with inheritance

Test11 inherited Test, Test22 inherited Test2.

In [1]: class Test:
    num = 100
    num2 = 200
   ...:     

In [2]: Test2 = type("Test2", (), {"num":100, "num2":200})

In [3]: class Test11(Test):
   ...:     pass
   ...: 

In [4]: Test22 = type("Test22", (Test2,), {})

Then use the help view, two classes are the same.

 

1.4.2. Examples of the method is added

In [15]: class Foo(object):
   ....:     bar = True
   ....:     

In [16]: def echo_bar(self):   # 定义了一个普通的函数
   ....:     print(self.bar)
   ....:    
 
In [17]: FooChild = type('FooChild', (Foo,), {'echo_bar': echo_bar})  # 让FooChild类中的echo_bar属性，指向了上面定义的函数

In [18]: hasattr(Foo, 'echo_bar')  # 判断Foo类中 是否有echo_bar这个属性
Out[18]: False

In [19]: hasattr(FooChild, 'echo_bar')  # 判断FooChild类中 是否有echo_bar这个属性
Out[19]: True

In [20]: my_foo = FooChild()

In [21]: my_foo.echo_bar()
True

1.4.3. Adding static method

On the basis of the above code

In [36]: @staticmethod
    ...: def test_static():
    ...:     print("static method ....")
    ...:

In [37]: Foochild = type('Foochild', (Foo,), {"echo_bar": echo_bar, "test_static": test_static})

In [38]: fooclid = Foochild()

In [39]: fooclid.test_static
Out[39]: <function __main__.test_static>

In [40]: fooclid.test_static()
static method ....

In [41]: fooclid.echo_bar()
True

 

1.4.4. Adding class methods

On the basis of the above code

In [42]: @classmethod
    ...: def test_class(cls):
    ...:     print(cls.bar)
    ...:

In [43]: Foochild = type('Foochild', (Foo,), {"echo_bar":echo_bar, "test_static": test_static, "test_class": test_class})

In [44]: fooclid = Foochild()

In [45]: fooclid.test_class()
True

 

1.4.5. Creating a more complete use of type classes

class A(object):
    num = 100

def print_b(self):
    print(self.num)

@staticmethod
def print_static():
    print("----haha-----")

@classmethod
def print_class(cls):
    print(cls.num)

# B继承于A,
B = type("B", (A,), {"print_b": print_b, "print_static": print_static, "print_class": print_class})
b = B()
b.print_b()
b.print_static()
b.print_class()
# 结果
# 100
# ----haha-----
# 100

 

1.5 metaclass defines

 

• Yuan class is used to create a class of "stuff" . Create a class to create an instance of an object class.
• However, classes in Python is an object, the metaclass is used to create these classes (objects), the metaclass is the class of the class .
• That metaclass create a class, the class creates an instance of an object .

MyClass = MetaClass() # 使用元类创建出一个对象，这个对象称为“类”
my_object = MyClass() # 使用“类”来创建出实例对象

Used earlier function type is actually a meta-class. Behind Python type is used to create all kind of metaclass .

• type is to create a class object class . You can verify this by checking __class__ property.

( Note : this example will point to the corresponding class instance calls __class__ attributes (ie, the class creates the instance), you can also go call other class attributes)

• Everything in Python, are objects . It includes an integer, string functions, and class. All they are objects, and they are coming from to create a class that is the type.

 

In [24]: class T(object):
   ....:     pass
   ....: 

In [25]: t = T()

In [26]: t.__class__
Out[26]: __main__.T

# 以上可见t由当前模块的T创建


In [27]: t.__class__.__class__
Out[27]: type

# 以上可见T由type创建 (注：type还是type创建的)

 

1.6. __Metaclass__ property

When defining a class, you can add its properties __metaclass__. __Metaclass__ definition defines the metaclass of this class .

class Foo(object):  # python2
    __metaclass__ = something…
    ...省略...

class Foo(metaclass=something):  # python3
    __metaclass__ = something…

In this way, Python will use metaclasses to create the class Foo . Execution to the statement class Foo (object): When, class Foo has not been created in memory. Python will look __metaclass__ attribute in the definition of the class, if found, Python will use it to create a class Foo, if not found, it will use the built-in type to create this class.

For example, the following code:

class Foo(Bar):
    pass

Python made the following actions:

1. Foo has __metaclass__ this property it? If so, Python will pass __metaclass__ create a name for the class Foo (object)
2. If Python does not find __metaclass__, it will continue to look __metaclass__ property in Bar (parent), and try to do the same procedure as above.
3. If any parent class in Python can not find __metaclass__, it will be in the module hierarchy to find __metaclass__, and try to do the same thing.
4. If you still can not find __metaclass __, Python will use the built-in type to create the class object.

Q: placed in __metaclass__ in what code?

A: You can create a class of things . So what can be used to create a class of it? type, or to use any type or subclass can type stuff.

 

1.7. Metaclasses

The main purpose of metaclasses is to create a class when the class can be changed automatically .

When you create a class, metaclass metaclass used to mark who is not written metaclass default call type to create the class.

Metaclass things done summed up :

1. Creating interception class
2. Modify the class
3. Return to class after modification

 

demo1：

Suppose you decide all classes of property in your module should be in uppercase. One way is by setting __metaclass__ at the module level. In this way, all of the classes in this module will be created by this meta-class, we just need to tell yuan class all the properties had been turned into uppercase everything will be fine.

Fortunately, __ metaclass__ actually be arbitrarily invoked, it does not need to be a formal class.

Examples below, the following is also equivalent to customize a metaclass .

#-*- coding:utf-8 -*-
def upper_attr(class_name, class_parents, class_attr):

    #遍历属性字典，把不是__开头的属性名字变为大写
    new_attr = {}
    for name,value in class_attr.items():
        if not name.startswith("__"):  # 不是下划线开头时
            new_attr[name.upper()] = value

    #调用type来创建一个类
    return type(class_name, class_parents, new_attr)

class Foo(object, metaclass=upper_attr):
    bar = 'bip'

print(hasattr(Foo, 'bar'))
print(hasattr(Foo, 'BAR'))

f = Foo()
print(f.BAR)

Above, the class name Foo passed class_name, the parent object passed class_parents, the contents of the internal class to the dictionary way passed class_attr. To pass over the dictionary's key to uppercase and then to the type and returns.

 

Pass above upper_attr is a function, not a class, not quite in line with the metaclass name. Here Demo2 to use a real class as metaclass.

Demo2：

#coding=utf-8

class UpperAttrMetaClass(type):  # 继承了type类的子类，相当于是个元类
    # __new__ 是在__init__之前被调用的特殊方法
    # __new__是用来创建对象并返回之的方法
    # 而__init__只是用来将传入的参数初始化给对象
    # 你很少用到__new__，除非你希望能够控制对象的创建
    # 这里，创建的对象是类，我们希望能够自定义它，所以我们这里改写__new__
    # 如果你希望的话，你也可以在__init__中做些事情
    # 还有一些高级的用法会涉及到改写__call__特殊方法，但是我们这里不用

    def __new__(cls, class_name, class_parents, class_attr):
        # 遍历属性字典，把不是__开头的属性名字变为大写
        new_attr = {}
        for name, value in class_attr.items():
            if not name.startswith("__"):
                new_attr[name.upper()] = value

        # 方法1：通过'type'来做类对象的创建
        return type(class_name, class_parents, new_attr)

        # 方法2：复用type.__new__方法
        # 这就是基本的OOP编程，没什么魔法
        # return type.__new__(cls, class_name, class_parents, new_attr)

# python3的用法
class Foo(object, metaclass=UpperAttrMetaClass):
    bar = 'bip'

# python2的用法
# class Foo(object):
#     __metaclass__ = UpperAttrMetaClass
#     bar = 'bip'


print(hasattr(Foo, 'bar'))
# 输出: False
print(hasattr(Foo, 'BAR'))
# 输出:True

f = Foo()
print(f.BAR)
# 输出:'bip'

Question: When implementing custom class, regardless of whether the write object, will inherit the object, how to achieve?

 

. 1.8 Update: the __call__ Method metaclasses

 

Note: new meta class, init, call

In metaclass:

Objects created the class, if you want to customize it, general rewrite __new__;
if necessary, you can also do something in __init__ in;
some of the advanced uses special methods involve __call__ rewritten.

 

 

1.9. Metaclasses usage scenarios 

• If defined the function, you need to function while adding features, you can use decorator;
• If defined the class, you need to modify the function of the class, you can use metaclasses;

Python community leader Tim Peters said, "Yuan class is the depth of the magic, 99% of users should simply do not have to worry about . If you want to find out exactly whether you need to use yuan class, then you do not need it. Those practical people used metaclasses are very well aware of what they need to do, and do not need to explain why use meta-class. " 

 

 

2. metaclass implement ORM (Object Relational Mapping)

 

What 2.1. ORM is

ORM (Object Relational Mapping), namely object - relational - mapping, is the core idea of ​​the backend programming language python web framework Django, and may not need to use the SQL statement with its operational database.

A sentence is understood that: create an instance of an object, its class name created by the data as a table name, the field created by its corresponding data class attribute table, when an object instance of this operation can correspond MySQL statement.

 

Description:

1. The so-called ORM is to allow developers time to operate the database, through objects like operation xxxx.属性=yyyy as simple as it is developed ORM mind
2. Just realize ORM is more complex, Django has been achieved in a very complex operation. This section of knowledge, mainly through the completion of a similar insert ORM, understanding the truth on it

 

class User(父类省略):
    uid = ('uid', "int unsigned")
    name = ('username', "varchar(30)")
    email = ('email', "varchar(30)")
    password = ('password', "varchar(30)")
    ...省略...


u = User(uid=12345, name='Michael', email='[email protected]', password='my-pwd')
u.save()
# 对应如下sql语句
# insert into User (username,email,password,uid)
# values ('Michael','[email protected]','my-pwd',12345)

Demo of the above thought , the definition of a class, the class name that corresponds to the table name, the attribute information related to class corresponds to each field.

To operate on the table by the save method to create an object instance, and then create an instance of the object. Examples of the operation by the object, automatically converted into SQL statements manipulation without writing SQL statements .

 

2.2. ORM simple implementation of the function insert by the metaclass

 

Demo:

• Defines a metaclass ModelMetaclass (inherited from type).
• User definition of a class, there are four class attributes, methods and __ init__ save method. User metaclass class uses the specified ModelMetaclass, it is not going to create a class with the default type, but the use of specified ModelMetaclass.
• Calls ModelMetaclass of __new__ method (pass the class name User name, empty tuple passed bases, as a class attribute Dictionary pass attrs), and finally return type .__ new__.
• u = User (uid = 12345, name = 'Michael', email='[email protected] ', password =' ​​my-pwd ') is performed, User's __init__ method is called;
• The last call to save () method: self directed instance of an object ...

class ModelMetaclass(type):
    def __new__(cls, name, bases, attrs):
        mappings = dict()
        # 判断是否需要保存
        for key, values in attrs.items():  # 注：字典(Dictionary) items() 函数以列表返回可遍历的(键, 值) 元组数组
            # 判断是否是指定的StringField或者IntegerField的实例对象
            if isinstance(values, tuple):  # 判断values是否是元组
                print('Found mapping: %s ==> %s' % (key, values))
                mappings[key] = values  # 循环结束时传进来的类属性保存到了字典mappings

        # 删除这些已经在字典中存储的属性
        for key in mappings.keys():
            attrs.pop(key)

        # 将之前的uid/name/email/password以及对应的对象引用、类名字
        attrs['__mappings__'] = mappings  # 保存属性和列的映射关系
        attrs['__table__'] = name  # 假设表名和类名一致
        return type.__new__(cls, name, bases, attrs)


class User(metaclass=ModelMetaclass):
    uid = ('uid', "int unsigned")  # ！！注意：在Django中等号右边是对象，而非元祖
    name = ('username', "varchar(30)")
    email = ('email', "varchar(30)")
    password = ('password', "varchar(30)")
    
    # 当指定元类之后，以上的类属性将不在类中，而是在__mappings__属性指定的字典中存储
    # 以上User类中有（即最终User类的类属性对应为下面的样子）
    # __mappings__ = {
    #     "uid": ('uid', "int unsigned")
    #     "name": ('username', "varchar(30)")
    #     "email": ('email', "varchar(30)")
    #     "password": ('password', "varchar(30)")
    # }
    # __table__ = "User"
    
    def __init__(self, **keywargs):
        for name, value in keywargs.items():
            setattr(self, name, value)  # setattr 往self指向的对象添加一个属性。

    def save(self):
        fields = []
        args = []
        for key, values in self.__mappings__.items(): # 注：实例对象没有__mappings__，但是类对象里面有
            fields.append(values[0])  # 取元组values第一个值添加（如：第一次循环取元组('uid', "int unsigned")的uid）
            args.append(getattr(self, key, None)) # getattr取对象的key的值，添加到列表args

        sql = 'insert into %s (%s) values (%s)' % (self.__table__, ','.join(fields), ','.join([str(i) for i in args]))
        print()
        print('SQL: %s' % sql)


u = User(uid=12345, name='Michael', email='[email protected]', password='my-pwd')
# print(u.__dict__)
u.save()

Output:

Found mapping: email ==> ('email', 'varchar(30)')
Found mapping: password ==> ('password', 'varchar(30)')
Found mapping: uid ==> ('uid', 'int unsigned')
Found mapping: name ==> ('username', 'varchar(30)')

SQL: insert into User (email,username,uid,password) values ([email protected],Michael,12345,my-pwd)

Notice how the generated SQL statement is problematic: ... values ​​(test @ orm.org, Michael, 12345, my-pwd), the brackets should be some value in quotation marks.

 

Improved version :

The statement

         sql = 'insert into %s (%s) values (%s)' % (self.__table__, ','.join(fields), ','.join([str(i) for i in args]))

Change

        args_temp = list()
        for temp in args:
            # 判断入如果是数字类型
            if isinstance(temp, int):
                args_temp.append(str(temp))
            elif isinstance(temp, str):
                args_temp.append("""'%s'""" % temp)
        sql = 'insert into %s (%s) values (%s)' % (self.__table__, ','.join(fields), ','.join(args_temp))
        # 例如，经过以上["12345", """'xxxx'""", """'[email protected]'""", """'567'"""]就会变成 12345,'xxxx','[email protected]','567'
       

The last line of output becomes:

SQL: insert into User (email,username,password,uid) values ('[email protected]','Michael','my-pwd',12345)

 

Extended:

• Dictionary (Dictionary) of items () function to return may traverse the list (key, value) tuples array.
• setattr (object, name, value) function corresponding to the function  getattr () , is used to set the property value, the property is not always present. The object pointed to self add an attribute.
  • object - the object.
  • name - String object properties.
  • value - the property value.

• getattr ()  function returns the value of an object attribute. getattr (object, name [, default ])
  • object - the object.
  • name - String object properties.
  • default - default return value, if the parameter is not available, there is no corresponding attribute in the trigger AttributeError
• join () method is used to specify the elements of a sequence in a character string to generate a new connection. Returns a new string generated after the sequence of elements connected by a specified character. str.join (sequence)
  • str-- specified character
  • sequence - a sequence of elements to be connected.

The last sentence, ORM to do the operation object, the equivalent of operating data table .

 

 

-----end-----