"Smooth Python Chapter 1 Python Data Model"

Chapter 1 python data model

Summary: This chapter mainly talks about some special methods (methods written with double underscores before and after, such as __len__, __getitem__), why there are special methods, and the applications of special methods.

Python is an object-oriented language, so why is it written like len(x) instead of x.len()?

When the Python interpreter encounters special syntax like len(), it will use special methods to activate some basic object operations. The names of these special methods begin with two underscores and end with two underscores. Such as __len__, __getitem__. For example, behind obj[key] is the __getitem__ method. In order to obtain the value of obj[key], the interpreter actually calls obj.__getitem__(key).

By using these methods appropriately, you can enable your objects to implement and support the following language structures and interact with them:

• Iterate
• Collection class
• Property access
• Operator overloading
• Function and method calls
• Object creation and destruction
• String representation and formatting
• Managing context (with module)

Magic method is the nickname of a special method, also called dunder method.

A stack of playing cards in python style

Use a very simple example to show how to implement the two special methods __getitem__ and __len__.

import collections
Card=collections.namedtuple('Card',['rank','suit'])
#使用collections.namedtuple创建了一个元组类型的子类，子类类名为Card，子类中有两个key，key值分别为'rank'、'suit'
class FrenchDeck:
    ranks=[str(n) for n in range(2,11)]+list('JQKA')#ranks=['2', '3', '4', '5', '6', '7', '8', '9', '10', 'J', 'Q', 'K', 'A']
    suits='spades diamonds clubs hearts'.split()#suits=['spades', 'diamonds', 'clubs', 'hearts']
    def __init__(self):
        self._cards=[Card(rank,suit) for rank in self.ranks for suit in self.suits]

    def __len__(self):
        return len(self._cards)

    def __getitem__(self, item):
        return self._cards[item]

Using namedtuple, we can easily get a playing card object:

bear_card=Card(rank='7',suit='spades')
print(bear_card)

/Users/zy/PycharmProjects/learnpython/venv/bin/python /Users/zy/PycharmProjects/learnpython/collect.py
Card(rank='7', suit='spades')

Process finished with exit code 0

Then, let's focus on the FrenchDeck class. After instantiating it, we can use the len() function to see how many cards there are in the stack.

deck=FrenchDeck()
print(len(deck))

/Users/zy/PycharmProjects/learnpython/venv/bin/python /Users/zy/PycharmProjects/learnpython/collect.py
52

Process finished with exit code 0

You can also use the index subscript to access a specific card:

print(deck[5])

/Users/zy/PycharmProjects/learnpython/venv/bin/python /Users/zy/PycharmProjects/learnpython/collect.py
Card(rank='3', suit='diamonds')

Process finished with exit code 0

If you comment out the __len__ method in the FrenchDeck class, you cannot use len(deck). Similarly, if you comment out the __getitem__ method in the FrenchDeck class, you cannot use deck[key] to access specific elements.

And we don't need to write another method to randomly draw a card, we only need to call python's built-in random.choice.

from random import choice
deck=FrenchDeck()
print(choice(deck))

/Users/zy/PycharmProjects/learnpython/venv/bin/python /Users/zy/PycharmProjects/learnpython/collect.py
Card(rank='A', suit='spades')

Process finished with exit code 0

From the above experiments, we discovered two benefits of leveraging python data models by implementing special methods:

• After the user instantiates this class, he does not need to bother to remember the format name of the standard operation, such as getting the total number of elements, whether it is the size method or the length method.
• You can more conveniently use the python standard library without having to rewrite the code yourself.

And because the __getitem__ method hands over the subscript access [] operation to the self._cards list, deck automatically supports slicing operations.

print(deck[:3])
print(deck[12::13])

/Users/zy/PycharmProjects/learnpython/venv/bin/python /Users/zy/PycharmProjects/learnpython/collect.py
[Card(rank='2', suit='spades'), Card(rank='2', suit='diamonds'), Card(rank='2', suit='clubs')]
[Card(rank='5', suit='spades'), Card(rank='8', suit='diamonds'), Card(rank='J', suit='clubs'), Card(rank='A', suit='hearts')]

Process finished with exit code 0

In addition, just by implementing the __getitem__ method, the deck becomes iterable:

for card in deck:
    if card.rank=='J':
        print(card)
    else:
        pass

The above code finds all 'J' cards

/Users/zy/PycharmProjects/learnpython/venv/bin/python /Users/zy/PycharmProjects/learnpython/collect.py
Card(rank='J', suit='spades')
Card(rank='J', suit='diamonds')
Card(rank='J', suit='clubs')
Card(rank='J', suit='hearts')

Process finished with exit code 0

Reverse iteration also works:

for card in reversed(deck):
    if card.rank=='J':
        print(card)
    else:
        pass

/Users/zy/PycharmProjects/learnpython/venv/bin/python /Users/zy/PycharmProjects/learnpython/collect.py
Card(rank='J', suit='hearts')
Card(rank='J', suit='clubs')
Card(rank='J', suit='diamonds')
Card(rank='J', suit='spades')

Process finished with exit code 0

Iteration is usually implicit. For example, if a collection type does not implement the __contains__ method, then the in operator will do an iterative search in order because it is iterable:

print(Card(rank='J', suit='hearts') in deck)
print(Card(rank='O', suit='hearts') in deck)

/Users/zy/PycharmProjects/learnpython/venv/bin/python /Users/zy/PycharmProjects/learnpython/collect.py
True
False

Process finished with exit code 0

Sort the cards: 2 is the smallest, Ace is the largest, Spades is the largest, Hearts is the second, Diamonds are the third, and Clubs are the smallest.

suit_values=dict(spades=3,hearts=2,diamonds=1,clubs=0)#spades:黑桃 hearts:红桃 diamonds:方块 clubs:梅花
# spades A>hearts A>diamonds A>clubs A>
# spades K>hearts K>diamonds K>clubs K>
# spades Q>hearts Q>diamonds Q>clubs Q>
# spades J>hearts J>diamonds J>clubs J>
# spades 10>hearts 10>diamonds 10>clubs 10>
# spades 9>hearts 9>diamonds 9>clubs 9>
# spades 8>hearts 8>diamonds 8>clubs 8>
# spades 7>hearts 7>diamonds 7>clubs 7>
# spades 6>hearts 6>diamonds 6>clubs 6>
# spades 5>hearts 5>diamonds 5>clubs 5>
# spades 4>hearts 4>diamonds 4>clubs 4>
# spades 3>hearts 3>diamonds 3>clubs 3>
#  1*4+3    1*4+2     1*4+1     1*4+0
# spades 2>hearts 2>diamonds 2>clubs 2
#  0*4+3    0*4+2     0*4+1     0*4+0
def spades_high(card):
    rank_value=FrenchDeck.ranks.index(card.rank)
    return rank_value*len(suit_values)+suit_values[card.suit]#排序规则的公式

for card in sorted(deck,key=spades_high):#先将card传入spades_high()函数进行计算，按最终返回值排序
    print(card,spades_high(card))

With the spades_high function, this pile of cards can be sorted in ascending order.

/Users/zy/PycharmProjects/learnpython/venv/bin/python /Users/zy/PycharmProjects/learnpython/collect.py
Card(rank='2', suit='clubs') 0
Card(rank='2', suit='diamonds') 1
Card(rank='2', suit='hearts') 2
Card(rank='2', suit='spades') 3
Card(rank='3', suit='clubs') 4
Card(rank='3', suit='diamonds') 5
Card(rank='3', suit='hearts') 6
Card(rank='3', suit='spades') 7
Card(rank='4', suit='clubs') 8
Card(rank='4', suit='diamonds') 9
Card(rank='4', suit='hearts') 10
Card(rank='4', suit='spades') 11
Card(rank='5', suit='clubs') 12
Card(rank='5', suit='diamonds') 13
Card(rank='5', suit='hearts') 14
Card(rank='5', suit='spades') 15
Card(rank='6', suit='clubs') 16
Card(rank='6', suit='diamonds') 17
Card(rank='6', suit='hearts') 18
Card(rank='6', suit='spades') 19
Card(rank='7', suit='clubs') 20
Card(rank='7', suit='diamonds') 21
Card(rank='7', suit='hearts') 22
Card(rank='7', suit='spades') 23
Card(rank='8', suit='clubs') 24
Card(rank='8', suit='diamonds') 25
Card(rank='8', suit='hearts') 26
Card(rank='8', suit='spades') 27
Card(rank='9', suit='clubs') 28
Card(rank='9', suit='diamonds') 29
Card(rank='9', suit='hearts') 30
Card(rank='9', suit='spades') 31
Card(rank='10', suit='clubs') 32
Card(rank='10', suit='diamonds') 33
Card(rank='10', suit='hearts') 34
Card(rank='10', suit='spades') 35
Card(rank='J', suit='clubs') 36
Card(rank='J', suit='diamonds') 37
Card(rank='J', suit='hearts') 38
Card(rank='J', suit='spades') 39
Card(rank='Q', suit='clubs') 40
Card(rank='Q', suit='diamonds') 41
Card(rank='Q', suit='hearts') 42
Card(rank='Q', suit='spades') 43
Card(rank='K', suit='clubs') 44
Card(rank='K', suit='diamonds') 45
Card(rank='K', suit='hearts') 46
Card(rank='K', suit='spades') 47
Card(rank='A', suit='clubs') 48
Card(rank='A', suit='diamonds') 49
Card(rank='A', suit='hearts') 50
Card(rank='A', suit='spades') 51

Process finished with exit code 0

By implementing the two special methods __len__ and __getitem__, the FrenchDeck class is just like Python's own sequence data type, with core Python language features such as iteration and slicing.

Exercise: Create a Book class

Book=collections.namedtuple('Book',['name','editor','id'])
# a_book=Book(name='救赎',editor='lily',id='20211101')
# print(a_book)
class MyBook:
    names='abandon baby cloud dead'.split()
    editors='lili susan yang alice'.split()
    ids=[x for x in range(2,6)]
    def __init__(self):
        self._book=[Book(name,editor,id) for name in self.names for editor in self.editors for id in self.ids] #这种写法对names,editors,ids进行了全排列组合

    def __len__(self):
        return len(self._book)

    def __getitem__(self, item):
        return self._book[item]

b=MyBook()
print(len(b))
print(b[2])

The execution results are as follows:

/Users/zy/PycharmProjects/learnpython/venv/bin/python /Users/zy/PycharmProjects/learnpython/collect.py
64
Book(name='abandon', editor='lili', id=4)

Process finished with exit code 0

How to use special methods

Special methods exist to be called by the Python interpreter, you don't need to call them yourself.

deck.__len__()
deck.__getitem__(3)

In other words, although the above two writing methods are executable, they are not compliant. The following writing should be used

len(deck)
deck[1]

If it is a custom class, such as FrenchDeck, when executing len(deck), python will automatically call the __len__ method implemented by yourself in the FrenchDeck class.

If it is a built-in type of Python, such as str, list, and len(str) is executed, CPython will take a shortcut and directly return the ob_size attribute in PyVarObject. Reading this value directly is much faster than calling a method.

The calling of special methods is implicit. For example, the statement for i in x: is actually behind iter(x), and behind this function is x.__iter__() (provided that the __iter__ method is implemented in x ).

Generally your code will not need to use special methods directly, other than calling the superclass's constructor in your own subclass's __init__. It is generally best to use special methods through built-in functions (len, str, iter, etc.).

Analog numeric type

Using special methods, custom objects can be operated with the "+" sign.

Example: Custom class to implement addition and subtraction of two-dimensional vectors

from math import hypot
class Vector:
    def __init__(self,x,y):
        self.x=x
        self.y=y

    def __repr__(self):
        '''
        自定义输出实例化对象时的信息
        :return:
        '''
        return 'Vector(%r,%r)' % (self.x,self.y)

    def __abs__(self):
        '''
        自定义向量求模
        :return:
        '''
        return hypot(self.x,self.y)

    def __add__(self, other):
        '''
        自定义向量相加方法
        :param other:
        :return:
        '''
        x=self.x+other.x
        y=self.y+other.y
        return Vector(x,y)

    def __mul__(self, scalar):
        '''
        自定义向量乘常数
        :param scalar:
        :return:
        '''
        return Vector(self.x*scalar,self.y*scalar)

    def __bool__(self):
        '''
        返回向量是否为0
        :return:
        '''
        return bool(abs(self))

vector1=Vector(3,4)
print("调用__repr__方法：%r" % vector1)
vector2=Vector(1,2)
print("调用__add__方法：%r" % (vector1+vector2))
print("调用__mul__方法：%r" % (vector1*3))
vector3=Vector(0,0)
print("调用__abs__方法：%r"% abs(vector1))
print("调用__bool__方法：%r" % bool(vector3))

Results of the:

/Users/zy/PycharmProjects/learnpython/venv/bin/python /Users/zy/PycharmProjects/learnpython/collect.py
调用__repr__方法：Vector(3,4)
调用__add__方法：Vector(4,6)
调用__mul__方法：Vector(9,12)
调用__abs__方法：5.0
调用__bool__方法：False

Process finished with exit code 0

Additional knowledge points:

python formatter

Format character	type code
%s	String (displayed using str())
%r	String (displayed using repr())
%c	single character
%b	binary integer
%d	decimal integer
%i	decimal integer
%o	Octal integer
%s	hexadecimal integer
%e	Exponent (base written as e)
%E	Exponent (base written as E)
%f	floating point number
%F	Floating point numbers (same as above)
%g	exponent (e) or floating point number (according to display length)
%G	Exponent (E) or floating point number (according to display length)

string representation

Python's built-in function repr() can express an object in a string representation. This is the "string representation". What is called behind the repr() function is __repr__. If __repr__ is not implemented, we print an object, which may be displayed as **< main .testrepr object at 0x108e6e9d0>**.

In the formatter, use %r to replace the result returned by the repr() function. The new format string syntax of str.format() also uses repr.

In the Vector example, we use **"Vector(%r,%r)" %(self.x,self.y)** to get the string representation of the object instead of using %s to get the object string value.

The difference between __repr__ and __str__ is that the latter will be called when the str() function and print() are used, and the returned string is more friendly to end users.

If you only want to implement one of the two special methods, it is recommended to implement __repr__. When str() is called, if the object does not implement __str__, the interpreter will use __repr__ instead. See example below:

class testrepr:
    def __init__(self,x):
        self.x=x

    def __repr__(self):
        return 'y'

    def __str__(self):
        return 'z'

tr=testrepr(1)
print(str(tr))

Executing the above code, you can see that the return value of str(tr) is z. If you comment out the __str__ method and execute it again, the return value of str(tr) is y.

A basic requirement of a python object is that it must have a reasonable string representation. We can meet this requirement through __repr__ and __str__. The former is convenient for us to debug and record logs, and the latter is convenient for end users to see.

arithmetic operators

In the example of Vector operation, we redefine the operation rules of vector addition and vector multiplication through the special methods __mul__ and __add__.

Custom boolean value

Any object can be used in contexts that require Boolean values, such as if, while, and, or, not. By default, instances of our custom classes are always considered True. Such as the following example:

class testbool:
    def __init__(self,x):
        self.x=x

tb=testbool(1)
print(bool(tb))

The execution result is True.

Behind bool(x) is the result of calling x.__bool__(). If the __bool__ method does not exist, the python interpreter will try to call the __len__() method. If 0 is returned, bool will return False, otherwise it will return True. . Examples are as follows:

x=""
print(bool(x))

The execution result is False.

We can customize the __bool__ method, like in the Vector vector example, to define judgment rules and return values.

List of special methods

Special methods independent of operators:

category	method name
String/byte sequence representation	__repr__、__str__、__format__、__bytes__
Numeric conversion	__abs__、__bool__、__complex__、__int__、__float__、__hash__、__index__
ensemble simulation	__len__、__getitem__、__setitem__、__delitem__、__contains__
Iterate over enumeration	__iter__、__reversed__、__next__
callable mock	__call__
context management	__enter__、__exit__
Instance creation and destruction	__new__、__init__、__del__
Property management	__getattr__、__getattribute__、__setattr__、__delattr__、__dir__
property descriptor	__get__、__set__、__delete__
Class-related services	__prepare__、__instancecheck__、__subclasscheck__

Special methods related to operators:

category	Method name and corresponding operator
unary operator	__neg__ -、__pos__ +、__abs__ abs()
Numerous comparison operators	__lt__ <、__le__ <=、__eq__ ==、__ne__ !=、__gt__ >、__ge__ >=
arithmetic operators	__add__ +、__sub__ -、__mul__ *、__truediv__ /、__floordiv__ //、__mod__ %、__divmod__ divmode()、__pow__ **或pow()、__round__ round()
reverse arithmetic operator	__radd__、__rsub__、__rmul__、__rtruediv__、__rfloordiv__、__rmod__、__rdivmod__、__rpow__
incremental assignment arithmetic operator	__iadd__、__isub__、__imul__、__itruediv__、__ifloordiv__、__imod__、__ipow__
Bit operators	__invert__ ~、__lshift__ <<、__rshift__ >>、__and__ &、__or__ |、__xor__ ^
reverse bitwise operator	__rlshift__、__rrshift__、__rand__、__rxor__、__ror__
Incremental assignment bitwise operator	__ilshift__、__irshift__、__iand__、__ixor__、__ior__

Reverse operator: The reverse operator is called when the positions of two numbers are swapped. (b*a instead of a*b)

Incremental assignment operator: The incremental assignment operator is a shortcut that turns the infix operator into an assignment operation. (For example, a=a+b becomes a+=b)

Why len is not a normal method

"Use is better than purity." - "The Zen of Python"

如果x是一个内置类型的实例，len(x)执行不需要调用任何方法（CPython会直接从一个C结构体里读取对象的长度），故而len(x)的速度会非常快而高效。len之所以不是一个普通方法，是因为它让python自带的数据结构可以走后门，abs也是同理。