Study notes: python operation SQLALchemy

ORM introduction:

The most famous ORM architecture in Python is SQLAlchemy, ORM: Object Relationship Mapping object model and database table mapping.

Installation: pip install SQLAlchemy

安装mysql：
yum install mysql-server mysql
service mysqld restart
sysctmctl restart mysql.service

Map the ORM model to the database:

Create an ORM base class based declarative_baseon .engine

from sqlalchemy.ext.declarative import declarative_base
engine = create_engine(DB_URI)
Base = declarative_base(engine)

Use this Baseclass as a base class to write your own ORM classes. To define __tablename__a class attribute, to specify the name of the table that this model maps to in the database.
```
class Person(Base):
    __tablename__ = 'person'
```

Create attributes to map to the fields in the table. All attributes that need to be mapped to the table should be of type Column:

class Person(Base):
    __tablename__ = 'person'
    # 2. 在这个ORM模型中创建一些属性，来跟表中的字段进行一一映射。这些属性必须是sqlalchemy给我们提供好的数据类型。
    id = Column(Integer,primary_key=True,autoincrement=True)
    name = Column(String(50))
    age = Column(Integer)

Use Base.metadata.create_all()to map the model to the database.
Once Base.metadata.create_all()the model is mapped to the database, even if the fields of the model are changed, it will not be remapped.

Use session to add, delete, modify and query data:

Build a session object: All ORM operations with the database must be sessionimplemented through a session object called, and the session object is obtained through the following code:
```
from sqlalchemy.orm import sessionmaker

engine = create_engine(DB_URI)
session = sessionmaker(engine)()
```

Add object:

Create an object, that is, create a piece of data:

p = Person(name='zhiliao',age=18,country='china')

Add this object to sessionthe session object:
```
session.add(p)
```
Commit the objects in the session (submit):
```
session.commit()
```

Add multiple pieces of data at once:

p1 = Person(name='zhiliao1',age=19,country='china')
p2 = Person(name='zhiliao2',age=20,country='china')
session.add_all([p1,p2])
session.commit()

Find objects:

# 查找某个模型对应的那个表中所有的数据：
all_person = session.query(Person).all()
# 使用filter_by来做条件查询
all_person = session.query(Person).filter_by(name='zhiliao').all()
# 使用filter来做条件查询
all_person = session.query(Person).filter(Person.name=='zhiliao').all()
# 使用get方法查找数据，get方法是根据id来查找的，只会返回一条数据或者None
person = session.query(Person).get(primary_key)
# 使用first方法获取结果集中的第一条数据
person = session.query(Person).first()

Modify the object: first search the object from the database, then modify this piece of data to the data you want, and finally do the commit operation to modify the data.
```
person = session.query(Person).first()
person.name = 'ketang'
session.commit()
```
Delete object: Find the data that needs to be deleted from the database, then use session.deletethe method to delete this data from the session, and finally do the commit operation.
```
person = session.query(Person).first()
session.delete(person)
session.commit()
```

SQLAlchemy common data types:

Integer: Integer, mapped to the database as an int type.
Float: Floating point type, mapped to the database as float type. He occupies the 32nd place.
Double: double-precision floating-point type, mapped to the database as a double type, occupying 64 bits.
String: Variable character type, mapped to the database as varchar type.
Boolean: Boolean type, which is mapped to the tinyint type in the database.
DECIMAL: Fixed-point type. It is specially designed to solve the problem of loss of precision of floating-point types. It is recommended that everyone use this data type when storing money-related fields. And when this type is used, two parameters need to be passed. The first parameter is used to mark how many numbers this field can always store, and the second parameter indicates how many digits there are after the decimal point.

Enum: enumeration type. Specifies that a field can only be a few values specified in the enumeration, and cannot be other values. In the ORM model, use Enum as an enumeration, the sample code is as follows:

class Article(Base):
    __tablename__ = 'article'
    id = Column(Integer,primary_key=True,autoincrement=True)
    tag = Column(Enum("python",'flask','django'))

In Python3, the enum enumeration module has been built in, and we can also use this module to define related fields. The sample code is as follows:

class TagEnum(enum.Enum):
    python = "python"
    flask = "flask"
    django = "django"

class Article(Base):
    __tablename__ = 'article'
    id = Column(Integer,primary_key=True,autoincrement=True)
    tag = Column(Enum(TagEnum))

article = Article(tag=TagEnum.flask)

Date: store time, only year, month and day can be stored. Mapped to the database is date type. In Python code, you can use datetime.dateto specify. The sample code is as follows:

class Article(Base):
    __tablename__ = 'article'
    id = Column(Integer,primary_key=True,autoincrement=True)
    create_time = Column(Date)

article = Article(create_time=date(2017,10,10))

DateTime: store time, can store year, month, day, hour, minute, second, millisecond, etc. Mapping to the database is also a datetime type. In Python code, you can use datetime.datetimeto specify. The sample code is as follows:

class Article(Base):
    __tablename__ = 'article'
    id = Column(Integer,primary_key=True,autoincrement=True)
    create_time = Column(DateTime)

article = Article(create_time=datetime(2011,11,11,11,11,11))

Time: Store time, which can store hours, minutes and seconds. Mapping to the database is also a time type. In Python code, you can use datetime.timefrom here to that. The sample code is as follows:

class Article(Base):
    __tablename__ = 'article'
    id = Column(Integer,primary_key=True,autoincrement=True)
    create_time = Column(Time)

article = Article(create_time=time(hour=11,minute=11,second=11))

Text: Store long strings. Generally, more than 6W characters can be stored. If it exceeds this range, you can use the LONGTEXT type. Mapped to the database is the text type.
LONGTEXT: Long text type, mapped to the database as longtext type.

Column common parameters:

primary_key: Set a field as the primary key.
autoincrement: Set this field to auto-increment.
default: Set the default value of a field. It is often used in fields such as publication time.
nullable: Specifies whether a field is null. The default value is True, which means it can be empty.
unique: Specifies whether the value of a field is unique. The default is False.
onupdate: The value or function specified by this parameter will be called when the data is updated. When inserting this data for the first time, the value of onupdate will not be used, only the value of default will be used. The most commonly used is update_time(the value to be updated every time the data is updated).
name: Specifies that an attribute in the ORM model is mapped to the field name in the table. If not specified, the name of this attribute will be used as the field name. If specified, the specified value will be used as the parameter. This parameter can also be used as a positional parameter, specified in the first parameter.
```
title = Column(String(50),name='title',nullable=False)
title = Column('my_title',String(50),nullable=False)
```

Query available parameters:

model object. Specifies to find all objects in this model.
attributes in the model. You can specify to find only a few attributes of a certain model.
aggregate function.
- func.count: counts the number of rows.
- func.avg: average value.
- func.max: Find the maximum value.
- func.min: Find the minimum value.
- func.sum: summation.
  funcIn fact, there is no aggregate function. But because he has done some magic at the bottom, as long as there are aggregation functions in mysql, they can be called through func.

filter filter conditions:

Filtering is a very important function of data extraction. Some commonly used filter conditions are explained below, and these filter conditions can only be realized through the filter method:

equals：

article = session.query(Article).filter(Article.title == "title0").first()
print(article)

not equals:
```
query.filter(User.name != 'ed')
```
like：
```
query.filter(User.name.like('%ed%'))
```

in：

query.filter(User.name.in_(['ed','wendy','jack']))
# 同时，in也可以作用于一个Query
query.filter(User.name.in_(session.query(User.name).filter(User.name.like('%ed%'))))

not in：

query.filter(~User.name.in_(['ed','wendy','jack']))

is null：

query.filter(User.name==None)
# 或者是
query.filter(User.name.is_(None))

is not null:

query.filter(User.name != None)
# 或者是
query.filter(User.name.isnot(None))

and：

from sqlalchemy import and_
query.filter(and_(User.name=='ed',User.fullname=='Ed Jones'))
# 或者是传递多个参数
query.filter(User.name=='ed',User.fullname=='Ed Jones')
# 或者是通过多次filter操作
query.filter(User.name=='ed').filter(User.fullname=='Ed Jones')

or：

from sqlalchemy import or_  query.filter(or_(User.name=='ed',User.name=='wendy'))

If you want to view the sql statement converted by the bottom of the orm, you can print it directly after the filter method without executing any methods. for example:
python articles = session.query(Article).filter(or_(Article.title=='abc',Article.content=='abc')) print(articles)

foreign key:

Creating foreign keys is very easy with SQLAlchemy. Add a field in the secondary table, and specify which field of the table is the foreign key of this field. The field of the foreign key in the secondary table must be consistent with the primary key field type of the parent table.
The sample code is as follows:

class User(Base):
    __tablename__ = 'user'
    id = Column(Integer,primary_key=True,autoincrement=True)
    username = Column(String(50),nullable=False)

class Article(Base):
    __tablename__ = 'article'
    id = Column(Integer,primary_key=True,autoincrement=True)
    title = Column(String(50),nullable=False)
    content = Column(Text,nullable=False)

    uid = Column(Integer,ForeignKey("user.id"))

Foreign key constraints have the following items:

RESTRICT: The parent table data is deleted and will prevent deletion. This is the default.
NO ACTION: In MySQL, same as RESTRICT.
1. CASCADE: Cascade deletion.
1. SET NULL: The parent table data is deleted, and the child table data will be set to NULL.

ORM relationships and one-to-many:

Mysql-level foreign keys are not enough for ORM. You must get the foreign key of a table, and then use this foreign key to search in another table, which is too troublesome. SQLAlchemy provides one relationship. This class can define attributes, which can be accessed directly through attribute access when accessing associated tables in the future. Sample code:

class User(Base):
    __tablename__ = 'user'
    id = Column(Integer,primary_key=True,autoincrement=True)
    username = Column(String(50),nullable=False)

    # articles = relationship("Article")

    def __repr__(self):
        return "<User(username:%s)>" % self.username

class Article(Base):
    __tablename__ = 'article'
    id = Column(Integer,primary_key=True,autoincrement=True)
    title = Column(String(50),nullable=False)
    content = Column(Text,nullable=False)
    uid = Column(Integer,ForeignKey("user.id"))

    author = relationship("User",backref="articles")

In addition, backrefthe property name for reverse access can be specified via . There are multiple articles. The relationship between them is a one-to-many relationship.

One-to-one relationship:

uselist=FalseIn sqlalchemy, if you want to map two models into a one-to-one relationship, you should pass this parameter in the parent model when specifying a reference . It is to tell the parent model that when this slave model is referenced in the future, it is no longer a list, but an object. The sample code is as follows:

class User(Base):
    __tablename__ = 'user'
    id = Column(Integer,primary_key=True,autoincrement=True)
    username = Column(String(50),nullable=False)

    extend = relationship("UserExtend",uselist=False)

    def __repr__(self):
        return "<User(username:%s)>" % self.username

class UserExtend(Base):
    __tablename__ = 'user_extend'
    id = Column(Integer, primary_key=True, autoincrement=True)
    school = Column(String(50))
    uid = Column(Integer,ForeignKey("user.id"))

    user = relationship("User",backref="extend")

Of course, you can also sqlalchemy.orm.backrefsimplify the code with:

class User(Base):
    __tablename__ = 'user'
    id = Column(Integer,primary_key=True,autoincrement=True)
    username = Column(String(50),nullable=False)

    # extend = relationship("UserExtend",uselist=False)

    def __repr__(self):
        return "<User(username:%s)>" % self.username

class UserExtend(Base):
    __tablename__ = 'user_extend'
    id = Column(Integer, primary_key=True, autoincrement=True)
    school = Column(String(50))
    uid = Column(Integer,ForeignKey("user.id"))

    user = relationship("User",backref=backref("extend",uselist=False))

Many-to-many relationship:

The many-to-many relationship needs to bind the relationship between them through an intermediate table.
First define two models that need to do many-to-many
Use Table to define an intermediate table. Generally, the intermediate table is just a foreign key field containing two models, and let them be used as a "composite primary key".
Choose a model randomly among the two models that need to do many-to-many, and define a relationship attribute to bind the relationship between the three. When using the relationship, you need to pass in a secondary=intermediate table.

Delete data at the ORM level:

Deleting data at the ORM level will ignore the foreign key constraints at the mysql level. The corresponding data will be deleted directly, and then the foreign key in the slave table will be set to NULL. If you want to avoid this behavior, you should move the foreign key from the table nullable=False.
In SQLAlchemy, as long as a piece of data is added to the session, all the data associated with it can be stored in the database together. How are these set up? In fact, when passing through the relationship, there is a keyword parameter cascade to set these properties:

save-update: The default option. When adding a piece of data, all other data associated with it will be added to the database. This behavior is affected by the save-update attribute.
delete: Indicates whether to delete the data associated with the relationship when deleting the data in a certain model.
delete-orphan: Indicates that when the associated object in the parent table is released for an ORM object, it will be deleted. Of course, if the data in the parent table is deleted, it will also be deleted. This option can only be used on one-to-many, not on many-to-many and many-to-one. And also need to add a single_parent=True parameter in the relationship in the submodel.
merge: The default option. When using session.merge to merge an object, the objects associated with the relationship will also be merged.
expunge: During the removal operation, the associated objects will also be removed. This operation is only removed from the session, and will not actually be deleted from the database.
all: It is an abbreviation for save-update, merge, refresh-expire, expunge, delete.

Sort by:

order_by: You can specify to sort according to a certain field in this table. If you add a - in front, it means sorting in descending order.
Specify the default sorting when defining the model: Sometimes, you don't want to specify the sorting method every time you query, you can specify the sorting method when defining the model. There are two ways:
- The order_by parameter of the relationship: When specifying the relationship, pass the order_by parameter to specify the sorted field.
- In the model definition, add the following code:
mapper_args = { "order_by": title } allows articles to be sorted by title.
Forward sorting and reverse sorting: The default is to use forward sorting. If you need to sort in reverse order, you can use desc()the method of this field, or use the string name of this field when sorting, and then add a minus sign in front.

limit, offset and slice operations:

limit: You can limit only a few pieces of data to be queried each time.
offset: You can limit how many previous items are filtered out when searching for data.
Slicing: You can use slicing operations on the Query object to get the desired data. You can use slice(start,stop)methods to do slice operations. You can also use [start:stop]the method to perform slice operations. Generally, in actual development, the form of square brackets is used more often. I hope everyone will master it. The sample code is as follows:

articles = session.query(Article).order_by(Article.id.desc())[0:10]

Lazy loading:

In one-to-many, or many-to-many situations, if you want to get more data, you can often get all of it through one attribute. For example, if there is an author who wants or wants all the articles of this author, then you can get all of them through user.articles. But sometimes we don't want to get all the data, for example, we just want to get the articles published by the author today, then we can pass a lazy='dynamic' to the relationship at this time, and what we get through user.articles in the future is not a list, but is an AppenderQuery object. In this way, another layer of filtering and sorting operations can be performed on this object.
Pass lazy='dynamic', the part of the data that is obtained more is an AppenderQueryobject. This kind of object can add new data, or it can be the Querysame, and can perform another layer of filtering.
In a word: If you want to filter the data on the side with more data when you get the data, then you can consider using it at this time lazy='dynamic'.
Available options for lazy:

select: This is the default option. Or take user.articlesthe example. If you don't have access to user.articlesthis attribute, then sqlalchemy won't look up articles from the database. Once you access this attribute, sqlalchemy will immediately search all articles from the database, and assemble the found data into a list and return it. This is also lazy loading.
dynamic: This is what we just talked about. That is, user.articleswhat is returned when accessing is not a list, but AppenderQueryan object.

group_by：

Group by a field. For example, if you want to group by gender to count how many people are in each group, you can use the following code to complete it:

session.query(User.gender,func.count(User.id)).group_by(User.gender).all()

having：

having is to further filter the search results. For example, if you only want to see the number of minors, you can first group by age to count the number of people, and then filter the groups by having. The sample code is as follows:

result = session.query(User.age,func.count(User.id)).group_by(User.age).having(User.age >= 18).all()

join：

join is divided into left join (left outer connection) and right join (right outer connection) and inner connection (equivalent connection).
Reference web page: http://www.jb51.net/article/15386.htm
In sqlalchemy, use join to complete the inner connection. When writing join, if you do not write the join condition, then the default will be to use the foreign key as the conditional connection.
What value is found by the query will not depend on what is behind the join, but on what parameters are passed in the query method. Just like the one behind select in native sql.
For example, to implement a function now, to find all users and sort them according to the number of published articles. The sample code is as follows:

result = session.query(User,func.count(Article.id)).join(Article).group_by(User.id).order_by(func.count(Article.id).desc()).all()

subquery：

Subqueries can turn multiple queries into one query, and the performance is relatively more efficient as long as the database is searched once. Some complex queries can be realized without writing multiple sql statements. Then in sqlalchemy, to implement a subquery, the following steps should be used:

Write the subquery code in the traditional way, and then queryexecute subquerythe method behind the object to turn this query into a subquery.
In the subquery, pass the field that needs to be used in the future labelto get an alias.
In the parent query, if you want to use the fields of the subquery, you can cget it through the attributes on the return value of the subquery.
The overall sample code is as follows:

stmt = session.query(User.city.label("city"),User.age.label("age")).filter(User.username=='李A').subquery()
result = session.query(User).filter(User.city==stmt.c.city,User.age==stmt.c.age).all()