SQLAlchemy-- basic CRUD

Brief introduction

SQLAlchemy ORM framework is a Python-based implementation. The framework based on DB API, using the relationship object mapping database operation, it is short: converting SQL into classes and objects, and then use the SQL API to perform the data acquisition and the execution result.

installation

pip3 install sqlalchemy

component

• Engine: The engine frame
• Connection Pooling: database connection pool
• Dialect: Select the type of DB API connection to the database
• Schema / Types: architecture and type
• SQL Exprression Language: SQL expression language

SQLAlchemy itself can not operate the database, since it must pymsql other third-party plug-ins, Dialect for API and data exchange, according to different profiles database API calls, enabling operation of the database, such as:

"""
MySQL-Python
    mysql+mysqldb://<user>:<password>@<host>[:<port>]/<dbname>
    
pymysql
    mysql+pymysql://<username>:<password>@<host>/<dbname>[?<options>]
    
MySQL-Connector
    mysql+mysqlconnector://<user>:<password>@<host>[:<port>]/<dbname>
    
cx_Oracle
    oracle+cx_oracle://user:pass@host:port/dbname[?key=value&key=value...]
    
更多：http://docs.sqlalchemy.org/en/latest/dialects/index.html
"""

How to generate models django reverse

python manage.py inspectdb > app/models.py

Simple to use

SQLAlchemy can only create table, drop table, not making any changes in the original table, if you want to make changes, can be modified in the database, and then on again to modify the corresponding class

Execute native sql (not used)

import time
import threading
import sqlalchemy
from sqlalchemy import create_engine
from sqlalchemy.engine.base import Engine

engine = create_engine(
    "mysql+pymysql://root:[email protected]:3306/test?charset=utf8",
    max_overflow=0,  # 超过连接池大小外最多创建的连接
    pool_size=5,  # 连接池大小
    pool_timeout=30,  # 池中没有线程最多等待的时间，否则报错
    pool_recycle=-1  # 多久之后对线程池中的线程进行一次连接的回收（重置）
)
def task(arg):
    conn = engine.raw_connection()
    cursor = conn.cursor()
    cursor.execute(
        "select * from app01_book"
    )
    result = cursor.fetchall()
    print(result)
    cursor.close()
    conn.close()

for i in range(20):
    t = threading.Thread(target=task, args=(i,))
    t.start()

orm use (Key)

connection

from sqlalchemy import create_engine

create_engine()Engine returns an instance, and it represents the core of the interface details of processing by the syntax database, in this case, the database will be explained grammar class, said method Python

engine = create_engine('mysql+pymysql://root:123456@localhost:3306/test',echo=True)

Connection echo parameter to True, will display each sql statement executed

engine = create_engine('mysql+pymysql://root:123456@localhost:3306/test')

Statement image

• By using Declarative approach, we can create a mapping class contains a description of the actual database table to be mapped guidelines.
• Declarative method using a mapping defined class based on a base class, the base class is a directory and to maintain class data table relationships - what we call the Declarative base class. In a common inlet module, usually only have a base application instance. We create a base class by declarative_base () function:

from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()

With this Base, we can be based on any number of the base to define the mapping class:

class User(Base):
    __tablename__ = 'users'  # 数据库表名称
    id = Column(Integer, primary_key=True)  # id 主键
    name = Column(String(32), index=True, nullable=False)  # name列，索引，不可为空
    # email = Column(String(32), unique=True)
    #datetime.datetime.now不能加括号，加了括号，以后永远是当前时间
    # ctime = Column(DateTime, default=datetime.datetime.now)
    # extra = Column(Text, nullable=True)

    __table_args__ = (
        # UniqueConstraint('id', 'name', name='uix_id_name'), #联合唯一
        # Index('ix_id_name', 'name', 'email'), #索引
    )

Note: with a class constructor requires a Declarative least tablename attribute, a primary row.

Make Table

SQLAlchemy can not be django's similar with makemigerationsand migerateautomatically generate the table, we need to generate a table of our own

def init_db():
    """
    根据类创建数据库表
    :return:
    """
    engine = create_engine(
        "mysql+pymysql://root:[email protected]:3306/aaa?charset=utf8",
        max_overflow=0,  # 超过连接池大小外最多创建的连接
        pool_size=5,  # 连接池大小
        pool_timeout=30,  # 池中没有线程最多等待的时间，否则报错
        pool_recycle=-1  # 多久之后对线程池中的线程进行一次连接的回收（重置）
    )

    Base.metadata.create_all(engine)

Change the table fields

SQLAlchemy is not supported after the table is created, then the field inside the table to modify, add, delete, modify the fields so if you want a table, there are two methods:

1. Modify the fields manually modify the database, then the corresponding class
2. After deleting the table, and then modify the field, and then create a table

Delete table

def drop_db():
    """
    根据类删除数据库表
    :return:
    """
    engine = create_engine(
        "mysql+pymysql://root:[email protected]:3306/aaa?charset=utf8",
        max_overflow=0,  # 超过连接池大小外最多创建的连接
        pool_size=5,  # 连接池大小
        pool_timeout=30,  # 池中没有线程最多等待的时间，否则报错
        pool_recycle=-1  # 多久之后对线程池中的线程进行一次连接的回收（重置）
    )

    Base.metadata.drop_all(engine)

The complete code

import datetime
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String, Text, ForeignKey, DateTime, UniqueConstraint, Index

Base = declarative_base()

class Users(Base):
    __tablename__ = 'users'  # 数据库表名称
    id = Column(Integer, primary_key=True)  # id 主键
    name = Column(String(32), index=True, nullable=False)  # name列，索引，不可为空
    age = Column(Integer, default=0)
    # email = Column(String(32), unique=True)
    #datetime.datetime.now不能加括号，加了括号，以后永远是当前时间
    # ctime = Column(DateTime, default=datetime.datetime.now)
    # extra = Column(Text, nullable=True)

    __table_args__ = (
        # UniqueConstraint('id', 'name', name='uix_id_name'), #联合唯一
        # Index('ix_id_name', 'name', 'email'), #索引
    )

def init_db():
    """
    根据类创建数据库表
    :return:
    """
    engine = create_engine(
        "mysql+pymysql://root:[email protected]:3306/aaa?charset=utf8",
        max_overflow=0,  # 超过连接池大小外最多创建的连接
        pool_size=5,  # 连接池大小
        pool_timeout=30,  # 池中没有线程最多等待的时间，否则报错
        pool_recycle=-1  # 多久之后对线程池中的线程进行一次连接的回收（重置）
    )

    Base.metadata.create_all(engine)

def drop_db():
    """
    根据类删除数据库表
    :return:
    """
    engine = create_engine(
        "mysql+pymysql://root:[email protected]:3306/aaa?charset=utf8",
        max_overflow=0,  # 超过连接池大小外最多创建的连接
        pool_size=5,  # 连接池大小
        pool_timeout=30,  # 池中没有线程最多等待的时间，否则报错
        pool_recycle=-1  # 多久之后对线程池中的线程进行一次连接的回收（重置）
    )

    Base.metadata.drop_all(engine)

if __name__ == '__main__':
    # drop_db()
    init_db()

Common data types

type of data	Explanation
Integer	Shaping, mapping to the database is of type int.
Float	Floating-point type, the database is mapped to a float. He occupied 32.
Double	Double precision floating point type, mapped into the database type double, occupy 64 bits.
String	Variable character type, the database is mapped to a varchar type.
Boolean	Boolean type, mapped to the database is tinyint type.
DECIMAL	Fixed-point types. Floating-point type is designed to solve the problem of loss of precision. When storing money in related fields suggest that you use this data type. And when the transmission of this type requires the use of two parameters, the first parameter is used to mark the total number digits field can store energy, the second parameter indicates the number of decimal places.
Enum	Enumerated type. Specify a field only a few enum values ​​specified, not value for others. In the ORM model, using Enum to enumerate as
Date	Storage time, can only store date. Mapping to the database is date type. In Python code, you may be used datetime.dateto specify
DateTime	Storage time, minutes and seconds milliseconds isochronous can store date. The database is mapped to datetime type. In Python code may be used datetime.datetimeto specify.
Time	Storage time, when every minute can be stored. Mapped to a database is time types. In Python code, you can use datetime.timeto that point.
Text	Store long strings. 6W general can store multiple characters. If you exceed this range, you can use LONGTEXT type. Mapping to the database type is text.
LONGTEXT	Long text type, are mapped to the database longtext type.

Column commonly used parameters

parameter	Detail
default	Defaults
nullable	Is empty
primary_key	Primary key
unique	Is unique
autoincrement	Whether increment
onupdate	When the update execution
name	After the database mapping properties
index	Whether indexing

Common Operation (CURD)

Create an instance of the class map

user1 = User(name='hades', age=18)
user2 = User(name='bonnie', age=16)

Create Session Session

Ready for the session and database, ORM establish a connection to the database via Session

When the application first loads, we define a Session class (statement Create_engine () at the same time), the Session class for a new Session object provides factory service.

from sqlalchemy.orm import sessionmaker
Session = sessionmaker(bind=engine)

The custom Session class will be bound to create a database of the Session object. If you need to establish a connection to the database, simply instantiate a session object

session =Session()

Although the above Session and has been associated with the database engine Engine, but has not opened any connection. When it is first used, there will be a connection from Engine maintenance of the pool to retrieve whether there is a connection, if there is to know we will stay connected submit all changes and / or close the session object.

Increase the add () / add_all ()

# 增加一个
session.add(user1)
session.add(user2)

# 增加多个,可以增加不同的映射实例
# session.add_all([user1, user2, Hosts(ip='127.0.0.1')])

Submit commit ()

At this point, we can say that this newly added object instance is still waiting ; user1 object is now does not mean a row of data in the database. Until the use of flush process, Session will allow SQL to stay connected. If this query the data, all the information will be waiting for the first time to refresh, the query results will be issued immediately.

1. By commit () can submit all remaining changes to the database.
2. Note: The author, a query will perform all waiting for information.
3. All add, modify, delete all need to commit to submit

 session.commit()

Rollback rollback ()

session.rollback()

Query (Key)

Creating a query object via Session of the query () method. The number of parameters of this function is variable, the parameter may describe any class or set of classes

The following is an example of an output of the User class iteration:

The first query

session.query(Users).filter_by(name='lqz').first()

Sequence

session.query(User).order_by(User.id).all()

# desc(): 降序，一定要加（）
session.query(User).order_by(User.id.desc()).all()

# asc()：升序
session.query(User).order_by(Users.name.desc(),User.id.asc()).all()

Query also supports ORM is described as a parameter. At any time, the entity is an entity or a plurality of classes based on expression of a column can be used as query () function parameters, return type is a tuple:

session.query(User.name,User.fullname)

session.query(User,User.name).all()

Surnamed

• Alias the field : label()the equivalent of row.name

session.query(User.name.label("name_label")).all()

• Table aliases :aliased()

from sqlalchemy.orm import aliased
user_alias = aliased(User,name='user_alias')

session.query(user_alias,user_alias.name).all()

Query basic operations including LIMIT and OFFSET, using an array of slice and python ORDERBY combination can make operation easy.

Limit for paging, zone

Only the second and third query data

session.query(User).order_by(User.id)[1:3]

filter

Use the keyword variable filter query results, filter and filter_by use

• pass filter is an expression, filter_by pass a parameter

session.query(User).filter(User.name=='hades').all()

session.query(User).filter_by(name='bonnie').all()

filter and filter_by difference:

• filter: you could use> <like, but must be listed: table column, filter number is equal ==
• filter: the combination does not support queries
• filter_by: you can write the column directly, do not support the <> filter_by tantamount ==
• filter_by can support a combination of query

Filtration method

• equals

  session.query(User).filter(User.name == 'ed')

• not equals

  session.query(User).filter(User.name != 'ed')

• like

  session.query(User).filter(User.name.like('%ed%'))

• in

  query.filter(User.name.in_(['ed','wendy','jack']))
  
  # 子查询
  session.query(User).filter(User.name.in_(session.query(User.name).filter(User.name.like('%ed%'))

• not in

  query.filter(~User.name.in_('ed','wendy','jack'))

• is null

  session.query(User).filter(User.name == None) 

• is not null

  session.query(User).filter(User.name != None)

• and

  session.query(Users).filter(and_(User.name =='ed',User.fullname =='Ed Jones')) # and
  
  session.query(Users).filter(User.name == 'ed',User.fullname =='Ed Jones') # and
  
  session.query(Users).filter(User.name == 'ed').filter(User.fullname == 'Ed Jones')# and

• or

  query.filter(or_(User.name='ed', User.name='wendy'))

• Find placeholders

  #:value 和:name 相当于占位符，用params传参数
  session.query(Users).filter(text("id<:value and name=:name")).params(value=224, name='fred').order_by(Users.id).all()

Custom sql query

session.query(Users).from_statement(text("SELECT * FROM users where name=:name")).params(name='ed').all()

Statistics count

count = session.query(User).filter(User.name.like("%t%")).count()

Packet

session.query(func.count(User.name),User.name).group_by(User.name)

having

having the packet filter by

session.query(func.min(User.id), func.avg(User.id)).group_by(Users.name).having(func.min(Users.id) >2).all()

Aggregate function

• func.count: Number of statistics rows, and count the same action

  fc=session.query(func.count(User.name),User.name).group_by(User.name).all()

• func.avg: Averaging

  fc=session.query(func.avg(User.age),User.name).group_by(User.name).all()

• func.max: Seeking maximum

  fc=session.query(func.max(User.age),User.name).group_by(User.name).all()

• func.min: For the minimum

  fc=session.query(func.min(User.age),User.name).group_by(User.name).all()

• func.sum: Summing

  fc=session.query(func.sum(User.age),User.name).group_by(User.name).all()

modify

• The first: to check out the object, and then to give new value object fields

  obj = session.query(User).filter(User.name=='hades').first()
  obj.age = 27
  session.commit()  # 一定要提交

• The second: update () method requires passing a dictionary

  session.query(User).filter(User.name=='hades').update({'age':27})
  session.commit()  # 一定要提交

• Third: in the original based on the increase, similar to the F query in django

  For example: add 1 year of age

  Note: must be followed with synchronize_session

  • String:synchronize_session=False
  • Digital Type:synchronize_session=evaluata

  session.query(User).filter(User.id > 0).update({User.name: User.name + "099"}, synchronize_session=False)
  # session.query(User).filter(User.id > 0).update({"age": User.age + 1}, synchronize_session="evaluate")
  # session.commit()

Delete delete ()

session.query(Users).filter(Users.id > 4).delete()
session.commit()