Operating mysql knowledge in Python (1)

The Python standard database interface is Python DB-API, which provides developers with a database application programming interface.

The Python database interface supports a very large number of databases, you can choose the database that suits your project:

• GadFly
• mSQL
• MySQL
• PostgreSQL
• Microsoft SQL Server 2000
• Informix
• Interbase
• Oracle
• Sybase

Python's DB-API implements interfaces for most databases. After using it to connect to each database, each database can be operated in the same way.

Python DB-API usage process:

1. Introduce the API module.
2. Obtain a connection to the database.
3. Execute SQL statements and stored procedures.
4. Close the database connection.
 

What is MySQLdb?

MySQLdb is an interface for Python to link Mysql database. It implements Python database API specification V2.0 and is based on MySQL C API.
How to install MySQLdb?
In order to write MySQL scripts with DB-API, you must ensure that MySQL is installed. Copy the following code and execute:

#!/usr/bin/python
# -*- coding: UTF-8 -*-

import MySQLdb

If the output after execution is as follows, it means that you do not have the MySQLdb module installed:

Traceback (most recent call last):
  File "test.py", line 3, in <module>
    import MySQLdb
ImportError: No module named MySQLdb

To install MySQLdb, please visit http://sourceforge.net/projects/mysql-python, (Linux platform can visit: https://pypi.python.org/pypi/MySQL-python) From here, choose the one that suits your platform Installation packages are divided into precompiled binary files and source code installation packages.
If you choose a binary distribution, the installation process is done with basic installation prompts. If installing from source, you need to change to the top-level directory of the MySQLdb distribution and type the following command:

$ gunzip MySQL-python-1.2.2.tar.gz
$ tar -xvf MySQL-python-1.2.2.tar
$ cd MySQL-python-1.2.2
$ python setup.py build
$ python setup.py install

It seems that pip does not support the installation of MySQLdb. We can download and install it through the website. The
download address is: https://pypi.python.org/pypi/MySQL-python/1.2.5
There are methods for installing windows and source code respectively.

Install dependencies:

yum install –y python-devel
yum install –y mysql-devel
yum install –y gcc

After Python 3, it seems that MySQLdb is not supported. You can use the pymysql package, which
can be used directly through pymysql.

pip install pymysql

MySQLdb is only applicable to python2.x, and it is found that pip cannot be installed. Its replacement in py3 is: import pymysql
 

Mysql transaction

Generally speaking, a transaction must meet 4 conditions (ACID): Atomicity (atomicity), Consistency (stability), Isolation (isolation), Durability (reliability)
1. The atomicity of a transaction: a set of transactions, or success; or withdraw.
2. Stability: If there is illegal data (foreign key constraints, etc.), the transaction is withdrawn.
3. Isolation: Transactions run independently. If the result of a transaction affects other transactions, then other transactions will be withdrawn. 100% isolation of transactions at the expense of speed.
4. Reliability: After the software and hardware crash, the InnoDB data table driver will use the log file to reconstruct and modify it. Reliability and high speed cannot have both, the innodb_flush_log_at_trx_commit option determines when to save the transaction to the log
 

MariaDB [(none)]> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
MariaDB [mysql]> show variables like 'auto%';
+--------------------------+-------+
| Variable_name            | Value |
+--------------------------+-------+
| auto_increment_increment | 1     |
| auto_increment_offset    | 1     |
| autocommit               | ON    |
| automatic_sp_privileges  | ON    |
+--------------------------+-------+
4 rows in set (0.01 sec)

MariaDB [mysql]> 

Install Mysql

1. Go to the official website and download the source code package

 

2. Install dependent tools:

yum -y install make gcc-c++ cmake bison-devel ncurses-devel libaio libaio-devel perl-Data-Dumper net-tools

3. Unzip

rz

cd /usr/local/src

tar xzvf mysql-5.6.32.tar.gz

cd mysql-5.6.32

4. Compile:

cmake -DCMAKE_INSTALL_PREFIX=/export/servers/mysql/ -DMYSQL_DATADIR=/export/Data/mysql/data -DSYSCONFDIR=/export/servers/mysql/etc -DWITH_INNOBASE_STORAGE_ENGINE=1 -DWITH_PERFSCHEMA_STORAGE_ENGINE=1 -DDEFAULT_CHARSET=utf8 -DDEFAULT_COLLATION=utf8_general_ci -DMYSQL_UNIX_ADDR=/export/Data/mysql/tmp/mysql.sock -DENABLED_LOCAL_INFILE=ON -DENABLED_PROFILING=ON -DWITH_DEBUG=0 -DMYSQL_TCP_PORT=3358

 

DCMAKE_INSTALL_PREFIX	/usr/local/mysql	installation manual
DMYSQL_DATADIR	/usr/local/mysql/data	database location
DSYSCONFDIR	/etc	Configuration file location
DWITH_MYISAM_STORAGE_ENGINE	1	Install the myisam storage engine
DWITH_INNOBASE_STORAGE_ENGINE	1	Install the innodb storage engine
DWITH_MEMORY_STORAGE_ENGINE	1	Install memory storage engine
DWITH_READLINE	1	Shortcut function
DMYSQL_UNIX_ADDR	/var/lib/mysql/mysql.sock	Unix socket file path
DMYSQL_TCP_PORT	3306	MySQL listening port
DENABLED_LOCAL_INFILE	1	Allow to import data from local -
DWITH_PARTITION_STORAGE_ENGINE	1	Mount the database partition
DEXTRA_CHARSETS	all	Install all extended character sets
DDEFAULT_CHARSET	utf8	use utf8 characters
DDEFAULT_COLLATION	utf8_general_ci	check character

As shown in the figure, the compilation is successful, and echo $? is performed each time to verify

make && make install

5. Create mysql user= and empower

useradd mysql -s /sbin/nologin

chown -R mysql:mysql /export/servers/mysql /export/Data/mysql

6. Initialize the system table:

cd /export/servers/mysql/scripts

./mysql_install_db --basedir=/export/servers/mysql/ --datadir=/export/Data/mysql/data --user=mysql

mkdir -p /export/servers/mysql/etc

cp /export/servers/mysql/my.cnf

yum remove -y mysql

7. Start mysql

cd /export/servers/mysql

cp support-files/mysql.server /etc/init.d/mysqld

Check whether the datadir, basedir, etc. of the configuration file are correct

service mysqld start

8. Check if mysql is started successfully

ps -ef |grep mysql

netstat -lnp |grep mysql

9. If it fails to start, look for the log corresponding to `hostname`.err in /export/Data/mysql/data

Mysql common operations

• Authorized superuser:

•grant all privileges on *.* to 'fengxiaoqing'@'%' identified by 'admin@123' with grant option;

• View the library:

•show databases;

• View which libraries are available show databases;

• View a database table use db; show tables \G;

• View table fields desc tb;

• View the table creation statement show create table tb;

•Which user is currently select user();

• The current library select database();

• Create database create database db1;

• Create table create table t1 (id int, name char(40) adress varchar(30)); 

•char(10)              'aaa       '

•varchar(10)          'aaa'

• View database version select version();

• View mysql status show status;

• Modify mysql parameters show variables like 'max_connect%'; set global max_connect_errors = 1000;

• View mysql queue show processlist;

•select * from information_schema.processlist where info is not null;

• sleep can be ignored, only query query

• Create a common user and authorize grant all on *.* to databases1.user1 identified by '123456';

•grant all on db1.* to 'user2'@'10.0.2.100' identified by '111222';

•grant all on db1.* to 'user3'@'%' identified by '231222';insert into tb1 (id,name) values(1,'aming');

•更改密码 UPDATE mysql.user SET password=PASSWORD("newpwd") WHERE user='username' ;  

•查询 select count(*) from mysql.user; select * from mysql.db; select * from mysql.db where host like '10.0.%';

•插入 update db1.t1 set name='aaa' where id=1; 

•Empty table truncate table db1.t1;

• drop table drop table db1.t1;

• drop database drop database db1;

• Repair table repair table tb1 [use frm];

• View permissions show grants for root@'localhost';

•echo "select user,host,password from mysql.user" |mysql -uroot -plingxiangxiang

•mysql -uroot -p1234556 -e "select user,host,password into outfile '/home/mysql/1.txt' from mysql.user;"

Mysql connection

•1. Create database create database python;

•2. Authorized Users

•grant all privileges on *.* to feng@’%’ identified by ‘123456’;

•flush privilege;

•conn=MySQLdb.connect(host="192.168.100.101",user="feng",passwd="123456",db="python",charset="utf8")

 

The more commonly used parameters include:

• host: database host name. The default is to use the local host

• user: database login name. The default is the current user

• passwd: The secret of the database login. The default is empty

• db: The database name to use. No default value

•port: The TCP port used by the MySQL service. The default is 3306, numeric type

• charset: database encoding

 

The recommended way to use the function:

def connect_mysql():
    db_config = {
        'host': '192.168.48.128',
        'port': 3306,
        'user': 'xiang',
        'passwd': '123456',
        'db': 'python',
        'charset': 'utf8'
    }
    cnx = MySQLdb.connect(**db_config)
return cnx

 

cursor

Cursor (Cursor) is a method of processing data, in order to view or process the data in the result set, the cursor provides the ability to browse data forward or backward in the result set one or more rows at a time. The cursor can be regarded as a pointer, which can specify any position in the result, and then allows the user to process the data at the specified position

Import…………
if __name__ == '__main__':
    cnx = connect_mysql()
    cus = cnx.cursor()
    sql  = ''' create table test(id int not null);insert into test(id) values (100);'''
    try:
        cus.execute(sql)
        cus.close()
        cnx.commit()
    except Exception as e:
        cnx.rollback()
        print('Error')
        # raise e
    finally:
        cnx.close()

Common methods of cursors:
Common methods:
cursor(): Create a cursor object
Cus = connect_mysql().cursour()
close(): Close this cursor object
execute(sql[, args]): Execute a database query or command
fetchone() : Get the next row of the result set
fetchmany([size = cursor.arraysize]): Get the next few rows of the result set
fetchall(): Get all the remaining rows in the result set
executemany (sql, args): Execute multiple database queries or Order

Personal recommendation, try not to use executemany, and continuously call the execute function through the program loop
 

database connection pool

import pymysql
from DBUtils.PooledDB import PooledDB

from DBUtils.PooledDB import PooledDB
db_config = {
        "host": "192.168.48.131",
        "port": 3306,
        "user": "xiang",
        "passwd": "xiang",
        "db": "python",
        # "charset": "utf8"
    }

spool = PooledDB(pymysql, 5, **db_config) # 5 is the minimum number of connections in the connection pool
# def connect_myssql():
conn = spool.connection() # Every time you need a database connection in the future, use the connection() function to get the connection
cur = conn.cursor()
SQL = "select * from tmp;"
r = cur.execute(SQL)
r = cur.fetchall()
print(r)
cur.close()
conn.close()

 

 

Reference article: http://www.runoob.com/python/python-mysql.html