In early September, I python 爬虫ignite interest, but crawling into a multi-channel real-time synchronization of data with read and write 文件is not easy, so start with a play mysql. This note, I will organize combat the most commonly used nearly a month's mysqlstatement will also relate to how python3and in mysqldata exchange.
About tools / libraries, special instructions:
1, I installed mysqland run directly from an administrator 命令行提示符（cmd）to view mysql, and did not install any mysqlvisual graphical interface tool.
2, in pythonthe script, I used pymysql, and sqlalchemyboth libraries and mysqlestablish a connection with pandasto process the data.
First, establish a connection with data exchange
And mysql interactive way, I am currently using a total of four kinds. Which uses running as administrator 命令行提示符（cmd）to view mysql, illustrating its operation can write another one. Here the space is not accounted for. mysql visual graphical interface tool, and I'm currently not used, there is no urgent need to use it. Three other way is through the python script.
Situational A: python calculus derived data, you want to write to the database
python script form class has been a lot of data, you want to write a one-time database, common code as follows:
import pandas as pd
# 与 mysql 建立连接
from sqlalchemy import create_engine
conn_eng = create_engine('mysql+pymysql://username:password@localhost:3306/databasename',encoding='utf8')
# 调用 pandas 的方法，数据写入mysql
pd.io.sql.to_sql(your_df, "table_name", conn_eng, if_exists='append',index=False)
Class data table, I use pandasthe dataframestructure. pd.io.sql.to_sql()There are many other uses of parameters, but this is my personal above the highest frequency of use. The effect is: no need to build your own table in advance, will automatically create a new table. The catch is: automatic generation of attribute table column, usually disagreeable, the need to check and modify.
If you do not want to pd.io.sql.to_sql()want to be more sophisticated, complex operations or is used in the following situations C.
Situation B: python script want to get data from mysql
If you already have a form, you want to submit an instruction to the table, need to return data, I use pandasthe read_sql ()data type returned is pandasof dataframe. sql query very good writing, specifically summarized below this article.
import pymysql
# 与 mysql 建立连接
conn = pymysql.connect('localhost','username','password','databasename')
# sql 语句定义为一个字符串
sql_search = 'select question_id from topic_monitor where is_title=0 ;'
# 调用 pandas 的 read_sql() 方法拿到 dataframe 结构的数据
question_ids = pd.read_sql(sql_search,conn)
# 关闭连接
conn.close()Situation C: python script unilaterally issued a directive to mysql, no need to get data
If you already have a form, when you want to submit an instruction to the table without returning data, such as: construction of the table, additions and changes to the data deletion of the name of the column, the column attribute modification, etc., as follows.
import pymysql
# 与 mysql 建立连接
conn = pymysql.connect('localhost','username','password','databasename')
cursor = conn.cursor()
# sql 语句定义为一个字符串，插入一行数据
sql_insert = 'INSERT INTO questions(q_id,q_title,q_description,q_keywords,q_people,q_pageview,time) VALUES( "'\
+ str(quesition_id) + '", "' + str(one[0])+ '", "' + str(one[1]) + '", "' + str(one[2]) + '", "' \
+ str(one[3]) + '", "' + str(one[4]) + '", "' + str(datetime.datetime.now()) + '");'
# sql 语句定义为一个字符串，修改某个数据（另一个表格）
sql_update = 'update topic_monitor SET is_title="1" where question_id = "' + str(quesition_id) + '";'
# 提交指令
cursor.execute(sql_insert)
cursor.execute(sql_update)
conn.commit()
# 插入一行数据；仅当该数据与表格已有数据不重复时才插入，否则就不会插入
sql_insert = 'INSERT INTO `topic_monitor`(question_id,is_title,q_type,topic_id,time) SELECT "'\
+ x[0] + '", "0", "0","' + str(topic_id) + '", "'+ str(now) + '" FROM DUAL WHERE NOT EXISTS(\
SELECT question_id FROM topic_monitor WHERE question_id = "' + x[0] + '")'
cursor.execute(sql_insert)
conn.commit()
# 关闭连接
cursor.close()
conn.close()Can be seen in several practical cases above, pythonthe mysqlprocess of implementing the interaction usually divided into: establishing a connection, the sql statement defined as a string, commit instruction, the connection is closed. Sql statement that the core skills; sql statement string In addition to defining the remaining three treatments were fixed wording.
I was in the first month of practice, the most frequently occurring errors are:
- Reference value is not in quotes;
- Symbol disorder: a multi-symbol, at least one symbol;
- The type of value does not conform: mysql table regardless of the value is the number, or text, when you define a sql statement string for each value needs to be converted to a string;
- When a copy of your code, forget to change databasename.
Two, sql statement: search query
Discovery is the query data that meet specific criteria in a table in the database and returns the query results. The basic structure is:
SELECT 【范围】FROM table_name 【条件】; Wherein, the range must be specified, and optional conditions.
Variable A: range, it refers to the return query results scope.
Return to the table all the fields with * expression:
SELECT * FROM table_name ;
Returns only a field of the form:
SELECT column_name FROM table_name ;Only return multiple fields of the form:
SELECT column_name_1,column_name_3,column_name_3 FROM table_name ;
The number of data returns only qualified:
SELECT count(*) FROM table_name ;
Variable B: refers to the condition, which satisfies the conditions expected to return data.
Is not limited to the conditions:
SELECT * FROM table_name ;Value classes: a field (numeric type, such as double or int), the numerical comparison operators may be used such as, greater than >, less than <, equal to =, greater than or equal >=, less than or equal <=:
SELECT * FROM table_name WHERE num_column_name >= 1;文本类：某个字段（字符串类型的，比如char，text）：
SELECT * FROM table_name WHERE str_column_name like “%your_str%”;
也可以表达多个条件，and，or等可用于表达条件之间的关系：
SELECT * FROM table_name WHERE num_column_name_1 >= 1 and str_column_name like “%your_str%” ;
三、sql语句：修改表属性
横向的一整条数据，叫做行；竖向的一整条数据，叫作列。列的名字，叫做 column，这是通用的知识点。
这段时间的实战中，我完全没有用到修改表的名称、重设index等知识点。最常用的，就是对列进行操作。每个列具备：列的名称、列的属性、列的数值。
列的名称，需要留心不使用保留词。我的技巧是，尽量用一些_来表达该数据，比如 article_title，press_date 这种命名虽然稍长，但易读，也不会装上保留词。
列的属性包括：类型，最大长度，是否为空，默认值，是否重复，是否为索引。通常，直接通过 pandas 的 pd.io.sql.to_sql() 一次性创建表格并保存数据时，列的默认属性并不合需求。要么提前自己定义表的结构，设置好每列属性；要么事后检查列属性，并逐列修改。所以，列的属性设定、修改是高频基础知识点。
列的数值，即除了列名称外的、该列其它值。修改某个值，也是高频操作。不过我把这个知识点放到第四部分了。
对列的名称、列的属性进行修改，主要的关键词都是 ALTER，具体又分为以下几种情况。
情境A：新增一列。关键词 ADD
在你所指定的 column_name 后面定义列的属性。
ALTER TABLE table_name ADD COLUMN column_name char(20);情境B：修改某列的名称。关键词 CHANGE
在修改列名的同时也可以重新指定列的属性。
ALTER TABLE table_name CHANGE old_column_name new_column_name char(50);情境C：修改某列的属性。关键词是 MODIFY
ALTER TABLE table_name MODIFY column_name char(100);四、sql语句：数据的增改删
通常提到数据库操作时，四字以蔽之：增删改查。
- 查询，请看第二部分。关键词是
SELECT。 - 对数据所依赖的属性的增、改，请看第三部分。关键词是
ALTER。 - 数据的增加，在第一部分的数据交互中也给出实例，就不重复了。关键词是
INSERT。 - 数据的修改，关键词是
UPDATE。 - 数据（甚至表格、库）的删除，关键词是
DELETE。
数据的修改，副关键词是 set 。
UPDATE table_name SET columns_name = new_value 【条件】;新数值如果是数值类型的，则直接写数值即可；如果是文本类型的，必须要加上双引号，比如，“your_new_value”。
如果把【条件】部分不写，就相当于修改整列的值；想要修改特定范围，就要用到条件表达式，这和前面的查询部分是一致的，就不再重复。
数据的删除，对于新手来说，是必须警惕的操作。因为一旦误操作，你将无力挽回。即便是职业程序员，也可能犯下无疑删库的惨剧。其基本语句为：
DELETE FROM table_name【条件】;想要修改特定范围，就要用到条件表达式，这和前面的查询部分也是一致的，稍微啰嗦两句：不要对自己设定的条件太自信，最好先用搜索语句检查一下，然后再执行删除语句。
- 删除单行数据：添加能唯一标识该行数据的条件语句。
- 删除多行数据：添加能标识该范围的条件语句。
- 删除整张表格：你是认真的吗？没有写错表格名字吧？！ 做这项操作前，必须确认清楚自己的意图，毕竟一旦发生，无可挽回。
如果条件留空，将保留表结构，而删除所有数据行。想要删除整张表格，什么都不留下，则执行：
DELETE TABLE table_name;俗称的“删库”就是删掉整个数据库，虽然实战中几乎不会用到，但作为新手经常手误，在练习阶段安全起见，最好还是专门创建一个 database 用于练手，练完直接删掉整个练习库：
DELETE DATABASE database_name;如果简单总结下过去一个月，使用mysql的体验，那就是：除了mysql 的安装激活太麻烦，数据的增删改查比操作文本方便太多了！！完全值得容忍安装激活的麻烦。另外 mysql 常用语法确实简单、非常有规律。
I want to give you a summary of help. Encouraged me to continue to share, then please point a praise it! Errata please leave a message, or even move my GitHub: https://github.com/liujuanjuan1984/ucanuupnobb/issues