Data analysis common skills and experience summary

1. Pandas formats the dateime type as a string

There are many data types in Pandas, one of which is datetime, that is, date and time. For example Timestamp(‘2020-09-22 20:43:00’), it means that it is a timestamp type. Many times it needs to be converted to a string in order to get the date or time. Its calling strftime()method, if strftime('%Y-%m-%d')you can get a string 2020-09-22.
In a DataFrame in pandas, it is generally a whole column replacement. At this time, lambda expressions and applymethods are needed , as follows:

order_detail['date'] = order_detail['date'].apply(lambda x:x.strftime('%Y-%m-%d'))

It can complete the datetime type into a string of the specified format.

2.Pandas reads .sql files

There are many ways to read data in pandas and supported formats, including reading database data, but generally you cannot read .sqlfiles directly . Instead, you generally execute SQL statements in the .sql file to import the data into the MySQL database, and then use pandas Read data from the database.
You can generally use database visualization tools such as Navicat and SQLYog to execute the SQL statements in the .sql file. Here we take Navicat as an example to import the data in the .sql file as follows:

Then use Python to read the data from the database, as follows:

import pandas as pd
import pymysql

sql = 'select * from table_name' # 换成自己的表名

con = pymysql.connect(host='127.0.0.1', port=3306,user='root',password='root',database='python_da',charset='gbk') # 换成自己的数据库
order_detail = pd.read_sql(sql,con)

If there is no mymysql library, you can execute pip install pymysqlor conda install pymysqlinstall it.