Python Data Analysis: Analyzing Pandas defaults

Python Data Analysis: Analyzing Pandas defaults

background

We extracted data into Pandas None NaN converted from a database or NaT. However, we will Pandas data is written to the database needs to be converted into time and None, otherwise it will error. Therefore, we need to deal with the default value of Pandas.

sample

   id         name  password  sn  sex  age  amount  content  remark  login_date login_at    created_at  
0   1  123456789.0       NaN NaN  NaN   20     NaN      NaN     NaN  NaN        NaT         2019-08-10 10:00:00  
1   2          NaN       NaN NaN  NaN   20     NaN      NaN     NaN  NaN        NaT         2019-08-10 10:00:00 

The default value is determined

If columnthe default value, the unity deal to None.

def judge_null(column):
    if pd.isnull(column):
        return None
    return column

Processing defaults

By column processing defaults.

df['id'] = df.apply(lambda row: judge_null(row['id']), axis=1)
df['name'] = df.apply(lambda row: judge_null(row['name']), axis=1)
df['password'] = df.apply(lambda row: judge_null(row['password']), axis=1)
df['sn'] = df.apply(lambda row: judge_null(row['sn']), axis=1)
df['sex'] = df.apply(lambda row: judge_null(row['sex']), axis=1)
df['age'] = df.apply(lambda row: judge_null(row['age']), axis=1)
df['amount'] = df.apply(lambda row: judge_null(row['amount']), axis=1)
df['content'] = df.apply(lambda row: judge_null(row['content']), axis=1)
df['remark'] = df.apply(lambda row: judge_null(row['remark']), axis=1)
df['login_date'] = df.apply(lambda row: judge_null(row['login_date']), axis=1)
df['login_at'] = df.apply(lambda row: judge_null(row['login_at']), axis=1)
df['created_at'] = df.apply(lambda row: judge_null(row['created_at']), axis=1)

After completion of the processing of data

   id         name  password  sn    sex    age   amount    content  remark  login_date  login_at  created_at  
0   1  123456789.0      None  None  None   20    None      None     None    None        None      2019-08-10 10:00:00  
1   2         None      None  None  None   20    None      None     None    None        None      2019-08-10 10:00:00 

supplement

Set the display of all rows, columns and worth length.

# 显示所有列
pd.set_option('display.max_columns', None)
# 显示所有行
pd.set_option('display.max_rows', None)
# 设置value的显示长度为100，默认为50
pd.set_option('max_colwidth', 100)

Built corresponding database table statement

create table test
(
  id         int(10)        not null primary key,
  name       varchar(32)    null,
  password   char(10)       null,
  sn         bigint         null,
  sex        tinyint(1)     null,
  age        int(5)         null,
  amount     decimal(10, 2) null,
  content    text           null,
  remark     json           null,
  login_date date           null,
  login_at   datetime       null,
  created_at timestamp      null
);