This document introduces how to get started with Pandas. Pandas is a data analysis library for Python, which can easily manipulate data and perform data analysis.
This section imports Pandas and NumPy in the following way:
In [1]: import numpy as np
In [2]: import pandas as pd
# generate object
When generating a Seriesopen in new window with a list of values , Pandas automatically generates integer indices by default:
In [3]: s = pd.Series([1, 3, 5, np.nan, 6, 8])
In [4]: s
Out[4]:
0 1.0
1 3.0
2 5.0
3 NaN
4 6.0
5 8.0
dtype: float64
Generate a DataFrameopen in new window from a NumPy array with datetime indices and labels :
In [5]: dates = pd.date_range('20130101', periods=6)
In [6]: dates
Out[6]:
DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03', '2013-01-04',
'2013-01-05', '2013-01-06'],
dtype='datetime64[ns]', freq='D')
In [7]: df = pd.DataFrame(np.random.randn(6, 4), index=dates, columns=list('ABCD'))
In [8]: df
Out[8]:
A B C D
2013-01-01 0.469112 -0.282863 -1.509059 -1.135632
2013-01-02 1.212112 -0.173215 0.119209 -1.044236
2013-01-03 -0.861849 -2.104569 -0.494929 1.071804
2013-01-04 0.721555 -0.706771 -1.039575 0.271860
2013-01-05 -0.424972 0.567020 0.276232 -1.087401
2013-01-06 -0.673690 0.113648 -1.478427 0.524988
Generate a DataFrame from a Series dictionary object:
In [9]: df2 = pd.DataFrame({'A': 1.,
...: 'B': pd.Timestamp('20130102'),
...: 'C': pd.Series(1, index=list(range(4)), dtype='float32'),
...: 'D': np.array([3] * 4, dtype='int32'),
...: 'E': pd.Categorical(["test", "train", "test", "train"]),
...: 'F': 'foo'})
...:
In [10]: df2
Out[10]:
A B C D E F
0 1.0 2013-01-02 1.0 3 test foo
1 1.0 2013-01-02 1.0 3 train foo
2 1.0 2013-01-02 1.0 3 test foo
3 1.0 2013-01-02 1.0 3 train foo
The columns of the DataFrame have different ** datatypes open in new window **.
In [11]: df2.dtypes
Out[11]:
A float64
B datetime64[ns]
C float32
D int32
E category
F object
dtype: object
IPython supports tab key auto-completion of column names and public attributes. The following are some of the attributes that can be automatically completed:
In [12]: df2.<TAB> # noqa: E225, E999
df2.A df2.bool
df2.abs df2.boxplot
df2.add df2.C
df2.add_prefix df2.clip
df2.add_suffix df2.clip_lower
df2.align df2.clip_upper
df2.all df2.columns
df2.any df2.combine
df2.append df2.combine_first
df2.apply df2.compound
df2.applymap df2.consolidate
df2.D
Columns A, B, C, D, and E are all autocompleted; for brevity, only some of the properties are shown here.
# view data
The following code shows how to view the DataFrame head and tail data:
In [13]: df.head()
Out[13]:
A B C D
2013-01-01 0.469112 -0.282863 -1.509059 -1.135632
2013-01-02 1.212112 -0.173215 0.119209 -1.044236
2013-01-03 -0.861849 -2.104569 -0.494929 1.071804
2013-01-04 0.721555 -0.706771 -1.039575 0.271860
2013-01-05 -0.424972 0.567020 0.276232 -1.087401
In [14]: df.tail(3)
Out[14]:
A B C D
2013-01-04 0.721555 -0.706771 -1.039575 0.271860
2013-01-05 -0.424972 0.567020 0.276232 -1.087401
2013-01-06 -0.673690 0.113648 -1.478427 0.524988
Show index and column names:
In [15]: df.index
Out[15]:
DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03', '2013-01-04',
'2013-01-05', '2013-01-06'],
dtype='datetime64[ns]', freq='D')
In [16]: df.columns
Out[16]: Index(['A', 'B', 'C', 'D'], dtype='object')
DataFrame.to_numpy()open in new window outputs a NumPy object of the underlying data. Note that when the columns of DataFrameopen in new window consist of multiple data types, this operation consumes a lot of system resources, which is also the essential difference between Pandas and NumPy: NumPy arrays have only one data type, and the data types of each column of DataFrame are different . When calling DataFrame.to_numpy() open in new window , Pandas looks for NumPy data types that support all data types in the DataFrame. As another data type object, the values in a DataFrame column can be coerced into Python objects.
Summarize
Using Pandas' DataFrame, you can do the following: Load and process data, including data in CSV, Excel, SQL database, etc. Data cleaning, reshaping, slicing and dicing statistics, computing aggregated and summarized information Visualize the results Save to files in various formats Upload data to databases or online applications, output data in other formats, such as HTML tables.