Six exploratory data analysis (EDA) tools, so practical!

When performing data analysis, Exploratory Data Analysis (EDA) is a crucial stage that helps us discover patterns, trends, and anomalies from the data. Choosing the right EDA tool can greatly improve work efficiency and analysis depth. In this article, the author will introduce 6 extremely practical exploratory data analysis (EDA) tools. These tools can help you better understand the data, discover hidden information, and provide strong support for subsequent analysis and decision-making. Let’s take a look at how these tools help us explore the world of data!

1. SweetViz

SweetViz is an open source Python library that can generate beautiful and high-density visualization charts for fast exploratory data analysis (EDA) with just two lines of code. Its output is a completely independent HTML application.

Its original design is to quickly visualize target values ​​​​and compare data sets to help quickly analyze target features, differences between training data and test data, as well as the structure of the data set, the relationship between features, the distribution of data, etc., thereby accelerating data The process of analysis.

Here is a simple example that demonstrates how to use SweetViz for exploratory data analysis:

import pandas as pd
import sweetviz as sv
import numpy as np

data = pd.DataFrame({'随机数': np.random.randint(1, 100, 100)})

# 创建SweetViz 报告
report = sv.analyze(data)

# 将报告保存为HTML文件
report.show_html('random_report.html')

2. ydata-profiling

ydata-profiling is a Python library for data exploration and analysis, which can help users quickly understand and analyze the contents of data sets. By using ydata-profiling, users can generate reports on statistical information, distribution, missing values, correlations, etc. of various variables in the data set. This can help users understand the characteristics of the data set faster during the data analysis stage, so as to better perform subsequent data processing and modeling work.

The following is a simple example code that shows how to use ydata-profiling to analyze a data set:

import pandas as pd
from ydata_profiling import ProfileReport

df = pd.read_csv('data.csv')
profile = ProfileReport(df, title="Profiling Report")

3. DataPrep

Dataprep is an open source Python package for analyzing, preparing, and processing data. DataPrep is built on top of Pandas and Dask DataFrame and can be easily integrated with other Python libraries.

Here is a simple example that demonstrates how to use DataPrep for exploratory data analysis:

from dataprep.datasets import load_dataset
from dataprep.eda import create_report

df = load_dataset("titanic.csv")
create_report(df).show_browser()

4. AutoViz

The Autoviz package can automatically visualize data sets of any size with one line of code and automatically generate HTML, bokeh and other reports. Users can interact with HTML reports generated by the AutoViz package.

Here is a simple example code showing how to use AutoViz:

from autoviz.AutoViz_Class import AutoViz_Class

AV = AutoViz_Class()
filename = "" # 如果有文件名，可以在这里指定
sep = "," # 数据集的分隔符
dft = AV.AutoViz(
    filename,
    sep=",",
    depVar="",
    dfte=None,
    header=0,
    verbose=0,
    lowess=False,
    chart_format="svg",
    max_cols_analyzed=30,
    max_rows_analyzed=150000,
    )

5. D-Tale

D-Tale is a tool that combines a Flask backend and a React frontend to provide users with an easy way to view and analyze Pandas data structures. It integrates perfectly with Jupyter notebooks and Python/IPython terminals. Currently, the tool supports Pandas data structures, including DataFrame, Series, MultiIndex, DatetimeIndex and RangeIndex. Users can visually view data, generate statistical information, create visual charts, and perform some data processing operations in the browser through D-Tale. The structure of D-Tale makes data analysis more intuitive and convenient, providing users with an efficient data exploration and analysis tool.

6. Table

Dabl focuses less on statistical measures of individual columns and more on providing a quick overview through visualization, as well as convenient machine learning preprocessing and model search. The Plot() function in Dabl can achieve visualization by drawing various graphs, including:

• target distribution map
• scattering pair diagram
• linear discriminant analysis

Here is a simple example code showing how to use Dabl:

import pandas as pd
import dabl

df = pd.read_csv("titanic.csv")
dabl.plot(df, target_col="Survived")