AJ-Report open source data visualization engine entry practice

background

A data visualization engine is a software tool or platform used in the process of turning data into visual charts, graphs, and dashboards. It helps users better understand and analyze data to support decision-making and insights.

The reasons for the increasing demand for data visualization engines are as follows:

Data Explosion: With the advent of the digital age, the amount of data is increasing exponentially. Data visualization engines can help people better process and understand large amounts of data, thereby discovering patterns and trends hidden in the data.
Visual insight: Humans understand graphs and charts far more than they can understand plain text data. By visualizing data, people can more intuitively understand the relationship and trends between data, so as to better discover and solve problems.
Instant decision-making: In today's fast-paced business environment, instant decision-making is becoming more and more important. The data visualization engine can help users quickly generate visual reports and dashboards, enabling them to quickly obtain key information and make timely decisions.
User-friendliness: Data visualization engines usually provide user-friendly interfaces and interactive features, enabling non-technical people to easily create and customize their own visualization charts and dashboards. This makes data visualization engines more and more widely used in various industries and fields.

This article is AJ-Reporta small summary of internal sharing, which can quickly realize data visualization and is easy to deploy and integrate; at that time, it was explained and demonstrated according to the following content.

Download, source code start
Data source, data set, big screen
Big screen visualization

Pie Charts: Static Data
line chart:SQL
Histogram:HTTP
import

Extended data source:ClickHouse

Selection analysis

Open source data visualization engine: DataEase, FlyFish, GoView, DataGear, Superset, Grafana, Metabase…

Advantage

AJ-Report is a fully open source domestic BI platform, based on Spring Boot+MyBatisPlus+Vue technology stack, which separates front and back ends, and is easy to deploy and integrate.
Multi-data source support, built-in MySQL, ElasticSearch, Kudu drivers, support for dynamic expansion of data sources (OLTP: domestic database; OLAP: ClickHouse; data warehouse/data lake: Hive).
Supports custom data sets (directly connected to the database) to save data interface development work; supports preprocessing operations such as cleaning, conversion, and preview of response data.
Supports calling existing interfaces in the form of HTTP interfaces; supports cleaning, conversion, preview and preprocessing of response data.
At present, it supports 30+ kinds of large-screen components/charts, and it does not know how to develop. You can also make a large-screen according to the design draft; you can use the provided template to import and develop.
Five steps to easily complete the design and release of the large screen: configure the data source ----> write SQL configuration data set ----> drag and drop to configure the large screen ----> save and release ----> one-click share/export .

disadvantage

The provided module components do not necessarily meet the actual needs, but can be realized through secondary development.
Large-screen charts do not support drill-down, but simple linkage of multiple components can be achieved.
Syntax JavaScriptis not supported when data conversion is selected .ES6

download start

https://gitee.com/anji-plus/report

basic environment

Node.js
JDK
Maven
MySQL

development tools

IDEA
VSCode

Operating procedures

Five steps to easily complete the design and release of the large screen:
configure the data source ----> write SQL configuration data set ----> drag and drop to configure the large screen ----> save and release ----> one-click share/export .
Data source, data set, big screen

Big screen visualization

Pie Chart (Static Dataset)

Line Chart (SQL Dataset)

data source

jdbc:mysql://127.0.0.1:3306/zaservice?useUnicode=true&characterEncoding=UTF-8&serverTimezone=GMT%2B8

Check for phrases

SELECT create_time, value FROM sensor_water WHERE CODE = "00035100003003" ORDER BY create_time LIMIT 30;

conversion script

function dataTransform(data) {
    
    
    //自定义脚本内容
    for (var i = 0; i < data.length; i++) {
    
    
        data[i].create_time = new Date(data[i].create_time).toLocaleTimeString()
    }
    return data;
    // return data.map(x => new Date(x.create_time).toLocaleTimeString());
}

Histogram (HTTP dataset)

API interface

http://localhost:8000/api/dashboard/getAssetsLiabilitiesRatio

function dataTransform(data) {
    
    
    //自定义脚本内容
    var d = JSON.parse(data[0]).data;
    var result = [];
    for (var i = 0; i < d.companies.length; i++) {
    
    
        result.push({
    
    
            axis: d.companies[i],
            data: d.assetsLiabilitiesRatio[i]
        });
    }
    return result;
}

import large screen

AJReportSome large-screen templates are provided, which can be downloaded and imported for direct use or modification.

Extended data source: ClickHouse

The following figure shows DataEasethe data sources supported by , which makes people feel very powerful when shown in this way. In fact, AJReportit also supports the expansion of data sources. Let's take ClickHousethis analytical database as an example to JDBCexpand the data sources in the form of .

Query billions of rows in milliseconds

ClickHouse is the fastest and most resource efficient open-source database for real-time apps and analytics.

Web interactive query page: http://192.168.44.148:8123/play

show databases;
select count(*) from poetry.poetry_mergetree;

SELECT author, count(*) AS count FROM poetry.poetry_mergetree GROUP BY author HAVING count >=1000 ORDER BY count DESC;
SELECT dynasty, count(*) AS count FROM poetry.poetry_mergetree GROUP BY dynasty;

Introduce dependencies

        <!--ClickHouse 依赖-->
        <dependency>
            <groupId>ru.yandex.clickhouse</groupId>
            <artifactId>clickhouse-jdbc</artifactId>
            <version>0.3.2</version>
        </dependency>

configuration information

ru.yandex.clickhouse.ClickHouseDriver
jdbc:clickhouse://your-ip:8123/poetry

Hands

Build a local development environment;
Front end: modify the source code to expand the chart component type;
Backend: Modify the source code to expand the data source type;
Full stack: integrated into the rapid development framework (ruoyi, jeecgboot) monomer and micro-service architecture;
Big data/data mining: use SQL to achieve multi-dimensional statistics on open data sets;
Business: Use tools to complete the design and development of large screens.

Reference

If you have any questions or any bugs are found, please feel free to contact me.
Your comments and suggestions are welcome!