Step-by-step tutorial MO | How to realize visual monitoring of MatrixOne through Superset

[Live broadcast preview] 99 yuan cloud virtual machine × 9 yuan cloud native architecture √

Part 1 Overview

Superset is an open source, modern, lightweight BI analysis tool that can connect multiple data sources, provide rich visual charts, and support customized dashboards to help users easily explore and present data.

MatrixOne version 1.0 now supports integration with the data visualization tool Superset. This guide will guide you through the rapid deployment of MatrixOne and Superset environments, by combining the visualization capabilities of MatrixOne and Superset, to create a simple monitoring panel for monitoring the MatrixOne database, using the 'system_metric' data in it.

If you wish to extend functionality further, you can also explore additional configuration options to monitor aspects of the entire MatrixOne database.

Part 2 Preparation

2.1 Hardware environment

This practice does not have high hardware requirements for the machine. A small 2C 4G virtual machine can complete the functional experience of this process.

Recommended hardware resources are: 8C 32G virtual machine.

2.2 Software environment

This practice requires the installation and deployment of the following software environments:

Docker, version requirement is 23.0.1 and above.
MatrixOne
Superset, the recommended version is 2.1.0.

You can refer to the following chapters for installation:

Install Docker

The installation of all software environments in this practice is based on Docker. You can refer to the official Docker documentation (Link: / https://docs.docker.com/get-docker/ ) to install and start Docker.

InstallMatrixOne

You can refer to the macOS environment to use Docker to deploy MatrixOne (Link / https://docs.matrixorigin.cn/1.0.0/MatrixOne/Get-Started/install-on-macos/install-on-macos-method3/ . / )

Or use Docker to deploy MatrixOne in Linux environment (Link: https://docs.matrixorigin.cn/1.0.0/MatrixOne/Get-Started/install-on-linux/install-on-linux-method3/ . / ) for installation. and start MatrixOne.

Install Superset

The steps to deploy a single-node Superset using Docker are as follows:

1. After completing the installation and starting Docker, use the following command to pull the Superset image from Docker Hub:

docker pull amancevice/superset

2. Start the Superset image:

docker run -e "SUPERSET_SECRET_KEY=your_secret_key_here" --name superset -u 0 -d -p 8088:8088 amancev5ce/supersetNote：

# Note

Security keys can be generated via openssl rand -base64 $num, for example, generate key openssl rand -base64 49.

参数可参考官网说明：Your App secret key will be used for securely signing the session cookie and encrypting sensitive information on the database. Make sure you are changing this key for your deployment with a strong key. You can generate a strong key using openssl rand -base64 42. Alternatively you can set it with SUPERSET_SECRET_KEY environment variable.

3. Initialize the Superset database using the following command:

 docker exec -it superset superset db upgrade

4. Use the following command to create a Superset administrator user and enter relevant registration information as prompted:

docker exec -it superset superset fab create-admin

5. Create the default account using the following command:

docker exec -it superset superset init

6. Use the following command to start the service and enable threads, automatic reloading, and debugging mode:

docker exec -it superset superset run --with-threads --reload --debugger

Part 3 Connecting MatrixOne through Superset

Step 1 - Visit Superset’s login page

Usually http:// ip:8080 , then enter your username and password to log in to Superset.

# Note

The port for Superset may be 8080 or 8088, depending on your configuration; the username and password are set when you deploy Superset.

After logging in, you will see the main interface of Superset.

Step 2 - Create database connection

In Superset, you first need to create a database connection with MatrixOne. Click Settings in the upper right corner and select Database Connections .

On the Database Connections page, click the + DATABASE button and select MySQL as the database type.

Fill in the connection information for the MatrixOne database, including host, port, username and password.

After filling in, click the CONNECT button, and then click FINISH.

Part 4 Create a visual monitoring dashboard

You can now create a monitoring dashboard using the MatrixOne database.

Click SQL > SQL Lab on the page , select the MatrixOne database connection you just created, and write a SQL query to select the data table to monitor.

You can write multiple queries to monitor different metrics. The following is the SQL statement for the sample query:

CPU utilization:

docker exec -it superset superset run --with-threads --reload --debugger

Storage usage:

1 SELECT metric_name, value, collecttime
2 FROM metric
3 WHERE metric_name = 'server_storage_usage'
4 ORDER BY collecttime DESC;

Number of connections:

1 SELECT metric_name, value, collecttime
2 FROM metric
3 WHERE metric_name = 'server_connections'
4 ORDER BY collecttime DESC;

Disk reading and writing:

1 SELECT metric_name, value, collecttime
2 FROM metric
3 WHERE metric_name = 'sys_disk_read_bytes' OR metric_name = 'sys_disk_write_bytes'
4 ORDER BY collecttime DESC;

Network receiving and sending:

1 SELECT metric_name, value, collecttime
2 FROM metric
3 WHERE metric_name = 'sys_net_sent_bytes' OR metric_name = 'sys_net_recv_bytes'
4 ORDER BY collecttime DESC;

Memory usage:

1 SELECT metric_name, value, collecttime
2 FROM metric
3 WHERE metric_name = 'sys_memory_available' OR metric_name = 'sys_memory_used'
4 ORDER BY collecttime DESC;

2. Total number of transaction errors

1 SELECT metric_name, value, collecttime
2 FROM metric
3 WHERE metric_name = 'sql_transaction_errors' OR metric_name = 'sql_transaction_total'
4 ORDER BY collecttime DESC;

3. Total number of SQL errors:

1 SELECT metric_name, value, collecttime
2 FROM metric
3 WHERE metric_name = 'sql_statement_errors' OR metric_name = 'sql_statement_total'
4 ORDER BY collecttime DESC;

4. Click SAVE > Save dataset > SAVE & EXPLORE to save each query above and use it as the data source for subsequent charts.

5. Edit chart: Here we use one of the queries as an example to demonstrate how to edit a visual chart. First, we select the disk_read_write query as the data source for chart production. Click CREATE CHART below the corresponding query in SQL Lab or save the Query in the previous step. The page will jump to the Edit Chart page:

6. Enter the chart editing page, select the chart type, time field, query indicator column, query group column and other options. After the configuration is completed, select Run :

7. Click UPDATE CHART > SAVE to save the edited Chart.

Part 5 Organizational Dashboard

1. After creating multiple charts, you can assemble them in Superset to create a monitoring dashboard: Click Dashboards , then + DASHBOARD to create a new dashboard, or edit an existing dashboard.

2. On the dashboard editing page, you can drag the created chart from the CHARTS list on the right to the dashboard for assembly. You can also freely adjust the position of the chart, add titles, etc.

You have successfully connected the MatrixOne database with Superset and created a simple monitoring dashboard to visually display important indicators of the MatrixOne database.

About MatrixOne

MatrixOne is a multi-mode database based on cloud-native technology that can be deployed in both public and private clouds. This product uses an original technical architecture that separates storage and computing, separation of reading and writing, and separation of hot and cold. It can simultaneously support multiple loads such as transaction, analysis, flow, timing, and vector under a set of storage and computing systems, and can perform real-time and on-demand Isolated or shared storage and computing resources. The cloud-native database MatrixOne can help users significantly simplify the increasingly complex IT architecture and provide minimalist, extremely flexible, cost-effective and high-performance data services.

Since their release, MatrixOne Enterprise Edition and MatrixOne Cloud Services have been applied in many industries such as the Internet, finance, energy, manufacturing, education, and medical care. Thanks to its unique architectural design, users can reduce hardware and operation and maintenance costs by up to 70%, increase development efficiency by 3-5 times, and at the same time respond more flexibly to changes in market demand and seize innovation opportunities more efficiently. When investing in the same hardware, MatrixOne can achieve several times more performance improvements.

MatrixOne adheres to the concept of open source and ecological co-construction. All core codes are open source and fully compatible with the MySQL protocol. MatrixOne has created multiple end-to-end solutions with partners, significantly reducing users' migration and usage costs, and helping users avoid supply problems. Vendor lock-in risk.

MatrixOrigin official website: A new generation of hyper-converged heterogeneous open source database-MatrixOrigin (Shenzhen) Information Technology Co., Ltd. MatrixOne

Github 仓库：GitHub - matrixorigin/matrixone: Hyperconverged cloud-edge native database

Keywords: hyper-converged database, multi-mode database, cloud native database, domestic database.