Grafana+TDengine realizes the display of data collected by the Internet of Things

First come two official documents:
https://www.taosdata.com/cn/documentation/connections-with-other-tools/#Grafana
https://grafana.com/docs/grafana/latest/

grafana installation

There are many ways to install, here is a more convenient way, if you want to try other ways, please refer to the official website

1. download

wget https://dl.grafana.com/oss/release/grafana-7.3.5.linux-amd64.tar.gz

2. Unzip

tar -xzvf grafana-7.3.5.linux-amd64.tar.gz

3. run

cd grafana-7.3.5/
cd bin
./grafana-server 

So far I have successfully run a grafana, open the browser, and visit http port 3000, http://192.168.137.164:3000. The
default user name: admin, and the password is also: admin.

Well, the interface still feels cool.

TDengine installation

1. Download
   https://www.taosdata.com/download/download.php?key=$2y $10$lTACLBcR68nBkt5.Ki90nuWS9/e3HAHJ0MlpU6XVp0RCN2Tc9G3Hi&[email protected]&pkg=tdengine_tar&pkgIndex=-
   2.解压

tar -zxvf TDengine-server-2.0.9.0-Linux-x64.tar.gz

3. Installation

Execute the install.sh script in the decompressed directory

sudo ./install.sh

By the way, when uninstalling, execute: rmtaos this command will do

So far TDengine has been installed, we can use the taos command to enter the database for operation (the client is installed by default when the server is installed)

Install TDengine plugin in grafana

1. Copy the entire grafana plugin directory in the TDengine installation package directory to the grafana plugin directory

cd /opt/grafana/grafana-7.3.5/data/plugins
cp -r /opt/taos/TDengine-server-2.0.9.0/connector/grafanaplugin/ ./TDengine

The following are renamed here to facilitate the installation of other plug-ins in the future, so as to distinguish where the
specific plug-ins are copied to, you can see the configuration file (default: conf/default.ini) to specify.
Or specified in the startup parameters, for example:
./grafana-server --homepath=/usr/share/grafana --config=/etc/grafana/grafana.ini --packaging=docker cfg:default.log.mode=console cfg :default.paths.data=/var/lib/grafana This specifies that the data directory is /var/lib/grafana, then the plugin should be copied to the directory /var/lib/grafana/plugins

2. Restart the grafana service
   Ctrl + c to stop (or kill the grafana process), and then use the command ./grafana-server to start

3. Configure grafana's data source as TDengine

Click the Settings button in the grafana web interface to select Data Sources

Click Add data source to add a data source

Find TDengine, click select

Enter host: http://localhost:6041, here you need to pay attention: the port is 6041 instead of the 6030 used by the client to connect. The
default user and password is fine.
Click Save & Test to save.

So far, the grafana TDengine data source is configured

Create a table in TDengine and write a few pieces of data

1. Connect to TDengine with the client
2. Create database

CREATE DATABASE power KEEP 365 DAYS 10 BLOCKS 4 UPDATE 1;

Create a library named power, the data in this library will be retained for 365 days (it will be automatically deleted if more than 365 days), a data file every 10 days, the number of memory blocks is 4, allowing data to be updated

3. Switch to the library just created

USE power;  

4. Create super table

CREATE TABLE sensor (ts timestamp, temp float, humi float) TAGS (location binary(64), groupdId int);

The first column must be a timestamp (ts in the example), the other columns are the collected physical quantities (temp, humi in the example), and the data type can be integer, floating point, string, etc.
The super table defines a type of physical quantity collected by the terminal, where we collect temperature temp and humidity humi

5. Create table

CREATE TABLE d1001 USING sensor TAGS ("Beijing.Chaoyang", 2);

Among them, d1001 is the table name, sensor is the table name of the super table, followed by the specific tag value "Beijing.Chaoyang" of the tag Location, and the specific tag value of the tag groupId 2.
One table corresponds to one terminal, one table stores the data collected by one terminal, and uses tags to identify different terminals

6. Insert a few pieces of data

insert into d1001 values (now,12.01,34);

Very simple, basically consistent with commonly used SQL

Create a dashboard in grafana to display data

Click the plus sign and Create

Click Add new panel

Select the data source, here select the TDengine data source just added

Enter the data query statement
select ts,temp from power.d1001 where ts >= $from and ts < $to
here. Note that here must be the query time and the physical quantity to be displayed (ie ts, temp), which is the display temperature.
from, to and interval are built-in variables of the TDengine plugin, which represent the query range and time interval obtained from the Grafana plugin panel

Set the name and unit of the physical quantity display here

Here set the time range of the displayed data, refresh the data, and set the time interval for automatic refresh

So far, we have displayed the data stored in TDengine with grafana.

Data collection and storage

TDengine provides a variety of SDKs. We can use these SDKs to store the data collected by the terminal in the corresponding data table of TDengine. Here we use the RESTful Connector to store the data to TDengine.
Official document: https://www.taosdata.com/cn/documentation/connector/#RESTful-Connector
first look at the code:

func InsertToDtEngine(temp interface{
    
    }, humi float32) (string, error) {
    
    
	//http://192.168.137.164:6041/rest/sql
	//insert into power.d1001 values (now,12.4,0);

	sqltr:="insert into power.d1001 values (now,"+fmt.Sprint(temp)+","+fmt.Sprint(humi)+");"

	logrus.Debug(sqltr)
	req, err := http.NewRequest("POST", "http://192.168.137.164:6041/rest/sql", bytes.NewBuffer([]byte(sqltr)))
	if err != nil {
    
    
		logrus.Error(err)
		return "", err
	}
	req.SetBasicAuth("root","taosdata")
	req.Close = true
	cli := createCli()
	resp, er := cli.Do(req)
	if er != nil {
    
    
		logrus.Error(er)
		return "", er
	}
	if resp.StatusCode < 200 || resp.StatusCode >=300  {
    
    
		logrus.Error(fmt.Sprintf("statusCode:%v",resp.StatusCode))
		return "",errors.New(fmt.Sprintf("statusCode:%v",resp.StatusCode))
	}
	defer resp.Body.Close()
	respBytes, errr := ioutil.ReadAll(resp.Body)
	if errr != nil {
    
    
		logrus.Error(errr)
		return "", errr
	}
	str := string(respBytes)
	logrus.Debugf(fmt.Sprintf("Post url[%v] -> req[%v] -> rsp[%v]","http://192.168.137.164:6041/rest/sql", sqltr, str))
	return str, nil

}

This method uses the relevant tools in the http package of go to call the restful interface of TDengine.
sqltr:="insert into power.d1001 values ​​(now,"+fmt.Sprint(temp)+","+fmt.Sprint(humi)+");" This is the sql for inserting data, you need to pay attention to the table name Add the database name
req.SetBasicAuth("root","taosdata") This means using Basic authentication
http://192.168.137.164:6041/rest/sql This is the restful interface for submitting SQL statements

Finally, let's simulate the data collected in real time:

r := rand.New(rand.NewSource(time.Now().UnixNano()))
	for {
    
    
		enthandler.InsertToDtEngine(r.Intn(100),4.5)//测试
		time.Sleep(1 * time.Second)
	}

Well, at this point, we can see a display of the physical quantity of the device collected in real time on the grafana webpage.

to sum up

1. The grafana plugin provided by TDengine must be copied to the correct grafana directory.
2. When adding a dashboard, you must pay attention to the results of the query when writing sql, that is, the time and the physical quantity to be displayed.
3. Here is just a simple demo, more features need to be explored.