1 Overview
Reprinted: Clickhouse Live View
Live view 是一种特殊的视图，不属于表引擎，作用类似于事件监听器，能够将一条SQL查询结果作为监控目标，当目标增加 Live view 可以及时作出响应。
In the field of data analysis, analysts often use moving average (MA). Moving averages help smooth data series and determine long-term trends. The new Live View function
began to bring 实时计算功能(Real-time) to ClickHouse .
One of the applications of the Live View table is real-time calculation of real-time indicators. Examples of event data streams include data from IoT sensors, price fluctuations on stock exchanges or certain monitoring indicators from production servers.
ClickHouse can store all these data with a good compression rate and provide excellent analytical query performance
Version evolution:
19.14 版本提供 Live view 视图
20.1.2.4 版本提供实时移动平均功能
Related parameters:
Clickhouse> select name ,value,changed,min,max,readonly,type from system.settings where name like '%live_view%';
SELECT
name,
value,
changed,
min,
max,
readonly,
type
FROM system.settings
WHERE name LIKE '%live_view%'
┌─name───────────────────────────────────────┬─value─┬─changed─┬─min──┬─max──┬─readonly─┬─type───────────┐
│ allow_experimental_live_view │ 0 │ 0 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 0 │ SettingBool │
│ live_view_heartbeat_interval │ 15 │ 0 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 0 │ SettingSeconds │
│ max_live_view_insert_blocks_before_refresh │ 64 │ 0 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 0 │ SettingUInt64 │
│ temporary_live_view_timeout │ 5 │ 0 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 0 │ SettingSeconds │
└────────────────────────────────────────────┴───────┴─────────┴──────┴──────┴──────────┴────────────────┘
4 rows in set. Elapsed: 0.018 sec.
Live view still 实验功能(experimental feature) needs to turn on the parameters allow_experimental_live_view:
manual setting:
Clickhouse> set allow_experimental_live_view=1;
SET allow_experimental_live_view = 1
Ok.
0 rows in set. Elapsed: 0.001 sec.
Clickhouse> select name ,value,changed,min,max,readonly,type from system.settings where name like '%live_view%';
┌─name───────────────────────────────────────┬─value─┬─changed─┬─min──┬─max──┬─readonly─┬─type───────────┐
│ allow_experimental_live_view │ 1 │ 1 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 0 │ SettingBool │
.........
└────────────────────────────────────────────┴───────┴─────────┴──────┴──────┴──────────┴────────────────┘
4 rows in set. Elapsed: 0.006 sec.
Simple example:
Clickhouse> create table t_lv(id UInt64)engine=Log;
CREATE TABLE t_lv
(
`id` UInt64
)
ENGINE = Log
Ok.
0 rows in set. Elapsed: 0.011 sec.
Clickhouse> create live view lv_count as select count(1) from t_lv;
CREATE LIVE VIEW lv_count AS
SELECT count(1)
FROM t_lv
Ok.
0 rows in set. Elapsed: 0.015 sec.
Clickhouse> watch lv_count;
WATCH lv_count
┌─count(1)─┬─_version─┐
│ 0 │ 1 │
└──────────┴──────────┘
Live view enters monitoring mode. In another client can perform the operation of inserting data:
Clickhouse> insert into t_lv select rand() from numbers(10);
┌─count(1)─┬─_version─┐
│ 10 │ 2 │
└──────────┴──────────┘
Example of moving average calculation:
Clickhouse> create table t_events(id UInt64,create_time datetime default now())ENGINE = Memory;
Clickhouse> select sum(id)/10 from(select * from t_events order by create_time desc limit 10);
Insert test data:
Clickhouse> INSERT INTO t_events VALUES (1,now()-9), (2,now()-8), (3,now()-7), (4,now()-6), (5,now()-5), (6,now()-4), (7,now()-3), (8,now()-2), (9,now()-1), (10,now());
Calculate the average:
Clickhouse> select sum(id)/10 from(select * from t_events order by create_time desc limit 10);
SELECT sum(id) / 10
FROM
(
SELECT *
FROM t_events
ORDER BY create_time DESC
LIMIT 10
)
┌─divide(sum(id), 10)─┐
│ 5.5 │
└─────────────────────┘
1 rows in set. Elapsed: 0.013 sec.
Clickhouse> CREATE LIVE VIEW latest_10 as select sum(id)/10 from(select * from t_events order by create_time desc limit 10);
Clickhouse> select * from latest_10;
SELECT *
FROM latest_10
┌─divide(sum(id), 10)─┐
│ 5.5 │
└─────────────────────┘
1 rows in set. Elapsed: 0.007 sec.
— Insert a new record to view the average of the last 10 data:
Clickhouse> select * from t_events;
SELECT *
FROM t_events
┌─id─┬─────────create_time─┐
│ 1 │ 2020-06-15 15:17:21 │
│ 2 │ 2020-06-15 15:17:22 │
│ 3 │ 2020-06-15 15:17:23 │
│ 4 │ 2020-06-15 15:17:24 │
│ 5 │ 2020-06-15 15:17:25 │
│ 6 │ 2020-06-15 15:17:26 │
│ 7 │ 2020-06-15 15:17:27 │
│ 8 │ 2020-06-15 15:17:28 │
│ 9 │ 2020-06-15 15:17:29 │
│ 10 │ 2020-06-15 15:17:30 │
└────┴─────────────────────┘
┌─id─┬─────────create_time─┐
│ 11 │ 2020-06-15 15:21:17 │
└────┴─────────────────────┘
11 rows in set. Elapsed: 0.004 sec.
Clickhouse> select * from latest_10;
SELECT *
FROM latest_10
┌─divide(sum(id), 10)─┐
│ 6.5 │
└─────────────────────┘
1 rows in set. Elapsed: 0.008 sec.
sum([2, 3, 4, 5, 6, 7, 8, 9, 10, 1])/10 = 5.5
sum([2, 3, 4, 5, 6, 7, 8, 9, 10, 11])/10 = 6.5