设计初衷
- 时间序列数据库: TSM 树(Time-Structured Merge Tree)
- scheme-less
- 写优化(读多写少) : 不支持 CRUD, 对 CR 进行优化, 写按照时间递增, 写没有按照时间递增, 性能差
- 支持集群 : 写和读的优先级高于一致性.
- 有限 SQL : 不支持 join
支持接口
- 命令行 及 shell
- 基于 json 的 RestFul 接口: 查询, 写
- SDK : go, java, haskell, js, lisp, python, ruby, scala, …
- Web 管理端 : RestFul 与 命令行的结合. 方便调试
基本概念
- database:
- measurement:
- tag: tag set, tag keys, tag values
- fields: field set, field keys, field values
- retention_policy : replication, duration, shard duration 的标记
- replication : 集群备份节点个数
- duration : 保存时间, 1d, 10h, 7d
- shard duration: shard 时间范围. 1d, 10h, 7d
- Line protocol : 数据格式
weather,location=us-midwest temperature=82 1465839830100400200
| -------------------- -------------- |
| | | |
| | | |
+-----------+--------+-+---------+-+---------+-----------------
|measurement|,tag_set| |field_set| |timestamp (RFC3339 format)|
+-----------+--------+-+---------+-+---------+-----------------
SHOW DATABASES
SHOW RETENTION POLICIES ON <database_name>
SHOW SERIES [FROM <measurement_name> [WHERE <tag_key> [= ‘<tag_value>’ | =~ <regular_expression>]]]
SHOW MEASUREMENTS [WITH MEASUREMENT <regular_expression>] [WHERE <tag_key> [= ‘<tag_value>’ | =~ <regular_expression>]]
SHOW TAG KEYS [FROM <measurement_name>]
SHOW TAG VALUES [FROM <measurement_name>] [WITH KEY [ = “<tag_key>” | IN ("<tag_key1>","<tag_key2")]] [WHERE <tag_key> [= ‘<tag_value>’ | =~ <regular_expression>]]
SHOW FIELD KEYS [FROM <measurement_name>]
SHOW RETENTION POLICIES on_clause
SHOW SHARDS
SHOW SHARD GROUPS
例子
show retention policies on collectd
show series
show series where dest=‘10.2.1.40:80’
show measurements
show tag keys
show tag values
show tag values with key = host
show tag values with key = dest
show tag values with key in (dest, host) where dest =~ /./
show field keys
show grants
show queries
基本操作
写
CREATE DATABASE <database_name> [WITH [DURATION ] [REPLICATION ] [SHARD DURATION ] [NAME ]]
CREATE USER user_name “WITH PASSWORD” password [ “WITH ALL PRIVILEGES” ]
DROP DATABASE <database_name>
DROP SERIES FROM <measurement_name[,measurement_name]> WHERE <tag_key>=’<tag_value>’
DELETE FROM <measurement_name> WHERE [<tag_key>=’<tag_value>’] | [
DROP MEASUREMENT <measurement_name>
DROP USER
GRANT privilege [ on_clause ] to_clause .
SHOW QUERYS
KILL QUERY query_id
SHOW STATS
GRANT ALL TO “jdoe”
GRANT READ ON “mydb” TO “jdoe”
DROP SERIES FROM “h2o_feet”
DROP SERIES FROM “h2o_feet” WHERE “location” = ‘santa_monica’
REVOKE privilege [ on_clause ] “FROM” user_name .
REVOKE ALL PRIVILEGES FROM “jdoe”
DROP SERIES WHERE “location” = ‘santa_monica’
DELETE FROM “h2o_feet”
DELETE FROM “h2o_quality” WHERE “randtag” = ‘3’
DELETE WHERE time < ‘2016-01-01’
DROP SHARD <shard_id_number>
DROP SUBSCRIPTION
CREATE USER “jdoe” WITH PASSWORD ‘1337password’
CREATE USER “jdoe” WITH PASSWORD ‘1337password’ WITH ALL PRIVILEGES
INSERT weather,location=us-midwest temperature=82 1465839830100400200
weather,location=us-midwest temperature=82 1465839830100400200
weather,location=us-midwest temperature=82i 1465839830100400200
weather,location=us-midwest temperature="too warm" 1465839830100400200
weather,location=us-midwest too_hot=true 1465839830100400200
delete from paging,procstat,queue,swap,swap_usage,system,task where host = ‘NEWA5-01’
delete from abplus,bench,wrk_bench where host = ‘NEWA5-01’
支持类型; Integer, String, Boolean, Float
注:
- 时间戳不要用双引号
- fields value 不要用单引号, 如果不是 String 不要加双引号
- measurement names, tag keys, tag values 和 field keys 都不要加任何引号
查询
基本查询
SELECT <field_key>[,<field_key>,<tag_key>] FROM <database_name>.<retention_policy_name>.<measurement_name>
SELECT_clause <field_key>:: FROM_clause
use <database_name>
SELECT <field_key>[,<field_key>,<tag_key>] FROM <measurement_name>[,<measurement_name>]
SELECT <field_key>[,<field_key>,<tag_key>] FROM <database_name>…<measurement_name>
SELECT *::field FROM measurement
SELECT “<field_key>”::field,"<tag_key>"::tag
SELECT * FROM /.*/
SELECT “water_level”::float FROM “h2o_feet” LIMIT 4
SELECT “water_level”::integer FROM “h2o_feet” LIMIT 4
注: 多个语句通过分号分割, 方便对比
WHERE
SELECT_clause FROM_clause WHERE <conditional_expression> [(AND|OR) <conditional_expression> […]]
fields:
= equal to
<> not equal to
!= not equal to
> greater than
>= greater than or equal to
< less than
<= less than or equal to
数学运算
正则表达式(后面讨论)
tags:
= equal to
<> not equal to
!= not equal to
正则表达式(后面讨论)
select * from cpu_ where host = ‘A5-01’ limit 2
SELECT * FROM “h2o_feet” WHERE “water_level” > 8
SELECT * FROM “h2o_feet” WHERE “level description” = ‘below 3 feet’
SELECT * FROM “h2o_feet” WHERE “water_level” + 2 > 11.9
SELECT “water_level” FROM “h2o_feet” WHERE “location” <> ‘santa_monica’ AND (water_level < -0.59 OR water_level > 9.95)
SELECT mean(“temp”) FROM /.north$/
SELECT mean(“temp”) FROM “weather_sensor” WHERE “region” = ‘north’
SELECT * FROM “foodships” WHERE “planet” = ‘Saturn’
SELECT * FROM “foodships” WHERE “planet” = ‘Saturn’ AND time > ‘2015-04-16 12:00:01’
SELECT * FROM cpu_ WHERE time < now() - 1m and time > now() - 3m limit 10
SELECT * FROM “foodships” WHERE time > now() - 1h
u microseconds
s seconds
m minutes
h hours
d days
w weeks
注意: 数字不要用引号, 字符串用单引号
GROUP BY
SELECT_clause FROM_clause [WHERE_clause] GROUP BY [* | <tag_key>[,<tag_key]]
select * from cpu_ where host = ‘A5-01’ group by host limit 2
select * from cpu_ where host = ‘A5-01’ group by type_instance limit 2
select * from cpu_ group by host limit 2
select * from cpu_ group by host,instance limit 2
select * from cpu_ group by * limit 2
注: 如果 WHERE 存在, 在 WHERE 之后
SELECT (<field_key>) FROM_clause WHERE <time_range> GROUP BY time(<time_interval>),[tag_key] [fill(<fill_option>)]
select mean(tps) from bench where time > now() - 2d and time < now() - 1d and dest=‘10.2.1.41:80’ group by time(1h) limit 10
select mean(tps) from bench where time > now() - 2d and time < now() - 1d group by time(1h),dest
select mean(tps) from bench where time > now() - 2d and time < now() - 1d group by time(5h),dest
select mean(tps) from bench where time > now() - 2d and time < now() - 1d group by time(5h, 1h),dest
select mean(tps) from bench where time > now() - 2d and time < now() - 1d group by time(5h, -1h),dest
select mean(tps) from bench where time > now() - 2d and time < now() - 1d group by time(5h, -1h),dest fill(none)
select mean(tps) from bench where time > now() - 2d and time < now() - 1d group by time(5h, -1h),dest fill(0)
select mean(tps) from bench where time > now() - 2d and time < now() - 1d group by time(5h, -1h),dest fill(null)
select mean(tps) from bench where time > now() - 2d and time < now() - 1d group by time(5h, -1h),dest fill(previous)
INTO
SELECT_clause INTO <measurement_name> FROM_clause [WHERE_clause] [GROUP_BY_clause]
SELECT MEAN(“water_level”) INTO “all_my_averages” FROM “h2o_feet” WHERE “location” = ‘coyote_creek’ AND time >= ‘2015-08-18T00:00:00Z’ AND time <= ‘2015-08-18T00:30:00Z’ GROUP BY time(12m)
SELECT MEAN() INTO “where_else”.“autogen”.:MEASUREMENT FROM /./ WHERE time >= ‘2015-08-18T00:00:00Z’ AND time <= ‘2015-08-18T00:06:00Z’ GROUP BY time(12m)
其他
SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] LIMIT
SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] GROUP BY *[,time(<time_interval>)] [ORDER_BY_clause] SLIMIT
SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] GROUP BY *[,time(<time_interval>)] [ORDER_BY_clause] LIMIT SLIMIT
SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] LIMIT_clause OFFSET [SLIMIT_clause]
SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] GROUP BY *[,time(time_interval)] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] SLIMIT_clause SOFFSET
例子
select mean(tps) from bench where time > now() - 3d group by *,time(1h) limit 5
select mean(tps) from bench where time > now() - 3d group by *,time(1h) limit 5 offset 2
select mean(tps) from bench where time > now() - 3d group by *,time(1h) limit 5 offset 2 slimit 3
select mean(tps) from bench group by dest limit 2 slimit 2
select mean(tps) from bench group by dest limit 2 slimit 2
select mean(*) from bench group by dest slimit 3
select mean(tps) from bench group by dest slimit 1
select * from bench limit 10; select * from bench limit 10 offset 2
注: 可以将多个语句通过分号分割, 对比结果
时间范围
SELECT_clause FROM_clause WHERE time [’<rfc3339_date_time_string>’ | ‘<rfc3339_like_date_time_string>’ | <epoch_time>] [AND [’<rfc3339_date_time_string>’ | ‘<rfc3339_like_date_time_string>’ | <epoch_time>] […]]
RFC3339 date time format: ‘YYYY-MM-DDTHH:MM:SS.nnnnnnnnnZ’ 其中 .nnnnnnnnn 部分可选
rfc3339_like_date_time_string : ‘YYYY-MM-DD HH:MM:SS.nnnnnnnnn’ 其中 HH:MM:SS.nnnnnnnnn 部分可选
= equal to
<> not equal to
!= not equal to
> greater than
>= greater than or equal to
< less than
<= less than or equal to
SELECT_clause FROM_clause WHERE time now() [[ - | + ] <duration_literal>] [(AND|OR) now() […]]
u microseconds
ms milliseconds
s seconds
m minutes
h hours
d days
w weeks
注: 1. 优先级 WHERE > GROUP BY > ORDER BY time DESC > LIMIT
2. 如果 SLIMIT 必须 GROUP BY 一起, SOFFSET 必须与 SLIMIT 一起
例子
select mean(tps) from bench where time > ‘2015-11-20T00:00:00.000000000Z’ group by *,time(1h) limit 5
select mean(tps) from bench where time > ‘2015-11-20’ group by *,time(1h) limit 5
select mean(tps) from bench where time > ‘2015-11-20 12:00:00’ group by *,time(1h) limit 5
select mean(tps) from bench where time > ‘2015-11-20T12:00:00Z’ group by *,time(1h) limit 5
select tps from bench where time > ‘2016-11-21T03:00:00Z’ limit 10
select tps from bench where time > ‘2016-11-21T03:00:00Z’ + 44m limit 10
select tps from bench where time > ‘2016-11-21T03:00:00Z’ + 1h limit 10
select tps from bench where time > ‘2016-11-21T03:00:00Z’ - 1h limit 10
正则表达式
SELECT_clause FROM /<regular_expression_measurement>/ WHERE [<tag_key> /<regular_expression_tag_value>/ | <field_key> /<regular_expression_field_value>/]
=~ 匹配
!~ 不匹配
- FROM 正则匹配 measurement
- WHERE 正则匹配 tag, fields
select * from /.*/
select * from bench where dest =~ /10.2.1.4.*/ limit 10
select * from bench where dest =~ /./ limit 10
高级
函数
对数据进行一些简单计算, 如 最大, 最小, 均值, 目前支持操作
SELECT FUNCTION(<field_key>) FROM <measurement_name> [WHERE ] [GROUP BY ]
- COUNT : 计数
- DISTINCT: 不同值的列表. 类型 linux 命令 uniq
- MEAN : 均值
- MEDIAN : 中值
- MODE : 次数最多的值
- SPREAD : 最大值和最小值的差值
- SUM : 求和
- STDDEV : 标准偏差
- BOTTOM : 最小的 N 个值; BOTTOM(<field_key>[,<tag_keys>],)[,<tag_keys>]
- FIRST : 最旧的值; FIRST(<field_key>)[,<tag_key(s)>]
- LAST : 最旧的值; LAST(<field_key>)[,<tag_key(s)>]
- MAX : 最大值; MAX(<field_key>)[,<tag_key(s)>]
- MIN : 最小值; MIN(<field_key>)[,<tag_key(s)>]
- PERCENTILE: 返回大于总数百分比的值; PERCENTILE(<field_key>, )[,<tag_key(s)>]
- TOP : 返回最大的 N 个值; TOP(<field_key>[,<tag_keys>],)
- CEILING : 还没有实现
- DERIVATIVE : 计算斜率, 差值/(相邻两个值时间间隔/时间间隔); DERIVATIVE(<field_key>, [])
- DIFFERENCE : 计算相邻两个值的差值; DIFFERENCE(<field_key>), DIFFERENCE((<field_key>))
- ELAPSED : 上一个值与当前值的时间差值; ELAPSED(<field_key>, )
- FLOOR : 还未实现
- HISTOGRAM : 还未实现
- MOVING_AVERAGE : 前 window 个值的均值为当前值; MOVING_AVERAGE(<field_key>,), MOVING_AVERAGE((<field_key>),)
- NON_NEGATIVE_DERIVATIVE: DERIVATIVE 去掉负值; NON_NEGATIVE_DERIVATIVE(<field_key>, []), NON_NEGATIVE_DERIVATIVE(AGGREGATION_FUNCTION(<field_key>),[])
- HOLT_WINTERS: 根据以前预测未来 N 个值; HOLT_WINTERS(FUNCTION(<field_key>),,< S >)
- HOLT_WINTERS_WITH_FIT : 根据以前预测未来 N 个值; HOLT_WINTERS_WITH_FIT(FUNCTION(< field_key >),< N >,< S >)
select count(*) from bench
select distinct(concurreny) from bench
select count(distinct(concurreny)) from bench
select min(tps),max(tps),median(tps),count(tps) from bench where dest =~ /./ limit 10 一个查询可以指定多个函数
select median(tps) from bench where time > now() - 4d
select bottom(tps, 3) from bench where time > now() - 4d
select bottom(tps, 3),concurreny from bench where time > now() - 4d
select bottom(tps, 3),* from bench where time > now() - 4d
select bottom(tps, concurreny, 3) from bench where time > now() - 4d
select bottom(tps, dest, 3) from bench where time > now() - 4d : 每个 dest 的最小值.
select bottom(tps, dest, 4) from bench where time > now() - 4d : 每个 dest 的最小值.
select percentile(tps, 100) from bench where dest =~ /./ group by dest
select top(tps, 3) from bench where dest =~ /./ group by dest
select top(tps, dest, 3) from bench where dest =~ /./
select (tps) from bench where dest =~ /./ limit 10 ; select DIFFERENCE(tps) from bench where dest =~ /./ limit 10
select ELAPSED(tps, 1s) from bench where dest =~ /./ limit 10
select tps from bench where dest =~ /./ limit 10; select MOVING_AVERAGE(tps,2) from bench where dest =~ /./ limit 10
select tps from bench where dest =~ /./ limit 10; select DERIVATIVE(tps,2m) from bench where dest =~ /./ limit 10
select tps from bench where dest =~ /./ limit 10; select NON_NEGATIVE_DERIVATIVE(tps,2m) from bench where dest =~ /./ limit 10
select tps as throughput from bench where dest =~ /./ limit 10
select HOLT_WINTERS(first(tps), 5, 1) as throughput from bench where time > now() - 3d group by time(1h)
注: limit 不起作用
备份策略
CREATE RETENTION POLICY <retention_policy_name> ON <database_name> DURATION REPLICATION [SHARD DURATION ] [DEFAULT]
ALTER RETENTION POLICY <retention_policy_name> ON <database_name> DURATION REPLICATION SHARD DURATION DEFAULT
DURATION
- m minutes
- h hours
- d days
- w weeks
- INF infinite
SHARD DURATION
- u microseconds
- ms milliseconds
- s seconds
- m minutes
- h hours
- d days
- w weeks
例子
CREATE RETENTION POLICY “one_day_only” ON “NOAA_water_database” DURATION 1d REPLICATION 1 DEFAULT
CREATE RETENTION POLICY “one_day_only” ON “NOAA_water_database” DURATION 1d REPLICATION 1
CREATE RETENTION POLICY “what_is_time” ON “NOAA_water_database” DURATION 2d REPLICATION 1
ALTER RETENTION POLICY “what_is_time” ON “NOAA_water_database” DURATION 3w SHARD DURATION 30m DEFAULT
DROP RETENTION POLICY <retention_policy_name> ON <database_name>
DROP RETENTION POLICY “what_is_time” ON “NOAA_water_database”
Continuous Queries
SHOW CONTINUOUS QUERIES
DROP CONTINUOUS QUERY <cq_name> ON <database_name>
DROP CONTINUOUS QUERY “idle_hands” ON “telegraf”`
CREATE CONTINUOUS QUERY <cq_name> ON <database_name>
BEGIN
<cq_query>
END
CREATE CONTINUOUS QUERY <cq_name> ON <database_name>
RESAMPLE EVERY FOR
BEGIN
<cq_query>
END
cq_query : SELECT <function[s]> INTO <destination_measurement> FROM [WHERE ] GROUP BY time()[,<tag_key[s]>]
例子
CREATE CONTINUOUS QUERY “cq_basic” ON “transportation”
BEGIN
SELECT mean(“passengers”) INTO “average_passengers” FROM “bus_data” GROUP BY time(1h)
END
CREATE CONTINUOUS QUERY “cq_basic_rp” ON “transportation”
BEGIN
SELECT mean(“passengers”) INTO “transportation”.“three_weeks”.“average_passengers” FROM “bus_data” GROUP BY time(1h)
END
CREATE CONTINUOUS QUERY “cq_basic_br” ON “transportation”
BEGIN
SELECT mean() INTO “downsampled_transportation”.“autogen”.:MEASUREMENT FROM /./ GROUP BY time(30m),*
END
CREATE CONTINUOUS QUERY “cq_basic_offset” ON “transportation”
BEGIN
SELECT mean(“passengers”) INTO “average_passengers” FROM “bus_data” GROUP BY time(1h,15m)
END
- 每半小时计算一次当前一小时的均值, 后面会覆盖前面
CREATE CONTINUOUS QUERY “cq_advanced_every” ON “transportation”
RESAMPLE EVERY 30m
BEGIN
SELECT mean(“passengers”) INTO “average_passengers” FROM “bus_data” GROUP BY time(1h)
END
- 计算过去一小时每半小时的平均值
CREATE CONTINUOUS QUERY “cq_advanced_for” ON “transportation”
RESAMPLE FOR 1h
BEGIN
SELECT mean(“passengers”) INTO “average_passengers” FROM “bus_data” GROUP BY time(30m)
END
- 每一小时执行一次, 计算过去一小时半, 每半小时的平均值
CREATE CONTINUOUS QUERY “cq_advanced_every_for” ON “transportation”
RESAMPLE EVERY 1h FOR 90m
BEGIN
SELECT mean(“passengers”) INTO “average_passengers” FROM “bus_data” GROUP BY time(30m)
END
其中:
- EVERY 表示每多长时间执行一次; 没有给定, 以 group by 为准, 如果比 group by 大, 并且时间跨度也以当前为准.
- FOR 表示计算多大时间跨度
- group by 表示, 时间跨度以多长时间为一份
使用场景
- 降低时间采样间隔
- 预计算, 将常用操作分散执行,降低系统压力
- 替代 hava 语句
- 替代 嵌套函数
备份与恢复
最直接的备份策略就是将 influxd 配置文件中数据路径指定的整个目录拷贝到一个新的地方。
修改配置文件路径即可。
influxd backup
influxd backup -database
influxd backup -database telegraf -retention autogen -since 2016-02-01T00:00:00Z /tmp/backup
influxd backup -database mydatabase -host 10.0.0.1:8088 /tmp/mysnapshot
influxd restore [ -metadir | -datadir ]
influxd restore -metadir /var/lib/influxdb/meta /tmp/backup
influxd restore -database telegraf -datadir /var/lib/influxdb/data /tmp/backup
存储引擎
监控数据的特点:
- 大量的基于时间数据
- 写吞吐量要求高
- 读吞吐量要求高
- 固定时间大量删除操作(数据过期)
- 大多数是写和追加
其他存储引擎(LevelDB, RocksDB, HyperLevelDB, and LMDB, BoltDB)目前不能满足需求:
- 动态备份(RocksDB, HyperLevelDB 可以支持)
- 定期大量删除
- LevelDB 单一进程 file handler 到达极限
- BoltDB : 虚拟环境 IOPS 销毁太高
基于以上原因,专门针对时间序列的数据库 TLSM 诞生
TSM
- 创建: 同时到 cache 和 WAL
- 更新:
- 删除: 写一个删除操作到 WAL, 之后更新 Cache 和 FileStore. 其中 Cache 过期对应的 entries, FileStore 标记对应的文件为 tombstone
- 读: 每个块都包含时间范围, 首先找到对应时间范围的块, 之后, 二分查找
WAL : 固定大小的块. 当写满固定大小后, 落盘, 创建新的块. 保存 delete, write 的记录, 每条记录是 TLV 标准
Cache : WAL 的缓存, 方便查询, 当到达一定大小后, snap 到 TSM; 如果写的速度过快, 将拒绝写. 也支持, 多长时间没有新的写, 就 snap.
TSM File: mmap 的只读的压缩的形式存储数据. 类似 SSTable, LSM Tree Variants
FileStore: 管理 TSM File, 增加或删除 TSM File
Compactor: 将对读没有优化的数据转为读优化的数据; , 多级压缩
Compaction Planer: 决定哪些 TSM 文件可以压缩, 确保并行压缩成为可能
Writers/Readers : WAL, TSM, tombstone 等组件提供的读写接口
TSM 文件数据格式
┌────────┬────────────────────────────────────┬─────────────┬──────────────┐
│ Header │ Blocks │ Index │ Footer │
│5 bytes │ N bytes │ N bytes │ 4 bytes │
└────────┴────────────────────────────────────┴─────────────┴──────────────┘
┌───────────────────┐
│ Header │
├─────────┬─────────┤
│ Magic │ Version │
│ 4 bytes │ 1 byte │
└─────────┴─────────┘
┌───────────────────────────────────────────────────────────┐
│ Blocks │
├───────────────────┬───────────────────┬───────────────────┤
│ Block 1 │ Block 2 │ Block N │
├─────────┬─────────┼─────────┬─────────┼─────────┬─────────┤
│ CRC │ Data │ CRC │ Data │ CRC │ Data │
│ 4 bytes │ N bytes │ 4 bytes │ N bytes │ 4 bytes │ N bytes │
└─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘
┌────────────────────────────────────────────────────────────────────────────┐
│ Index │
├─────────┬─────────┬──────┬───────┬─────────┬─────────┬────────┬────────┬───┤
│ Key Len │ Key │ Type │ Count │Min Time │Max Time │ Offset │ Size │...│
│ 2 bytes │ N bytes │1 byte│2 bytes│ 8 bytes │ 8 bytes │8 bytes │4 bytes │ │
└─────────┴─────────┴──────┴───────┴─────────┴─────────┴────────┴────────┴───┘
┌─────────┐
│ Footer │
├─────────┤
│Index Ofs│
│ 8 bytes │
└─────────┘
每个 block 是压缩过的. 格式如下
┌───────┬─────┬─────────────────┬──────────────────┐
│ Type │ Len │ Timestamps │ Values │
│1 Byte │VByte│ N Bytes │ N Bytes │
└───────┴─────┴─────────────────┴──────────────────┘
集群与高可用
商业版本支持. 高可用有一个基本的版本
参考
全部 SQL 语句: https://docs.influxdata.com/influxdb/v1.0/query_language/spec/
附录
与展示层对接
- grafana
与数据源对接
- collectd
- telegraf
- graphite
- openTSDB
使用建议
- tag 优于 fields : tag 都被索引, 而 fields 没有
- 将长的 tag 拆分 : blueberries.plot-1.north 拆分为 crop=blueberries,plot=1,region=north
- tag 根据 key 排序
问题
[httpd] 10.9.1.13 - - [22/Jan/2017:10:27:20 +0800] “POST /write?consistency=any&db=telegraf&precision=ns&rp=autogen HTTP/1.1” 204 0 “-” “InfluxDBClient” 4d035372-e04a-11e6-8287-000000000000 89343
[tsm1] 2017/01/22 10:27:21 Snapshot for path /repo/influxdb/data/telegraf/autogen/103 written in 26.194993ms
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x0]
goroutine 2289614 [running]:
panic(0xba8be0, 0xc82000e090)
/usr/local/go/src/runtime/panic.go:481 +0x3e6
github.com/influxdata/influxdb/tsdb/engine/tsm1.Values.Size(0xc838bb4000, 0x150, 0x200, 0xc847639ca0)
/root/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/encoding.gen.go:28 +0x88
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Cache).Snapshot(0xc826707680, 0x0, 0x0, 0x0)
/root/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/cache.go:276 +0x369
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).WriteSnapshot.func2(0xc8202110a0, 0xc839e49e28, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/root/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:732 +0x245
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).WriteSnapshot(0xc8202110a0, 0x0, 0x0)
/root/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:738 +0x8d
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactCache(0xc8202110a0)
/root/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:814 +0x25d
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).SetCompactionsEnabled
/root/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:166 +0x13f
[run] 2017/01/22 10:34:17 InfluxDB starting, version 1.0.2, branch master, commit ff307047057b7797418998a4ed709b0c0f346324
[run] 2017/01/22 10:34:17 Go version go1.6.2, GOMAXPROCS set to 8
[run] 2017/01/22 10:34:17 Using configuration at: /etc/influxdb/influxdb.conf