influxdb 介绍

版权声明:本文系原创,转发请注明出处,商业用途联系作者 https://blog.csdn.net/wenxueliu/article/details/83099443

设计初衷

  • 时间序列数据库: TSM 树(Time-Structured Merge Tree)
  • scheme-less
  • 写优化(读多写少) : 不支持 CRUD, 对 CR 进行优化, 写按照时间递增, 写没有按照时间递增, 性能差
  • 支持集群 : 写和读的优先级高于一致性.
  • 有限 SQL : 不支持 join

支持接口

  1. 命令行 及 shell
  2. 基于 json 的 RestFul 接口: 查询, 写
  3. SDK : go, java, haskell, js, lisp, python, ruby, scala, …
  4. Web 管理端 : RestFul 与 命令行的结合. 方便调试

基本概念

  • database:
  • measurement:
  • tag: tag set, tag keys, tag values
  • fields: field set, field keys, field values
  • retention_policy : replication, duration, shard duration 的标记
  • replication : 集群备份节点个数
  • duration : 保存时间, 1d, 10h, 7d
  • shard duration: shard 时间范围. 1d, 10h, 7d
  • Line protocol : 数据格式

    weather,location=us-midwest temperature=82 1465839830100400200
      |    -------------------- --------------  |
      |             |             |             |
      |             |             |             |
    +-----------+--------+-+---------+-+---------+-----------------
    |measurement|,tag_set| |field_set| |timestamp (RFC3339 format)|
    +-----------+--------+-+---------+-+---------+-----------------

SHOW DATABASES

SHOW RETENTION POLICIES ON <database_name>

SHOW SERIES [FROM <measurement_name> [WHERE <tag_key> [= ‘<tag_value>’ | =~ <regular_expression>]]]

SHOW MEASUREMENTS [WITH MEASUREMENT <regular_expression>] [WHERE <tag_key> [= ‘<tag_value>’ | =~ <regular_expression>]]

SHOW TAG KEYS [FROM <measurement_name>]

SHOW TAG VALUES [FROM <measurement_name>] [WITH KEY [ = “<tag_key>” | IN ("<tag_key1>","<tag_key2")]] [WHERE <tag_key> [= ‘<tag_value>’ | =~ <regular_expression>]]

SHOW FIELD KEYS [FROM <measurement_name>]

SHOW RETENTION POLICIES on_clause

SHOW SHARDS

SHOW SHARD GROUPS


例子

show retention policies on collectd

show series

show series where dest=‘10.2.1.40:80’

show measurements

show tag keys

show tag values

show tag values with key = host

show tag values with key = dest

show tag values with key in (dest, host) where dest =~ /./

show field keys

show grants

show queries


基本操作

CREATE DATABASE <database_name> [WITH [DURATION ] [REPLICATION ] [SHARD DURATION ] [NAME ]]

CREATE USER user_name “WITH PASSWORD” password [ “WITH ALL PRIVILEGES” ]

DROP DATABASE <database_name>

DROP SERIES FROM <measurement_name[,measurement_name]> WHERE <tag_key>=’<tag_value>’

DELETE FROM <measurement_name> WHERE [<tag_key>=’<tag_value>’] | [

DROP MEASUREMENT <measurement_name>

DROP USER

GRANT privilege [ on_clause ] to_clause .

SHOW QUERYS

KILL QUERY query_id

SHOW STATS

GRANT ALL TO “jdoe”

GRANT READ ON “mydb” TO “jdoe”

DROP SERIES FROM “h2o_feet”

DROP SERIES FROM “h2o_feet” WHERE “location” = ‘santa_monica’

REVOKE privilege [ on_clause ] “FROM” user_name .

REVOKE ALL PRIVILEGES FROM “jdoe”

DROP SERIES WHERE “location” = ‘santa_monica’

DELETE FROM “h2o_feet”

DELETE FROM “h2o_quality” WHERE “randtag” = ‘3’

DELETE WHERE time < ‘2016-01-01’

DROP SHARD <shard_id_number>

DROP SUBSCRIPTION

CREATE USER “jdoe” WITH PASSWORD ‘1337password’

CREATE USER “jdoe” WITH PASSWORD ‘1337password’ WITH ALL PRIVILEGES

INSERT weather,location=us-midwest temperature=82 1465839830100400200

weather,location=us-midwest temperature=82 1465839830100400200
weather,location=us-midwest temperature=82i 1465839830100400200
weather,location=us-midwest temperature="too warm" 1465839830100400200
weather,location=us-midwest too_hot=true 1465839830100400200

delete from paging,procstat,queue,swap,swap_usage,system,task where host = ‘NEWA5-01’
delete from abplus,bench,wrk_bench where host = ‘NEWA5-01’

支持类型; Integer, String, Boolean, Float

注:

  • 时间戳不要用双引号
  • fields value 不要用单引号, 如果不是 String 不要加双引号
  • measurement names, tag keys, tag values 和 field keys 都不要加任何引号

查询

基本查询

SELECT <field_key>[,<field_key>,<tag_key>] FROM <database_name>.<retention_policy_name>.<measurement_name>

SELECT_clause <field_key>:: FROM_clause

use <database_name>

SELECT <field_key>[,<field_key>,<tag_key>] FROM <measurement_name>[,<measurement_name>]

SELECT <field_key>[,<field_key>,<tag_key>] FROM <database_name>…<measurement_name>

SELECT *::field FROM measurement

SELECT “<field_key>”::field,"<tag_key>"::tag

SELECT * FROM /.*/

SELECT “water_level”::float FROM “h2o_feet” LIMIT 4

SELECT “water_level”::integer FROM “h2o_feet” LIMIT 4

注: 多个语句通过分号分割, 方便对比


WHERE

SELECT_clause FROM_clause WHERE <conditional_expression> [(AND|OR) <conditional_expression> […]]

fields:

=   equal to
<> not equal to
!= not equal to
>   greater than
>= greater than or equal to
<   less than
<= less than or equal to
数学运算
正则表达式(后面讨论)

tags:

=   equal to
<> not equal to
!= not equal to
正则表达式(后面讨论)

select * from cpu_ where host = ‘A5-01’ limit 2

SELECT * FROM “h2o_feet” WHERE “water_level” > 8

SELECT * FROM “h2o_feet” WHERE “level description” = ‘below 3 feet’

SELECT * FROM “h2o_feet” WHERE “water_level” + 2 > 11.9

SELECT “water_level” FROM “h2o_feet” WHERE “location” <> ‘santa_monica’ AND (water_level < -0.59 OR water_level > 9.95)

SELECT mean(“temp”) FROM /.north$/

SELECT mean(“temp”) FROM “weather_sensor” WHERE “region” = ‘north’

SELECT * FROM “foodships” WHERE “planet” = ‘Saturn’

SELECT * FROM “foodships” WHERE “planet” = ‘Saturn’ AND time > ‘2015-04-16 12:00:01’

SELECT * FROM cpu_ WHERE time < now() - 1m and time > now() - 3m limit 10

SELECT * FROM “foodships” WHERE time > now() - 1h

u 	microseconds
s 	seconds
m 	minutes
h 	hours
d 	days
w 	weeks

注意: 数字不要用引号, 字符串用单引号


GROUP BY

SELECT_clause FROM_clause [WHERE_clause] GROUP BY [* | <tag_key>[,<tag_key]]

select * from cpu_ where host = ‘A5-01’ group by host limit 2

select * from cpu_ where host = ‘A5-01’ group by type_instance limit 2

select * from cpu_ group by host limit 2

select * from cpu_ group by host,instance limit 2

select * from cpu_ group by * limit 2

注: 如果 WHERE 存在, 在 WHERE 之后


SELECT (<field_key>) FROM_clause WHERE <time_range> GROUP BY time(<time_interval>),[tag_key] [fill(<fill_option>)]

select mean(tps) from bench where time > now() - 2d and time < now() - 1d and dest=‘10.2.1.41:80’ group by time(1h) limit 10

select mean(tps) from bench where time > now() - 2d and time < now() - 1d group by time(1h),dest

select mean(tps) from bench where time > now() - 2d and time < now() - 1d group by time(5h),dest

select mean(tps) from bench where time > now() - 2d and time < now() - 1d group by time(5h, 1h),dest

select mean(tps) from bench where time > now() - 2d and time < now() - 1d group by time(5h, -1h),dest

select mean(tps) from bench where time > now() - 2d and time < now() - 1d group by time(5h, -1h),dest fill(none)

select mean(tps) from bench where time > now() - 2d and time < now() - 1d group by time(5h, -1h),dest fill(0)

select mean(tps) from bench where time > now() - 2d and time < now() - 1d group by time(5h, -1h),dest fill(null)

select mean(tps) from bench where time > now() - 2d and time < now() - 1d group by time(5h, -1h),dest fill(previous)


INTO

SELECT_clause INTO <measurement_name> FROM_clause [WHERE_clause] [GROUP_BY_clause]

SELECT MEAN(“water_level”) INTO “all_my_averages” FROM “h2o_feet” WHERE “location” = ‘coyote_creek’ AND time >= ‘2015-08-18T00:00:00Z’ AND time <= ‘2015-08-18T00:30:00Z’ GROUP BY time(12m)

SELECT MEAN() INTO “where_else”.“autogen”.:MEASUREMENT FROM /./ WHERE time >= ‘2015-08-18T00:00:00Z’ AND time <= ‘2015-08-18T00:06:00Z’ GROUP BY time(12m)


其他

SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] LIMIT

SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] GROUP BY *[,time(<time_interval>)] [ORDER_BY_clause] SLIMIT

SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] GROUP BY *[,time(<time_interval>)] [ORDER_BY_clause] LIMIT SLIMIT

SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] LIMIT_clause OFFSET [SLIMIT_clause]

SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] GROUP BY *[,time(time_interval)] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] SLIMIT_clause SOFFSET


例子

select mean(tps) from bench where time > now() - 3d group by *,time(1h) limit 5

select mean(tps) from bench where time > now() - 3d group by *,time(1h) limit 5 offset 2

select mean(tps) from bench where time > now() - 3d group by *,time(1h) limit 5 offset 2 slimit 3

select mean(tps) from bench group by dest limit 2 slimit 2

select mean(tps) from bench group by dest limit 2 slimit 2

select mean(*) from bench group by dest slimit 3

select mean(tps) from bench group by dest slimit 1

select * from bench limit 10; select * from bench limit 10 offset 2

注: 可以将多个语句通过分号分割, 对比结果


时间范围

SELECT_clause FROM_clause WHERE time [’<rfc3339_date_time_string>’ | ‘<rfc3339_like_date_time_string>’ | <epoch_time>] [AND [’<rfc3339_date_time_string>’ | ‘<rfc3339_like_date_time_string>’ | <epoch_time>] […]]

RFC3339 date time format: ‘YYYY-MM-DDTHH:MM:SS.nnnnnnnnnZ’ 其中 .nnnnnnnnn 部分可选

rfc3339_like_date_time_string : ‘YYYY-MM-DD HH:MM:SS.nnnnnnnnn’ 其中 HH:MM:SS.nnnnnnnnn 部分可选

=   equal to
<> not equal to
!= not equal to
>   greater than
>= greater than or equal to
<   less than
<= less than or equal to

SELECT_clause FROM_clause WHERE time now() [[ - | + ] <duration_literal>] [(AND|OR) now() […]]

u microseconds
ms  milliseconds
s seconds
m minutes
h hours
d days
w weeks

注: 1. 优先级 WHERE > GROUP BY > ORDER BY time DESC > LIMIT
2. 如果 SLIMIT 必须 GROUP BY 一起, SOFFSET 必须与 SLIMIT 一起


例子

select mean(tps) from bench where time > ‘2015-11-20T00:00:00.000000000Z’ group by *,time(1h) limit 5

select mean(tps) from bench where time > ‘2015-11-20’ group by *,time(1h) limit 5

select mean(tps) from bench where time > ‘2015-11-20 12:00:00’ group by *,time(1h) limit 5

select mean(tps) from bench where time > ‘2015-11-20T12:00:00Z’ group by *,time(1h) limit 5

select tps from bench where time > ‘2016-11-21T03:00:00Z’ limit 10

select tps from bench where time > ‘2016-11-21T03:00:00Z’ + 44m limit 10

select tps from bench where time > ‘2016-11-21T03:00:00Z’ + 1h limit 10

select tps from bench where time > ‘2016-11-21T03:00:00Z’ - 1h limit 10


正则表达式

SELECT_clause FROM /<regular_expression_measurement>/ WHERE [<tag_key> /<regular_expression_tag_value>/ | <field_key> /<regular_expression_field_value>/]

=~ 匹配
!~ 不匹配
  • FROM 正则匹配 measurement
  • WHERE 正则匹配 tag, fields

select * from /.*/

select * from bench where dest =~ /10.2.1.4.*/ limit 10

select * from bench where dest =~ /./ limit 10


高级

函数

对数据进行一些简单计算, 如 最大, 最小, 均值, 目前支持操作

SELECT FUNCTION(<field_key>) FROM <measurement_name> [WHERE ] [GROUP BY ]

  • COUNT : 计数
  • DISTINCT: 不同值的列表. 类型 linux 命令 uniq
  • MEAN : 均值
  • MEDIAN : 中值
  • MODE : 次数最多的值
  • SPREAD : 最大值和最小值的差值
  • SUM : 求和
  • STDDEV : 标准偏差
  • BOTTOM : 最小的 N 个值; BOTTOM(<field_key>[,<tag_keys>],)[,<tag_keys>]
  • FIRST : 最旧的值; FIRST(<field_key>)[,<tag_key(s)>]
  • LAST : 最旧的值; LAST(<field_key>)[,<tag_key(s)>]
  • MAX : 最大值; MAX(<field_key>)[,<tag_key(s)>]
  • MIN : 最小值; MIN(<field_key>)[,<tag_key(s)>]
  • PERCENTILE: 返回大于总数百分比的值; PERCENTILE(<field_key>, )[,<tag_key(s)>]
  • TOP : 返回最大的 N 个值; TOP(<field_key>[,<tag_keys>],)
  • CEILING : 还没有实现
  • DERIVATIVE : 计算斜率, 差值/(相邻两个值时间间隔/时间间隔); DERIVATIVE(<field_key>, [])
  • DIFFERENCE : 计算相邻两个值的差值; DIFFERENCE(<field_key>), DIFFERENCE((<field_key>))
  • ELAPSED : 上一个值与当前值的时间差值; ELAPSED(<field_key>, )
  • FLOOR : 还未实现
  • HISTOGRAM : 还未实现
  • MOVING_AVERAGE : 前 window 个值的均值为当前值; MOVING_AVERAGE(<field_key>,), MOVING_AVERAGE((<field_key>),)
  • NON_NEGATIVE_DERIVATIVE: DERIVATIVE 去掉负值; NON_NEGATIVE_DERIVATIVE(<field_key>, []), NON_NEGATIVE_DERIVATIVE(AGGREGATION_FUNCTION(<field_key>),[])
  • HOLT_WINTERS: 根据以前预测未来 N 个值; HOLT_WINTERS(FUNCTION(<field_key>),,< S >)
  • HOLT_WINTERS_WITH_FIT : 根据以前预测未来 N 个值; HOLT_WINTERS_WITH_FIT(FUNCTION(< field_key >),< N >,< S >)

select count(*) from bench

select distinct(concurreny) from bench

select count(distinct(concurreny)) from bench

select min(tps),max(tps),median(tps),count(tps) from bench where dest =~ /./ limit 10 一个查询可以指定多个函数

select median(tps) from bench where time > now() - 4d

select bottom(tps, 3) from bench where time > now() - 4d

select bottom(tps, 3),concurreny from bench where time > now() - 4d

select bottom(tps, 3),* from bench where time > now() - 4d

select bottom(tps, concurreny, 3) from bench where time > now() - 4d

select bottom(tps, dest, 3) from bench where time > now() - 4d : 每个 dest 的最小值.

select bottom(tps, dest, 4) from bench where time > now() - 4d : 每个 dest 的最小值.

select percentile(tps, 100) from bench where dest =~ /./ group by dest

select top(tps, 3) from bench where dest =~ /./ group by dest

select top(tps, dest, 3) from bench where dest =~ /./

select (tps) from bench where dest =~ /./ limit 10 ; select DIFFERENCE(tps) from bench where dest =~ /./ limit 10

select ELAPSED(tps, 1s) from bench where dest =~ /./ limit 10

select tps from bench where dest =~ /./ limit 10; select MOVING_AVERAGE(tps,2) from bench where dest =~ /./ limit 10

select tps from bench where dest =~ /./ limit 10; select DERIVATIVE(tps,2m) from bench where dest =~ /./ limit 10

select tps from bench where dest =~ /./ limit 10; select NON_NEGATIVE_DERIVATIVE(tps,2m) from bench where dest =~ /./ limit 10

select tps as throughput from bench where dest =~ /./ limit 10

select HOLT_WINTERS(first(tps), 5, 1) as throughput from bench where time > now() - 3d group by time(1h)

注: limit 不起作用


备份策略

CREATE RETENTION POLICY <retention_policy_name> ON <database_name> DURATION REPLICATION [SHARD DURATION ] [DEFAULT]

ALTER RETENTION POLICY <retention_policy_name> ON <database_name> DURATION REPLICATION SHARD DURATION DEFAULT

DURATION

  • m minutes
  • h hours
  • d days
  • w weeks
  • INF infinite

SHARD DURATION

  • u microseconds
  • ms milliseconds
  • s seconds
  • m minutes
  • h hours
  • d days
  • w weeks

例子

CREATE RETENTION POLICY “one_day_only” ON “NOAA_water_database” DURATION 1d REPLICATION 1 DEFAULT

CREATE RETENTION POLICY “one_day_only” ON “NOAA_water_database” DURATION 1d REPLICATION 1

CREATE RETENTION POLICY “what_is_time” ON “NOAA_water_database” DURATION 2d REPLICATION 1

ALTER RETENTION POLICY “what_is_time” ON “NOAA_water_database” DURATION 3w SHARD DURATION 30m DEFAULT

DROP RETENTION POLICY <retention_policy_name> ON <database_name>

DROP RETENTION POLICY “what_is_time” ON “NOAA_water_database”


Continuous Queries

SHOW CONTINUOUS QUERIES

DROP CONTINUOUS QUERY <cq_name> ON <database_name>

DROP CONTINUOUS QUERY “idle_hands” ON “telegraf”`

CREATE CONTINUOUS QUERY <cq_name> ON <database_name>
BEGIN
<cq_query>
END

CREATE CONTINUOUS QUERY <cq_name> ON <database_name>
RESAMPLE EVERY FOR
BEGIN
<cq_query>
END

cq_query : SELECT <function[s]> INTO <destination_measurement> FROM [WHERE ] GROUP BY time()[,<tag_key[s]>]


例子

CREATE CONTINUOUS QUERY “cq_basic” ON “transportation”
BEGIN
SELECT mean(“passengers”) INTO “average_passengers” FROM “bus_data” GROUP BY time(1h)
END

CREATE CONTINUOUS QUERY “cq_basic_rp” ON “transportation”
BEGIN
SELECT mean(“passengers”) INTO “transportation”.“three_weeks”.“average_passengers” FROM “bus_data” GROUP BY time(1h)
END

CREATE CONTINUOUS QUERY “cq_basic_br” ON “transportation”
BEGIN
SELECT mean() INTO “downsampled_transportation”.“autogen”.:MEASUREMENT FROM /./ GROUP BY time(30m),*
END

CREATE CONTINUOUS QUERY “cq_basic_offset” ON “transportation”
BEGIN
SELECT mean(“passengers”) INTO “average_passengers” FROM “bus_data” GROUP BY time(1h,15m)
END

  1. 每半小时计算一次当前一小时的均值, 后面会覆盖前面

CREATE CONTINUOUS QUERY “cq_advanced_every” ON “transportation”
RESAMPLE EVERY 30m
BEGIN
SELECT mean(“passengers”) INTO “average_passengers” FROM “bus_data” GROUP BY time(1h)
END

  1. 计算过去一小时每半小时的平均值

CREATE CONTINUOUS QUERY “cq_advanced_for” ON “transportation”
RESAMPLE FOR 1h
BEGIN
SELECT mean(“passengers”) INTO “average_passengers” FROM “bus_data” GROUP BY time(30m)
END

  1. 每一小时执行一次, 计算过去一小时半, 每半小时的平均值

CREATE CONTINUOUS QUERY “cq_advanced_every_for” ON “transportation”
RESAMPLE EVERY 1h FOR 90m
BEGIN
SELECT mean(“passengers”) INTO “average_passengers” FROM “bus_data” GROUP BY time(30m)
END

其中:

  1. EVERY 表示每多长时间执行一次; 没有给定, 以 group by 为准, 如果比 group by 大, 并且时间跨度也以当前为准.
  2. FOR 表示计算多大时间跨度
  3. group by 表示, 时间跨度以多长时间为一份

使用场景

  1. 降低时间采样间隔
  2. 预计算, 将常用操作分散执行,降低系统压力
  3. 替代 hava 语句
  4. 替代 嵌套函数

备份与恢复

最直接的备份策略就是将 influxd 配置文件中数据路径指定的整个目录拷贝到一个新的地方。
修改配置文件路径即可。

influxd backup

influxd backup -database

influxd backup -database telegraf -retention autogen -since 2016-02-01T00:00:00Z /tmp/backup

influxd backup -database mydatabase -host 10.0.0.1:8088 /tmp/mysnapshot

influxd restore [ -metadir | -datadir ]

influxd restore -metadir /var/lib/influxdb/meta /tmp/backup

influxd restore -database telegraf -datadir /var/lib/influxdb/data /tmp/backup


存储引擎

监控数据的特点:

  • 大量的基于时间数据
  • 写吞吐量要求高
  • 读吞吐量要求高
  • 固定时间大量删除操作(数据过期)
  • 大多数是写和追加

其他存储引擎(LevelDB, RocksDB, HyperLevelDB, and LMDB, BoltDB)目前不能满足需求:

  1. 动态备份(RocksDB, HyperLevelDB 可以支持)
  2. 定期大量删除
  3. LevelDB 单一进程 file handler 到达极限
  4. BoltDB : 虚拟环境 IOPS 销毁太高

基于以上原因,专门针对时间序列的数据库 TLSM 诞生

TSM

  1. 创建: 同时到 cache 和 WAL
  2. 更新:
  3. 删除: 写一个删除操作到 WAL, 之后更新 Cache 和 FileStore. 其中 Cache 过期对应的 entries, FileStore 标记对应的文件为 tombstone
  4. 读: 每个块都包含时间范围, 首先找到对应时间范围的块, 之后, 二分查找

WAL : 固定大小的块. 当写满固定大小后, 落盘, 创建新的块. 保存 delete, write 的记录, 每条记录是 TLV 标准

Cache : WAL 的缓存, 方便查询, 当到达一定大小后, snap 到 TSM; 如果写的速度过快, 将拒绝写. 也支持, 多长时间没有新的写, 就 snap.

TSM File: mmap 的只读的压缩的形式存储数据. 类似 SSTable, LSM Tree Variants

FileStore: 管理 TSM File, 增加或删除 TSM File

Compactor: 将对读没有优化的数据转为读优化的数据; , 多级压缩

Compaction Planer: 决定哪些 TSM 文件可以压缩, 确保并行压缩成为可能

Writers/Readers : WAL, TSM, tombstone 等组件提供的读写接口

TSM 文件数据格式

    ┌────────┬────────────────────────────────────┬─────────────┬──────────────┐
    │ Header │               Blocks               │    Index    │    Footer    │
    │5 bytes │              N bytes               │   N bytes   │   4 bytes    │
    └────────┴────────────────────────────────────┴─────────────┴──────────────┘

    ┌───────────────────┐
    │      Header       │
    ├─────────┬─────────┤
    │  Magic  │ Version │
    │ 4 bytes │ 1 byte  │
    └─────────┴─────────┘

    ┌───────────────────────────────────────────────────────────┐
    │                          Blocks                           │
    ├───────────────────┬───────────────────┬───────────────────┤
    │      Block 1      │      Block 2      │      Block N      │
    ├─────────┬─────────┼─────────┬─────────┼─────────┬─────────┤
    │  CRC    │  Data   │  CRC    │  Data   │  CRC    │  Data   │
    │ 4 bytes │ N bytes │ 4 bytes │ N bytes │ 4 bytes │ N bytes │
    └─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘

    ┌────────────────────────────────────────────────────────────────────────────┐
    │                                   Index                                    │
    ├─────────┬─────────┬──────┬───────┬─────────┬─────────┬────────┬────────┬───┤
    │ Key Len │   Key   │ Type │ Count │Min Time │Max Time │ Offset │  Size  │...│
    │ 2 bytes │ N bytes │1 byte│2 bytes│ 8 bytes │ 8 bytes │8 bytes │4 bytes │   │
    └─────────┴─────────┴──────┴───────┴─────────┴─────────┴────────┴────────┴───┘

    ┌─────────┐
    │ Footer  │
    ├─────────┤
    │Index Ofs│
    │ 8 bytes │
    └─────────┘

    每个 block 是压缩过的. 格式如下

    ┌───────┬─────┬─────────────────┬──────────────────┐
    │ Type  │ Len │   Timestamps    │      Values      │
    │1 Byte │VByte│     N Bytes     │     N Bytes      │
    └───────┴─────┴─────────────────┴──────────────────┘

集群与高可用

商业版本支持. 高可用有一个基本的版本

参考

全部 SQL 语句: https://docs.influxdata.com/influxdb/v1.0/query_language/spec/

附录

与展示层对接

  • grafana

与数据源对接

  • collectd
  • telegraf
  • graphite
  • openTSDB

使用建议

  • tag 优于 fields : tag 都被索引, 而 fields 没有
  • 将长的 tag 拆分 : blueberries.plot-1.north 拆分为 crop=blueberries,plot=1,region=north
  • tag 根据 key 排序

问题

[httpd] 10.9.1.13 - - [22/Jan/2017:10:27:20 +0800] “POST /write?consistency=any&db=telegraf&precision=ns&rp=autogen HTTP/1.1” 204 0 “-” “InfluxDBClient” 4d035372-e04a-11e6-8287-000000000000 89343
[tsm1] 2017/01/22 10:27:21 Snapshot for path /repo/influxdb/data/telegraf/autogen/103 written in 26.194993ms
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x0]

goroutine 2289614 [running]:
panic(0xba8be0, 0xc82000e090)
/usr/local/go/src/runtime/panic.go:481 +0x3e6
github.com/influxdata/influxdb/tsdb/engine/tsm1.Values.Size(0xc838bb4000, 0x150, 0x200, 0xc847639ca0)
/root/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/encoding.gen.go:28 +0x88
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Cache).Snapshot(0xc826707680, 0x0, 0x0, 0x0)
/root/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/cache.go:276 +0x369
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).WriteSnapshot.func2(0xc8202110a0, 0xc839e49e28, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/root/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:732 +0x245
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).WriteSnapshot(0xc8202110a0, 0x0, 0x0)
/root/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:738 +0x8d
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactCache(0xc8202110a0)
/root/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:814 +0x25d
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).SetCompactionsEnabled
/root/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:166 +0x13f

[run] 2017/01/22 10:34:17 InfluxDB starting, version 1.0.2, branch master, commit ff307047057b7797418998a4ed709b0c0f346324
[run] 2017/01/22 10:34:17 Go version go1.6.2, GOMAXPROCS set to 8
[run] 2017/01/22 10:34:17 Using configuration at: /etc/influxdb/influxdb.conf

猜你喜欢

转载自blog.csdn.net/wenxueliu/article/details/83099443