本文介绍了cassandra 之commit_log
1. Cassandra 写入数据流程如下
a) 写入commit_log 中
b) 写入内存的memtable 中
c) 最后从内存的memtable中flush 到磁盘的SStables中
注意 a) 和 b) 是并行执行的
commit_log 的作用类似于oracle的redo log 和 mongo 的 jouney log. 用于意外宕机例如断电机器或者自动重启后数据恢复
默认存在路径 /var/lib/cassandra/commitlog
可以在配置文件 cassandra.yaml 修改默认路径
# commit log. when running on magnetic HDD, this should be a
# separate spindle than the data directories.
# If not set, the default directory is $CASSANDRA_HOME/data/commitlog.
commitlog_directory: /u01/cassandra/commitlog
commit_log 的默认大小是8G
# Total space to use for commit logs on disk.
#
# If space gets above this value, Cassandra will flush every dirty CF
# in the oldest segment and remove it. So a small total commitlog space
# will tend to cause more flush activity on less-active columnfamilies.
#
# The default value is the smaller of 8192, and 1/4 of the total space
# of the commitlog volume.
#
# commitlog_total_space_in_mb: 8192
commit_log 设置压缩(默认不压缩，生产环境不建议压缩)
# Compression to apply to the commit log. If omitted, the commit log
# will be written uncompressed. LZ4, Snappy, and Deflate compressors
# are supported.
#commitlog_compression:
# - class_name: LZ4Compressor
# parameters:
# -
关于commit log 的归档
commit log 的归档发生在如下的几种情况:
- 当节点启动的时候
- 当commit_log 写入到磁盘的时候
- 手动执行归档命令的时候
关于commit_log 归档的设置
在配置文件 $CASSANDRA_HOME\conf\commitlog_archiving.properties 中
设置归档 参数archive_command
# commitlog archiving configuration. Leave blank to disable.
# Command to execute to archive a commitlog segment
# Parameters: %path => Fully qualified path of the segment to archive
# %name => Name of the commit log.
# Example: archive_command=/bin/ln %path /backup/%name
#
# Limitation: *_command= expects one command with arguments. STDOUT
# and STDIN or multiple commands cannot be executed. You might want
# to script multiple commands and add a pointer here.
archive_command=
创建归档路径
[root@tjtestrac1 ~]# mkdir -p /u01/cassandra/arch
[root@tjtestrac1 ~]# chown -R cassandra:cassandra /u01/cassandra/arch
开发一个简单的备份脚本
-rwxrwxrwx 1 cassandra cassandra 64 Dec 19 11:02 archive_commit_log.sh
#!/bin/bash
cp /u01/cassandra/commitlog/* /u01/cassandra/arch/
设置归档参数
archive_command=/u02/cassandra/cassandra/apache-cassandra-2.2.7/script/archive_commit_log.sh
重启Cassandra 观察归档路径
[cassandra@tjtestrac1 arch]$ ls -lhtr
total 128K
-rw-rw-r-- 1 cassandra cassandra 32M Dec 19 11:09 CommitLog-5-1545188942068.log
-rw-rw-r-- 1 cassandra cassandra 32M Dec 19 11:09 CommitLog-5-1545188702567.log
-rw-rw-r-- 1 cassandra cassandra 32M Dec 19 11:09 CommitLog-5-1545188702566.log