- To prevent hadoop cluster metadata such as version due to loss lead to start the error Namenode not format, so that all data is lost, you need to back up data services
- CDH primarily used data Hive, hbase tables, it is only necessary to back up Hive, hbase data (no data to be re-formatted hadoop)
Test hdfs reformatting, data recovery
- Description 1: hdfs manually reformat / catalog no data, this time related to the need to create a basic directory: (/ hbaes, / user / history)
- Note 2: This test only assume hdfs reformatting the data mysql database cdh use is normal (scm library)
a, hdfs directory structure: is a group owner
| hdfs directory | Explanation |
|---|---|
| /hbase/data/ | hbase: hbase, data storage hbase |
| /user/history | mapred: hadoop, store mapreduce task data, this directory does not fail to start JobHistory |
[root@c6 ~]# hdfs dfs -ls /
Found 3 items
drwxr-xr-x - hbase hbase 0 2020-01-13 11:49 /hbase
drwx-wx-wx - hdfs supergroup 0 2020-01-13 11:49 /tmp
drwxr-xr-x - hdfs supergroup 0 2020-01-13 11:34 /user
[root@c6 ~]# hdfs dfs -ls /user
Found 3 items
drwx------ - hdfs supergroup 0 2020-01-13 11:48 /user/hdfs
drwxr-xr-x - mapred hadoop 0 2020-01-13 11:28 /user/history
drwxr-xr-x - hdfs supergroup 0 2020-01-13 11:34 /user/hive
b, ahead of backup hive, hbase data
## 1，备份hive数据
[root@c6 ~]# hdfs dfs -ls /user/hive/warehouse
Found 2 items
drwxr-xr-x - hdfs supergroup 0 2020-01-13 11:37 /user/hive/warehouse/t1
drwxr-xr-x - hdfs supergroup 0 2020-01-13 11:35 /user/hive/warehouse/test
[root@c6 ~]# hdfs dfs -get /user/hive/warehouse /hive.warehouse
## 2，备份hbase数据: 只需备份有表的namespace( default)
[root@c6 ~]# hdfs dfs -ls /hbase/data
Found 2 items
drwxr-xr-x - hbase hbase 0 2020-01-13 11:39 /hbase/data/default
drwxr-xr-x - hbase hbase 0 2020-01-13 11:49 /hbase/data/hbase
[root@c6 ~]# hdfs dfs -ls /hbase/data/default
Found 1 items
drwxr-xr-x - hbase hbase 0 2020-01-13 11:39 /hbase/data/default/t1
[root@c6 ~]# hdfs dfs -get /hbase/data/default /hbase.data.default
c, reformatting hdfs, recovery hive, hbase data
Note: to ensure the absence of data state, hive, hbase normal service starts
## 1，恢复hive数据
[root@c6 ~]# hdfs dfs -ls /user/hive/warehouse
Found 0 items
[root@c6 ~]# su hdfs
[hdfs@c6 ~]# hdfs dfs -put /hive.warehouse/t1 /user/hive/warehouse/
## 2，恢复hbase数据
[root@c6 ~]# ls /hdfs/hbase.data/default/t1 -R
/hdfs/hbase.data/default/t1:
6172eae317480a6645bc88ba24e66f88
/hdfs/hbase.data/default/t1/6172eae317480a6645bc88ba24e66f88:
f recovered.edits
/hdfs/hbase.data/default/t1/6172eae317480a6645bc88ba24e66f88/f:
360bbc97c6214a81b323c7ed18b759b6
/hdfs/hbase.data/default/t1/6172eae317480a6645bc88ba24e66f88/recovered.edits:
2.seqid
[root@c6 ~]# su hdfs
[hdfs@c6 ~]# hdfs dfs -put /hbase.data/default/t1 /hbase/data/default/
[hdfs@c6 ~]# hdfs dfs -chown -R hbase:hbase /hbase
#修复Hbase数据
[hbase@c6 ~]# hbase hbck -repair #自动修复meta元数据表，使得hbase业务表数据正常访问
