HDFS
Hadoop fs command,
hadoop fs <>
Three-node cluster structures.
Hdfs local file system and isolated.
hdfs dfs <>
start-all.sh Where?
cd $HADOOP
/opt/bigdata/hadop-2.7.3/etc/hadoop
slaves file contents look.
ssh node-02 -> datanode
？？？
jsp view the process
hdfs principle :( interview)
fault recovery
Each will have a quick copy. 128M maximum segmentation.
A datanode corresponds to a machine?
hdfs-site.xml configuration
configuration
Documentation: https: //hadoop.apache.org/docs/r2.7.3
blk stored in the frame flexibility, how to configure.
Rack awareness.
A copy of zero increase and improve throughput.
hadoop fs -settrep -R 4 /path
hdfa fsck /salary.txt -files -bllocks -locations
You can also view via the web: node-01: 50070
4 HDFS master-slave architecture
Read mode
B1->dn1, dn2, dn3
B2->dn4,dn2, dn1
Metadata -> namenode memory and to retrieve namenode each block, consisting hadoop.dat
HDFS for large files
HDFS has an upper limit because each metadata 150Byte, and the memory is limited, it is suitable for storing large files
HDFS will be stored on the hard disk memory for safety
Fsimage
Edits log
Secondary namenode
Dual Asynchronous write cache
When t-t1> 1h trigger Check-point
Heartbeat mechanism
10min deemed hang above datanode
1 hour block report
Load Balancing
The difference becomes smaller disk utilization