Detailed explanation of hadoop2.0 configuration file

Go to: http://www.cnblogs.com/yinghun/p/6230436.html

Hadoop operating mode is divided into secure mode and non-secure mode. Here, I will describe the important parameter functions and functions of the main configuration files in non-secure mode. The Hadoop version used in this article is 2.6.4.

etc/hadoop/core-site.xml

parameter	attribute value	explain
fs.defaultFS	NameNode URI	hdfs://host:port/
io.file.buffer.size	131072	SequenceFiles file. Read and write cache size setting

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://192.168.1.100:900</value>
        <description>192.168.1.100 is the server IP address, in fact, the host name can also be used</description>
    </property>
    <property>
        <name>io.file.buffer.size</name>
        <value>131072</value>
        <description>The unit of the attribute value is KB, 131072KB is the default 64M</description>
    </property>
</configuration>

etc/hadoop/hdfs-site.xml

Placement NameNode

parameter	attribute value	explain
dfs.namenode.name.dir	Storage space and persistent processing logs on the NameNode where the local file system is located	If this is a comma-separated list of directories, then the name table is copied to all directories in case it is needed.
dfs.namenode.hosts/ dfs.namenode.hosts.exclude	Datanodes permitted/excluded列表	If necessary, these files can be used to control the list of allowed data nodes
dfs.blocksize	268435456	Large file system HDFS block size is 256MB
dfs.namenode.handler.count	100	Set up more namenode threads to handle the high volume of RPC requests from the datanode

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
        <description>Number of shards, configure it to 1 for pseudo-distribution</description>
    </property>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:/usr/local/hadoop/tmp/namenode</value>
        <description>Path where namespaces and transactions are permanently stored in the local file system</description>
    </property>
    <property>
        <name>dfs.namenode.hosts</name>
        <value>datanode1, datanode2</value>
        <description>datanode1, datanode2 respectively correspond to the host name of the server where the DataNode is located</description>
    </property>
    <property>
        <name>dfs.blocksize</name>
        <value>268435456</value>
        <description>The large file system HDFS block size is 256M, the default value is 64M</description>
    </property>
    <property>
        <name>dfs.namenode.handler.count</name>
        <value>100</value>
        <description>More NameNode server threads to handle RPCS from DataNodes</description>
    </property>
</configuration>

Placement DataNode

parameter	attribute value	explain
dfs.datanode.data.dir	A comma-separated list of local filesystem paths on a DataNode where it should save its blocks	If this is a comma-separated list of directories, then data will be stored in all named directories, usually on different devices.

<configuration>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>file:/usr/local/hadoop/tmp/datanode</value>
        <description>The path where the DataNode stores blocks in the local file system</description>
    </property>
</configuration>