CDH starts lzo compression

CDH installation is complete
image.png

2. Add parcel

image.png

image.png

image.png

image.png


image.png

image.png

image.png

image.png



HDFS configuration

Search for " compression codec " in the HDFS configuration item and add com.hadoop.compression.lzo.LzopCodec

image.png

Search for " Hive auxiliary JAR directory " in Hive configuration items, and add /opt/cloudera/parcels/GPLEXTRAS/lib/hadoop/lib

image.png

Sqoop configuration sqoop-conf / sqoop-env.sh of Sqoop 1 Client client code segment Advanced Configuration (safety)

HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/opt/cloudera/parcels/GPLEXTRAS/lib/hadoop/lib/*

JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:/opt/cloudera/parcels/GPLEXTRAS/lib/hadoop/lib/native

image.png

yarn configuration

Search for " yarn.nodemanager.resource.memory-mb " in the yarn configuration item and modify it to 4G .

Search for " yarn.scheduler.maximum-allocation-mb " in the yarn configuration item and modify it to 2G .





Guess you like

Origin blog.51cto.com/wsxxsl/2668231