Cloudera's software architecture includes the following modules:
System deployment and management, data storage, resource management, processing engine, security, data management, tool library and access interface.
Role information for some key components:
Hardware Configuration
Cluster servers are divided into management nodes and worker nodes according to the tasks undertaken by the nodes .
The management role of each component is generally deployed on the management node;
Worker nodes are generally deployed with storage, container or computing roles of various roles.
Depending on the type of business, the specific configuration of the cluster is also different:
1. Real-time stream processing service cluster: Hadoop's real-time stream processing performance has high requirements on node memory and CPU. The stream processing message throughput based on Spark Streaming can increase linearly with the increase of the number of nodes.