Spark deployment mode
Standalone mode
Spark on Yarn
Mesos and Yarn connection
Container: Doctor
A combination: First allocate the underlying Mesos resources to the container, run Hadoop (MapReduce) inside the container, and use the Yarn framework in Hadoop
Spark can also be deployed separately with Mesos, and the practice between them is very close
Spark can also use Yarn for resource scheduling management, but because the Hadoop ecosystem uses Yarn a lot, Spark has to support him to do so
Spark Modern Enterprise Architecture
Spark Streaming is a batch calculation that cannot achieve millisecond-level calculations, but can only achieve second-level calculations. If you want to achieve millisecond-level real-time response enterprise applications, you need to use the stream computing framework Flink
Unified deployment of Hadoop and Spark
benefit:
Can Spark replace Hadoop?
Hadoop is an ecosystem. Although it has flaws, the problem lies in MapReduce . It also has storage parts such as HDFS, Hive, HBase...
As a computing framework, Spark is benchmarked against MapReduce in Hadoop, and the computing framework replaces the computing framework.
