The development process of hadoop spark hive storm

1. Data can be saved, hdfs (distributed file system)

2, can perform resource scheduling yarn

3. It can calculate the stored big data, mapreduce (multiple hard disks can be processed at the same time)

4. More flexible and faster computing framework spark sparksql

5. Simplify the development of map reduce, hive (data warehouse using sql)

6. Machine Learning Mahout

7. Process storm in real time (the disadvantage is that it can only process pre-determined data and logic)

 

Basic architecture: hdfs+yarn spark hive mahout

Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=326525459&siteId=291194637