Big Data processing ideas

1. Process Optimization: optimization sql, consider the use of temporary tables or intermediate table

2. Database Optimization: reasonable construction of the index, reasonable design table structure, database cluster

3. Flows: and not separated by, and not commonly used separately; the data stored in the database sub-library sub-table partitions; split data file storage, consider a batch process; the principle is to minimize the technical data of each operation ;

4. Cache Technology: reading and writing less cache

The rational use nosql: mongodb, redis, memched, hbase etc.

6. Large distributed data scheme: hadoop, spark, storm etc.

 

 

Kind nosql database: temporary key-value store (Memcached Redis), persistent key-value store (ROMA Redis), document-oriented storage (mongoDB couchDB), column-oriented storage (Casssandra HBase)

 

Guess you like

Origin www.cnblogs.com/hzq3554055/p/11999765.html