Introduction to big data
Let me share with you an attempt to expand our knowledge.
1 Byte (B) = 8 bit
1 Kilo Byte (KB) = 1024B
1 Mega Byte (MB) = 1024 KB
1 Giga Byte (GB) = 1024 MB
1 Tera Byte (TB) = 1024 GB (most people know this )
1 Peta Byte (PB) = 1024 TB (awareness over 60%)
1 Exa Byte (EB) = 1024 PB
1 Zetta Byte (ZB) = 1024 EB
1 Yotta Byte (YB) = 1024 ZB (awareness over 80% of people)
1 Bronto Byte (BB) = 1024 YB
1 Nona Byte (NB) = 1024 BB
1 Dogga Byte (DB) = 1024 NB (people who know more than 90%)
1 Corydon Byte (CB) = 1024DB( People who know more than 95%)
1 Xero Byte (XB)=1024CB (people who know more than bloggers, 噫...)
Some people think that their phone memory is small after reading it, hehehe
concept
Big data, IT industry terminology, refers to a collection of data that cannot be captured, managed and processed with conventional software tools within a certain time frame. It requires a new processing model to have stronger decision-making power, insight and discovery. Mass, high growth rate and diversified information assets of process optimization capabilities.
Features (proposed by IBM)
Volume (large)
Velocity (high speed)
Variety (various)
Value (low value density)
Veracity (authenticity)
Since the amount of data is large, traditional data processing technology is no longer competent. What is the core technology to sort out the massive amount ?
Mass data storage (distributed)
Mass data calculation (distributed)
These seemingly awesome core technologies do not actually require us to study from scratch. We only need to stand on the shoulders of giants, see farther away, and use the formed framework for storage and computing .
Storage type framework
- HDFS-Distributed File Storage System
- HBase-distributed database system
- Kafka-distributed message caching system
Operation type framework
- Hive-data warehouse tool (can receive SQL, translate it into mapreduce or spark program to run)
- Flume-data collection
- Sqoop-data migration
- Elisticsearch-distributed search engine
Big data application scenarios (just understand)
1. Logistics and warehousing: help merchants refine operations, increase sales, and save costs
2. Retail: analyze user consumption habits, provide convenience for users to purchase goods, thereby increasing product sales
3. Tourism: deeply integrate big data capabilities with the needs of the tourism industry, Jointly build the future of smart management, smart services and smart marketing in the tourism industry
4. Product recommendation: recommend products that users like
5. Insurance: massive data mining and risk prediction, help the insurance industry to accurately promote sales, and improve the ability of refined pricing
6. Finance : Reflect user characteristics in multiple dimensions, help financial institutions recommend high-quality customers, and prevent fraud risks
7. Real estate: Big data fully assists the real estate industry to create precise investment strategies and marketing
8. Artificial intelligence
Big data department structure
note!!!
From another perspective, big data is actually more data. What we have to do is to change our thinking to better operate the data.
It’s not that difficult to think about big data, hehehe!!!
The blogger also came from my comfort, hahaha
Starting today, bloggers may have to take the first step towards big data
In the process of self-love learning, know what it is and why it is
Looking forward to learning with you in the exchange
This blogger will combine his own learning to share with you the bits and pieces of technology
In the future, the blogger will continue to update the framework technology... Welcome everyone to pay attention, like and comment!