With the outbreak of big data, China's IT industry environment will also face a new round of reshuffle, not only enterprises, but also opportunities for practitioners to transform. If the IT people are unified as the sailors on a ship, big data is the biggest wave, and it is made by the tide. Here, through the analysis of everyone's doubts on the Internet, here are 5 questions that everyone is relatively concerned about.
In the United States, the average annual salary of big data engineers is 175,000 US dollars, and in China's top Internet companies, the salary of big data engineers is more than 30% higher than that of other positions at the same level. The DT era came too suddenly, the domestic development momentum is very strong, but the talents related to big data are very limited, and the supply will be in short supply in the next few years.
1. What is big data? How to understand big data?
If data is compared to the water on the earth, a single data is a drop of water. Big data is like the ocean on the earth. It is big enough and there are enough water droplets. way of estimating the total amount of water droplets in the ocean. So, do you understand big data? It is said that we are in the ocean of data. When you go out to sea in Phuket and play on the beach in Nha Trang, you are really in the ocean and have close contact with it.
The 4Vs of big data are "large volume", "variety", "high value" and "fast Velocity". Take the ocean as an example:
A. The amount of water in the ocean is very large;
B. The sea water is diverse, the sea water in the Pacific Ocean is different from the sea water in the Atlantic Ocean, and the substances and living species contained in the sea water in different places are different;
C. The ocean brings too many benefits to all human beings. People use the ocean to develop the infinite value in the ocean;
D. The speed is fast. There are two levels of meaning. One is that the seawater flows fast, and the other is that with the improvement of technology, our use of seawater has also accelerated (see the development of speedboats and cruise ships). PS: The analogy is a bit far-fetched.
2. What are the career directions in the field of big data?
Entry direction 1: Big data system research and development field
To put it bluntly, in the field of big data system research and development, IT is engaged in big data, and is responsible for the construction and maintenance of the entire operating system, data preparation, platform and tool development.
Entry Direction 2: Big Data Modeling and Mining
Big data mining refers to the use of algorithms and models to improve data processing efficiency, mine data value, and realize the transformation from data to knowledge. Big data mining engineers, also known as machine learning algorithm engineers, must first have a solid foundation in mathematical statistics. Statistics is the most basic. At the same time, they are proficient in common statistical analysis and machine learning model algorithms, and have curiosity, patience, and research spirit. .
Entry direction 3: big data analysis application field
Big data analysis application is to help enterprises convert data and technology into business value. We often say junior/senior data analysts, or data analysis project managers belong to this field. Compared with the first two fields, the skills required in the field of analysis and application are the most comprehensive, requiring practitioners to have a complex knowledge structure and background, including a deep understanding of the industry and business, data analysis, processing and interpretation, communication and management. three aspects of ability.
3. Am I late to learn big data now?
Gartner, a leading market and technology research firm, is featured in the Gartner 2015 Emerging Technology Development Cycle report. Blockchain, self-driving cars, and the Internet of Things are at their peak, while big data has seen a decline.
Many people use this as a basis for saying that big data is nearly obsolete, but the truth is that this Gartner chart reflects the development trend of emerging technologies. After 10 years of development, with the increasing attention of new technologies such as blockchain and artificial intelligence, it is natural that big data, as a relatively mature technology, has received less attention. into the commercial stage. Big data technology has become the cornerstone of many of these 12 technologies that may change the world, including mobile Internet, knowledge work automation, Internet of Things, cloud computing, advanced robotics, autonomous vehicles, and genomics.
Big data does not have the so-called "overheating" and "false fire" problems. The next 10 years or even longer will be the golden stage of big data development, and related industries will bring huge development opportunities. From the perspective of market and industry trends, now is the time to learn big data.
4. How to change career as a big data engineer?
A common big data engineer recruitment requirement is:
1. Bachelor degree or above in computer or related major
2. Rich experience in data development, deep understanding and practical experience in data processing, data modeling, data analysis, etc.
3. Familiar with SQL and have some experience in SQL performance optimization
4. Proficiency in Java language, MapReduce programming, one of the scripting languages Shell/Python/Perl
5. 业务理解力强，对数据、新技术敏感，对云计算、大数据技术充满热情
6. 积极乐观、诚信、有责任心；具备强烈的进取心、求知欲及团队合作精神
数据工程师属于典型的技术线，负责搭建仓库搭建、数据的存储、处理、计算处理、报表开发等。
五、给想转做大数据相关工作的建议
重视基础。无论各种岗位，基础是成长的基石。
发挥专长。从能够发挥自己现有专长的岗位做起，可以让新团队更欢迎你的加入。比如算法模型的工程化，偏重于业务的数据挖掘，大数据平台开发，机器学习系统开发等等，这些工作对于普通工程师更容易上手。而普通工程师直接转偏研究方向的算法工程师，难度更高。
准备充分。请预先做好相关知识的学习，有动手实践更佳。如果没有一点准备，雇主如何相信你对这个领域真的有兴趣呢？
最后，如果你确实对大数据、数据挖掘有浓厚兴趣，最好的办法是立刻开始实践。推荐下我自己创建的大数据交流学习群 724693112，不管你是小白还是大牛我们都欢迎，学习文档资料已上传到群文件，不定期分享干货每周有一期免费的大数据技术分享课给大家，还有整理好的各编程技术教程可以私聊群主获取，希望有兴趣的朋友都可以来了解一下。
也许你不会以此为职业，但是可以多一技傍身。也许，未来这些技能对于程序员而言，就好比现在 MS Office 对于职场人一样普遍。