1. The basic situation of the industry
1.1 Market size
At present, the statistical caliber of the big data industry has not yet been established. As for the scale of my country's big data industry, various research institutions have adopted indirect methods to estimate. The China Academy of Information and Communications Technology combined with the survey and calculation of big data-related companies, the market size of my country's big data core industry in 2016 was about 16.8 billion yuan, a growth rate of 45% compared with 2015, and reached 23.4 billion yuan in 2017, compared with 2016 The growth rate reached 39.3%.
With national policy incentives and the gradual maturity of big data application models, China's big data market will continue to maintain a rapid growth of more than 30% per year in the next few years. It is estimated that by 2020, China's big data market will reach 57.8 billion yuan.
1.2 The main regional distribution of my country's big data industry
1.2.1 Regional distribution
my country's big data industry clusters are mainly located in economically developed areas. Beijing, Shanghai, and Guangdong are the core areas for development. These areas have well-known Internet and technology companies, high-end scientific and technological talents, and strong national policies to support the development of good information technology industries. The foundation has formed a relatively complete industrial format, and the scale of the industry is still expanding.
In addition, the big data industry circle with Guizhou and Chongqing as the center, although located in the economically backward southwest region, but Guizhou, Chongqing and other places rely on the policy guidance provided by the government for the development of their big data industry, and actively introduce big data Relevant enterprises and core talents are trying to occupy the commanding heights of the big data industry and drive the new development of the regional economy.
Relying on the advantages of the electronic information industry in Guangzhou, Shenzhen and other regions, the Pearl River Delta region has played a role in the agglomeration of the two national supercomputing centers of Guangzhou and Shenzhen. Driven by a group of backbone enterprises such as Tencent, Huawei, and ZTE, the Pearl River Delta region has gradually formed The development trend of big data aggregation.
1.2.2 Distribution of application areas
<div align="center"><font color="leight" size="3" >图2 大数据生态链</font></div>
- 1
According to the "2018-2023 China's Big Data Industry Development Prospects and Investment Strategic Planning Analysis Report" released by the Foresight Industry Research Institute, in 2017, in the "Big Data Innovation Top 100" selected by the Industrial Innovation Venture Capital Data Platform, data processing analysis The number of companies in the company reached 19, accounting for 19%, 14 financial big data, 13 marketing big data, and 5 data collection, data storage and data visualization followed closely. Data processing analysis and data visualization, as the core technology part of other big data projects, accounted for 24% of the list, indicating that the development of big data core technology is still a hot direction in the industry. At the same time, in the application field, financial big data and marketing big data together account for 26% of the share, which also indicates that these two fields will have greater development in the future.
2. Development status and trends
2.1 Development status
Big data is a new stage of informatization development. With the convergence and integration of information technology and human production and life, the rapid popularization of the Internet and the explosive growth of global data have promoted the rapid development of the big data industry. All parts of our country are very enthusiastic about the development of big data. Significant progress has been made in policy, technology, industry, application and other levels. Industry applications have been promoted rapidly, and the market scale has grown significantly. This has promoted my country to accelerate the pace of building a data power in order to achieve The manufacturing power and the network provide industry support for the call. China's big data industry market will continue to maintain rapid growth in the future. It is estimated that by 2020, the scale of China's big data industry will reach 1.010 billion yuan, with an average compound growth rate of 29.25% from 2015 to 2020.
Source: Compiled by China Electronics Information Industry Research Institute and China Commercial Industry Research Institute
2.2 Development Trend
2.2.1 Important factors that promote the development of big data
1. The construction of big data infrastructure continues to grow: Infrastructure is the prerequisite and guarantee for the rapid development of the big data industry. my country is accelerating the promotion of the "Broadband China" strategy and speeding up the construction of network infrastructure such as the next-generation Internet, 4G communication network, public wireless network, e-government network and the Internet of Things.
2. The progress of open sharing of big data is accelerating: In the era of big data, national competitiveness will be partly reflected in the scale and activity of a country's data and its ability to interpret and use data, while national data sovereignty reflects the occupation and control of data . Therefore, in the era of big data, data sovereignty has become a game space for another big country.
3. In-depth application of government big data: Government agencies at all levels have accumulated a large amount of data in their daily management, but have not fully explored the value of these data. In the future integration of various data, government application scenarios will be more abundant. The auxiliary effect of the results of data mining and analysis on management decision-making can be gradually revealed.
4. Big data related legislation is accelerated: At present, my country has no special laws on the protection of personal data information, and the industry power and industry organizations of the big data industry are not strong enough, enterprise self-discipline is difficult to achieve, and the government's regulation and protection capabilities are not strong enough. In the future, through the establishment of a personal information and privacy protection system, a good information and privacy security environment will be created for the public. In the future, with the implementation of the national big data strategy and the implementation of supporting policies, the development environment of the big data industry will be further optimized, the demand for big data services in various social and economic fields will further increase, and the new technologies, new formats and new models of big data will Constantly emerging, the scale of the industry will continue to maintain a trend of rapid growth.
5. Technological influence trends: The technological development of big data will be more closely linked with new technology fields such as the Internet of Things, cloud computing, and artificial intelligence. The development of the Internet of Things will greatly improve the ability to obtain data, and cloud computing and artificial intelligence will deepen Integration into the data analysis system, fusion innovation will continue to emerge and continue to deepen.
6. Talent impact trend: In 2017, the Ministry of Education announced the second batch of universities approved to open "Data Science and Big Data Technology", plus the first batch of approved universities were approved to open this major. Starting from 2018, the compound talents required by big data will continue to form. Coupled with the continuous addition of cross-border talents from overseas and traditional industries to the big data industry, the big data industry will usher in innovative development.
7. Capital influence trend: In recent years, with the large amount of capital entering the big data industry, there has been a phenomenon of overvaluation of start-up companies. In the big data industry during the bubble period, the development of many enterprises was far from returning to the essence of the enterprise, resulting in a disconnect between the supply of start-up enterprises and market demand. With the sinking of capital, rational capital will lead the healthy development of the big data industry. Some enterprises that can develop along the normal track after the capital boom fades will occupy a new round of capital advantages and get further development and growth.
2.2.2 The scale of the big data application market is further expanded
. The useable value of big data is very high. For example, the scene-based application of big data analysis can help all kinds of enterprises dig out all kinds of supporting enterprises from the originally worthless massive data. The high-quality data for decision-making development can make the data change from quantity to quality, and truly generate application value. It has a strong economic driving effect. Therefore, in the next few years, the application field of big data will surely expand rapidly.
The 27 EU countries will receive an additional GDP growth of 1.9% by 2020 due to the introduction of big data. It is estimated that by 2020 the added value brought by the application of big data in the United States will account for 2%-4% of GDP in 2020. By 2020, big data will drive 2.8-4.2% of China's GDP. At present, my country's big data applications have continuously penetrated into various fields such as finance, automobiles, medical care, and service industries, and the scope of industry applications has gradually expanded. It is estimated that by 2020, my country's big data application market will reach 545 billion yuan.
3. Employment analysis
3.1 Analysis of employment prospects
First of all, let’s look at two reports,
"The 2016 China Internet Hottest Jobs Talent Pool Report" content summary:
R&D engineers are the most popular, and data analysis talents are most in short supply. The six most popular positions in the Internet industry are R&D engineer, product manager, human resources, marketing, operations, and data analysis. Among them, data analysis, as an emerging hot position, has the most talent in short supply.
Summary of the content of the "Big Data Talent Report":
At present, there are only 460,000 big data talents in the country, and the gap of big data talents in the next 3-5 years will be as high as 1.5 million.
The above reports are the talent report of the hottest Internet jobs in China released by the professional social platform LinkedIn in 2016 and the big data talent report released by Shulianxunying. The two reports indicate a theme, that is, the “scarcity” of big data talents.
PS: Job requirements of major recruitment websites,
51job big data job search, a total of 54893 jobs meet the conditions;
Zhaopin recruitment big data job search, a total of 50,000+ jobs meet the conditions;
3.2 Analysis of employment direction
As a basic discipline, big data has core technologies and job demands in the fields of data development and analysis, Internet of Things and artificial intelligence algorithm training. So what are the corresponding job positions?
1. Big data engineer, big data development engineer, big data operation and maintenance engineer, big data architect;
2. big data analyst, big data senior engineer, big data analyst expert, big data miner, big data algorithmist;
3.3 Analysis of job requirements and remuneration
What we are currently studying is the development on the hadoop cluster. Take this as an example to explain:
Hadoop development engineer:
Beijing hadoop development engineer average salary: ¥ 20130/month, taken from 1734 samples.
1. Job responsibilities of Hadoop development engineer:
a. Participate in the optimization and improvement of basic services of the data platform, participate in the optimization of the data transmission system with a daily transmission volume of more than 100 TB, the improvement of the data processing platform with a daily processing volume of more than PB level, and the construction of a multi-dimensional real-time query analysis system Optimization;
b. Construction and optimization of distributed machine learning algorithms in the data platform.
c. Deepen the source code and improve various open source big data projects (including Hadoop, Spark, HBase, etc.).
2.
Job requirements: a. Bachelor degree or above in computer or related major;
b. Familiar with development under Linux environment, proficient in more than one programming language such as C++/Java/Scala;
c. Familiar with Hadoop ecosystem related projects, proficient in one of the following projects One source code (Hadoop/
Spark/Kafka/HBase/Flume/ElasticSearch/Druid/Kylin);
d. Have good learning ability, analytical ability and problem-solving ability;
1. The basic situation of the industry
1.1 Market size
At present, the statistical caliber of the big data industry has not yet been established. As for the scale of my country's big data industry, various research institutions have adopted indirect methods to estimate. The China Academy of Information and Communications Technology combined with the survey and calculation of big data-related companies, the market size of my country's big data core industry in 2016 was about 16.8 billion yuan, a growth rate of 45% compared with 2015, and reached 23.4 billion yuan in 2017, compared with 2016 The growth rate reached 39.3%.
With national policy incentives and the gradual maturity of big data application models, China's big data market will continue to maintain a rapid growth of more than 30% per year in the next few years. It is estimated that by 2020, China's big data market will reach 57.8 billion yuan.
1.2 The main regional distribution of my country's big data industry
1.2.1 Regional distribution
my country's big data industry clusters are mainly located in economically developed areas. Beijing, Shanghai, and Guangdong are the core areas for development. These areas have well-known Internet and technology companies, high-end scientific and technological talents, and strong national policies to support the development of good information technology industries. The foundation has formed a relatively complete industrial format, and the scale of the industry is still expanding.
In addition, the big data industry circle with Guizhou and Chongqing as the center, although located in the economically backward southwest region, but Guizhou, Chongqing and other places rely on the policy guidance provided by the government for the development of their big data industry, and actively introduce big data Relevant enterprises and core talents are trying to occupy the commanding heights of the big data industry and drive the new development of the regional economy.
Relying on the advantages of the electronic information industry in Guangzhou, Shenzhen and other regions, the Pearl River Delta region has played a role in the agglomeration of the two national supercomputing centers of Guangzhou and Shenzhen. Driven by a group of backbone enterprises such as Tencent, Huawei, and ZTE, the Pearl River Delta region has gradually formed The development trend of big data aggregation.
1.2.2 Distribution of application areas
<div align="center"><font color="leight" size="3" >图2 大数据生态链</font></div>
- 1
According to the "2018-2023 China's Big Data Industry Development Prospects and Investment Strategic Planning Analysis Report" released by the Foresight Industry Research Institute, in 2017, in the "Big Data Innovation Top 100" selected by the Industrial Innovation Venture Capital Data Platform, data processing analysis The number of companies in the company reached 19, accounting for 19%, 14 financial big data, 13 marketing big data, and 5 data collection, data storage and data visualization followed closely. Data processing analysis and data visualization, as the core technology part of other big data projects, accounted for 24% of the list, indicating that the development of big data core technology is still a hot direction in the industry. At the same time, in the application field, financial big data and marketing big data together account for 26% of the share, which also indicates that these two fields will have greater development in the future.
2. Development status and trends
2.1 Development status
Big data is a new stage of informatization development. With the convergence and integration of information technology and human production and life, the rapid popularization of the Internet and the explosive growth of global data have promoted the rapid development of the big data industry. All parts of our country are very enthusiastic about the development of big data. Significant progress has been made in policy, technology, industry, application and other levels. Industry applications have been promoted rapidly, and the market scale has grown significantly. This has promoted my country to accelerate the pace of building a data power in order to achieve The manufacturing power and the network provide industry support for the call. China's big data industry market will continue to maintain rapid growth in the future. It is estimated that by 2020, the scale of China's big data industry will reach 1.010 billion yuan, with an average compound growth rate of 29.25% from 2015 to 2020.
Source: Compiled by China Electronics Information Industry Research Institute and China Commercial Industry Research Institute
2.2 Development Trend
2.2.1 Important factors that promote the development of big data
1. The construction of big data infrastructure continues to grow: Infrastructure is the prerequisite and guarantee for the rapid development of the big data industry. my country is accelerating the promotion of the "Broadband China" strategy and speeding up the construction of network infrastructure such as the next-generation Internet, 4G communication network, public wireless network, e-government network and the Internet of Things.
2. The progress of open sharing of big data is accelerating: In the era of big data, national competitiveness will be partly reflected in the scale and activity of a country's data and its ability to interpret and use data, while national data sovereignty reflects the occupation and control of data . Therefore, in the era of big data, data sovereignty has become a game space for another big country.
3. In-depth application of government big data: Government agencies at all levels have accumulated a large amount of data in their daily management, but have not fully explored the value of these data. In the future integration of various data, government application scenarios will be more abundant. The auxiliary effect of the results of data mining and analysis on management decision-making can be gradually revealed.
4. Big data related legislation is accelerated: At present, my country has no special laws on the protection of personal data information, and the industry power and industry organizations of the big data industry are not strong enough, enterprise self-discipline is difficult to achieve, and the government's regulation and protection capabilities are not strong enough. In the future, through the establishment of a personal information and privacy protection system, a good information and privacy security environment will be created for the public. In the future, with the implementation of the national big data strategy and the implementation of supporting policies, the development environment of the big data industry will be further optimized, the demand for big data services in various social and economic fields will further increase, and the new technologies, new formats and new models of big data will Constantly emerging, the scale of the industry will continue to maintain a trend of rapid growth.
5. Technological influence trends: The technological development of big data will be more closely linked with new technology fields such as the Internet of Things, cloud computing, and artificial intelligence. The development of the Internet of Things will greatly improve the ability to obtain data, and cloud computing and artificial intelligence will deepen Integration into the data analysis system, fusion innovation will continue to emerge and continue to deepen.
6. Talent impact trend: In 2017, the Ministry of Education announced the second batch of universities approved to open "Data Science and Big Data Technology", plus the first batch of approved universities were approved to open this major. Starting from 2018, the compound talents required by big data will continue to form. Coupled with the continuous addition of cross-border talents from overseas and traditional industries to the big data industry, the big data industry will usher in innovative development.
7. Capital influence trend: In recent years, with the large amount of capital entering the big data industry, there has been a phenomenon of overvaluation of start-up companies. In the big data industry during the bubble period, the development of many enterprises was far from returning to the essence of the enterprise, resulting in a disconnect between the supply of start-up enterprises and market demand. With the sinking of capital, rational capital will lead the healthy development of the big data industry. Some enterprises that can develop along the normal track after the capital boom fades will occupy a new round of capital advantages and get further development and growth.
2.2.2 The scale of the big data application market is further expanded
. The useable value of big data is very high. For example, the scene-based application of big data analysis can help all kinds of enterprises dig out all kinds of supporting enterprises from the originally worthless massive data. The high-quality data for decision-making development can make the data change from quantity to quality, and truly generate application value. It has a strong economic driving effect. Therefore, in the next few years, the application field of big data will surely expand rapidly.
The 27 EU countries will receive an additional GDP growth of 1.9% by 2020 due to the introduction of big data. It is estimated that by 2020 the added value brought by the application of big data in the United States will account for 2%-4% of GDP in 2020. By 2020, big data will drive 2.8-4.2% of China's GDP. At present, my country's big data applications have continuously penetrated into various fields such as finance, automobiles, medical care, and service industries, and the scope of industry applications has gradually expanded. It is estimated that by 2020, my country's big data application market will reach 545 billion yuan.
3. Employment analysis
3.1 Analysis of employment prospects
First of all, let’s look at two reports,
"The 2016 China Internet Hottest Jobs Talent Pool Report" content summary:
R&D engineers are the most popular, and data analysis talents are most in short supply. The six most popular positions in the Internet industry are R&D engineer, product manager, human resources, marketing, operations, and data analysis. Among them, data analysis, as an emerging hot position, has the most talent in short supply.
Summary of the content of the "Big Data Talent Report":
At present, there are only 460,000 big data talents in the country, and the gap of big data talents in the next 3-5 years will be as high as 1.5 million.
The above reports are the talent report of the hottest Internet jobs in China released by the professional social platform LinkedIn in 2016 and the big data talent report released by Shulianxunying. The two reports indicate a theme, that is, the “scarcity” of big data talents.
PS: Job requirements of major recruitment websites,
51job big data job search, a total of 54893 jobs meet the conditions;
Zhaopin recruitment big data job search, a total of 50,000+ jobs meet the conditions;
3.2 Analysis of employment direction
As a basic discipline, big data has core technologies and job demands in the fields of data development and analysis, Internet of Things and artificial intelligence algorithm training. So what are the corresponding job positions?
1. Big data engineer, big data development engineer, big data operation and maintenance engineer, big data architect;
2. big data analyst, big data senior engineer, big data analyst expert, big data miner, big data algorithmist;
3.3 Analysis of job requirements and remuneration
What we are currently studying is the development on the hadoop cluster. Take this as an example to explain:
Hadoop development engineer:
Beijing hadoop development engineer average salary: ¥ 20130/month, taken from 1734 samples.
1. Job responsibilities of Hadoop development engineer:
a. Participate in the optimization and improvement of basic services of the data platform, participate in the optimization of the data transmission system with a daily transmission volume of more than 100 TB, the improvement of the data processing platform with a daily processing volume of more than PB level, and the construction of a multi-dimensional real-time query analysis system Optimization;
b. Construction and optimization of distributed machine learning algorithms in the data platform.
c. Deepen the source code and improve various open source big data projects (including Hadoop, Spark, HBase, etc.).
2.
Job requirements: a. Bachelor degree or above in computer or related major;
b. Familiar with development under Linux environment, proficient in more than one programming language such as C++/Java/Scala;
c. Familiar with Hadoop ecosystem related projects, proficient in one of the following projects One source code (Hadoop/
Spark/Kafka/HBase/Flume/ElasticSearch/Druid/Kylin);
d. Have good learning ability, analytical ability and problem-solving ability;