Link: https://reurl.cc/qg5gQg From: Open Source Linux
Douyin, Baidu, Alibaba Cloud, and Tencent are all self-built data centers, all of which are T-level export bandwidth (total export bandwidth), that is, the export bandwidth of 1T=1024G/s, and the total number of servers is basically more than 200,000. , and even Alibaba Cloud has more than 1 million units.
The total bandwidth of ByteDance's data center may be around 10TB, and it is not far from the expected breakthrough of 15TB.
In general, the total egress bandwidth is 1TB, and the actual egress bandwidth of the equipment room may only be up to 100G. This is a dual (multi) link design. The dual egress realizes dynamic traffic sharing, and the total egress bandwidth can reach T level.
large data center
If you want to have hundreds of millions of people online at the same time, TB-level bandwidth, CDN acceleration, multi-node, load balancing and other technologies are indispensable. (This design technology is too complicated, if you have relevant professional friends, you can comment on a brief overview)
CDN acceleration
How many servers does ByteDance have?
Organized according to network data (data from the network)
In 2017, there were 20,000 to 30,000 servers. At this time, it was mainly leased servers.
In 2018, ByteDance built its own data center. The largest data center is in Guantinghu New Media Industrial Park, Huailai, Hebei, with 50,000 servers in the first phase. 90,000 servers in the second phase.
In 2018, the number of leased + self-built servers reached 170,000 servers.
In 2020, according to data from ByteDance's recruitment announcement, there are 420,000 servers. 1.5 times more than in 2018.
10-20 servers in a rack
This part of the server is used in China, and mainly uses domestic products such as Douyin, Xigua Video, Toutiao, Feishu, etc.
TIKTOK in the United States is an independent operation, and data is stored and distributed locally in the United States.
TIktok also leased nearly 100,000 servers in the US in 2020
According to data released by Business Insider, in the first half of 2020, ByteDance leased a data center with energy consumption of 53 megawatts in northern Virginia. It can accommodate hundreds of thousands of servers and cover an area of up to hundreds of thousands of square feet.
TikTok is investing in data centers in India and Singapore.
After talking about the number of servers, let's talk about the hard core: What is the exit bandwidth of ByteDance's large data center?
Knowledge point: The so-called export bandwidth is actually what ordinary people call download bandwidth. It is the total speed at which the server distributes data to each mobile phone.
Under normal circumstances, small IDC companies build their own computer rooms, such as some website companies that rent computer rooms from China Unicom, China Mobile, and China Telecom, and their overall export bandwidth may only be 5G. More than 30G is a company with a certain scale. The network company's revenue is less than tens of millions of enterprises.
Therefore, it is often seen that some enterprises with a good scale basically no longer build their own computer rooms and use cloud hosts. For example, Alibaba Cloud's ECS, Tencent Cloud, Baidu Cloud, AWS (Amazon). Generally, a corporate website (enterprise official website), 20M bandwidth, 4G memory, 100G hard disk, the price of 4000-5000 yuan a year is enough, and the price may be cheaper to catch up with the event.
The bandwidth is the most expensive. Of course, to increase the bandwidth to a certain level, for example, if the access volume increases, it is necessary to increase the memory and hard disk. In contrast, if the bandwidth is increased, the cost is more expensive. [Here I will explain to you: bandwidth is relatively expensive and is a scarce resource]
Let's take a look at a data center of China Mobile: China Mobile (Shijiazhuang, Hebei) data center:
It covers an area of 174 acres and has a total construction area of 130,000 square meters. It is planned to have 10 single buildings, which can provide an installed capacity of about 30,000 racks when all completed. The three IDC rooms can provide a total of 31,000 racks of cabinets and 15T bandwidth resources.
In a cabinet, the number of full 1U devices generally does not exceed 16, the full 2U devices generally do not exceed 12, and the full 4U devices generally do not exceed 4 to 7. We take the high-performance 2U and 4U servers for an average trade-off, and each is calculated by half (after all, mobile is considered to be a wealthy person and cannot buy low-end 1U devices). Then 31,000 racks can be installed, with a maximum of 210,000-360,000 servers.
Here is a rough average: 300,000 servers. Enjoy 15T export bandwidth resources. Of course, as the backbone network of telecommunications, mobile bandwidth resources are definitely more advantageous than ByteDance.
Therefore, we roughly estimate ByteDance's self-built data center with 170,000 servers. The total egress bandwidth may be around 7Tb-10TB. Basically, a dual egress traffic design will be adopted, coupled with a multi-link deployment method: the actual egress bandwidth can be 800G-1TG to achieve a total egress bandwidth of about 10T. 【This is the technical way】
ByteDance 2018 Hebei Huaihua Data Center
What is the concept of T-level export bandwidth? If I tell you that in 2009, the export bandwidth of the entire Shanghai was only 1140G, which just reached 1TB. In just 10 years, the egress bandwidth of an enterprise's data center exceeds 1Tb, which is really unimaginable.
You must know that in 2009, although smartphones were underdeveloped, personal PC sales were already huge.
CDN acceleration, let the public swipe Douyin, and watch videos are not stuck
Knowledge point: CDN (Content Delivery Network, Content Delivery Network)
Publish the content of the server to the edge node closest to the user, so that the user can obtain the desired content nearby.
Solve the Internet network congestion and improve the response speed of users visiting the website. A collection of various accelerated solutions.
Explaining CDN in common words is: some pages will be specially compressed, and some will be compressed as static pages, and the static pages will be distributed quickly. Users can see the content within 2s, and the experience is better. [This is a static transmission]
For dynamic video, first through intelligent routing, find the best path, and then protocol optimization compresses long connections and content to remove redundancy. [This is dynamic compression]
Let's take a look at Tencent's 500 million daily active users in 2015, a CDN level that integrates music, instant messaging and other products, reaching a bandwidth of 10TB. Trillions of requests per day.
Level of Tencent CDN in 2015
Therefore, I say here that ByteDance's overall server has 10TB, which should be not much. After all, Douyin has 600 million DAUs, and Xigua Video + Today’s Toutiao roughly counts 200 million, with a total of 800 million DAUs.