On one, and share the 12306 architecture optimization ideas, the current peace share YouTube architecture, reading this article you will learn early what it is like YouTube architecture, in order to enhance their confidence in the site architecture design.
YouTube site architecture tucao (on)
YouTube as a multi-billion-level traffic video site, its site maintenance personnel has few, how these technicians are designed YouTube architecture, it has such a powerful compression capabilities of it, we then look down.
The core technical points
1) Apache: Site Server
2) Python: Web application is mainly used Python buttoned
3) Linux (SuSe): Operating System (author: Why choose SuSe it)
4) Mysql: Database
5) psyco (author: psyco is a C language extensions to Python)
6) lighttpd: no video server with Apache, choosing instead to use the lighttpd
Web service points
1) Use NetScalar load balancing and caching static content (author: NetScalar is a Web application optimized solution, commonly used in Web application acceleration, load balancing, Web security features, etc.)
2) the Apache use mod_fast_cgi mode (author: fastcgi CGI model is a resident of the process, mainly to solve the "fork and execute" model of traditional CGI model is criticized)
3) consists of a Python service responsible for routing Web requests (writer: Nginx has not become very popular at that time )
complex calculations 4) CPU-intensive, use psyco to get (I used C language: Python this is really the right technology selection it)
5) Web page caching layer do
6) do data caching data layer
7) for some data, calculated in advance, and the formation of a local cache, but no large-scale application of this optimization
Web service effectiveness
1) can be horizontally by adding Server Extensions
2) Select Python, because of its high development efficiency, the era of the Internet, rapid iteration and the ability to frequently issued, you know (I: Why not choose php it)
3 ) Python code efficiency is not the bottleneck, the bottleneck in Web services RPC requests (author: meaning, bottlenecks in Web applications or back-end)
4) response time control in less than 100ms (author: not reviewed)
Video service points
1) each video stored on more than one machine, but in a small cluster A benefit of clustering is storage: higher read performance B: availability high C: online data backup (author: that era, replica of the video is more advanced technology)
2) use as a Web server lighttpd video, lighttpd has the advantage of a: Apache too B: there epoll mode C: how the process model, however, YouTube hopes to handle more concurrent connections
3) on the popular video on CDN
4) popular video, in this case PV is lower than 20 video, the use of technology to optimize XXOO (author: this place does not understand, the original is "Less popular content uses YouTube servers in various colo sites ", this" colo "did not understand what that means, if you know, please message to me)
5) due to the special nature of the video, minimize between the server and end-user devices such as routers and switches
6) with SATA disks random seek optimization
Video preview point
1) with a single cluster storage preview (author: you guess right, the preview is the existence) on the file system
2) use squid as an Apache front end (author: squid is a popular Web server anti to the agent, often used for static file access acceleration, understood as a cache service bar)
3) ready to use in the future to do the preview BigTable storage
Database points
1) database using Mysql, course, it is only used to store metadata
2) and other sites as applications, YouTube through a single, master-slave, the process of segmentation level
3) primary server machine hardware better conditions , using multi-process multiple instances pattern; almost machine using hardware condition from the server, using the singleton pattern
4) separate read and write
Lessons learned
1) insisted that victory, innovative solutions to solve short-term problems of risk, need to be careful. Has always insisted, must be able to find long-term solutions
2) to find major contradictions and focus resources to address
3) selective cooperation, do not be afraid to outsource a key part of the project, such as YouTube's CDN
4) the Keep the Simple IT, from simple ideas, not much to say
5) data are separated, split level
6) iteration bottleneck optimization, including software layer, operating system layer, hardware layer
7) is the cornerstone of a successful team
