twitter system architecture analysis
(A) core business twitter
twitter core business, and that following the Followed BE:
(1) following- attention
into the personal home page, you will see a message you follow people published (no more than 140 words), which is following the process;
(2) followed- attention
you post a message, follow your people will see this message, it is be followed in the process;
(B) the business logic twitter of
the business logic is not complicated twitter
following operations, check who follow the message, and who published;
the Followed business, front-end back-end js polling, see follow the people have no new messages, there is updated (updated in time depends on the polling time);
(C) three-tier architecture (three-tier architecture)
website architecture, the traditional approach is three-tier structure, the so-called "traditional" does not mean "out of date", trendy immature technology, the traditional way is more robust.
(1) the presentation layer (presentation tier): apache web server , the main task is to parse the http protocol, a request to the logical layer;
(2) logical layer (logic tier): mongrel rails server , using rails ready module, to reduce the operating amount;
(3) the data layer (data tier): mysql;
Data layer to come:
the core twitter is (1) a user; (2) message; (3) user relationship;
surrounding these core, schema design data of the core:
(1) user table User
ID, name, Pass , Status, ...
(2) table message MSG
msgid, the author_id, MSG, Time, ...
(. 3) user relationship table relation
ID, following_ids, followed_ids
Logical layer :
When a user issues a message, performed sequentially:
(1) to the message msg table stored;
(2) the user check the relation table, find out followed_ids;
(. 3) acquired in the user state followed_ids;
(4) line ids , push the message into a queue queue;
(5) in the queue msg, ids update the home page;
there is a queue to use, there are many ways to achieve it, such as apache mina, twitter own team achieved a kestrel.
Presentation Layer :
a main layer has two functions:
(1) processing HTTP protocol (HTTP Processor);
(2) the distributor (dispatcher);
of course, more than just a browser to access twitter, there may be cell phones, due to possible There are other protocols, so there may be other processor.
In any case, clear architectural framework as follows:
Figure 1: Schema Version 1
(D) cache = cash income that is equal to the cache
using a cache architecture is essential for large sites, the site is user response time to experience the most significant factors, which affect the response speed is the biggest enemy of disk io.
twitter engineers believe a good experience for the site average response time should be around 500ms, the ideal time is 200-300ms.
On the use of cache, and is a major part of twitter architecture with clear cache architecture as follows:
Figure 2: With version 2 cache architecture
Where required cache? IO more frequent the place, the more need to cache.
IO is the most frequently accessed database, the three core table if it is necessary to put in memory?
twitter approach is to split the table, in which the most frequently accessed fields loaded cache.
(1) vector cache and row cache line that is an array of cache and cache
the Vector cache : newly published messages msgids, related to the author's ids, id these high frequency of visit, store them in the cache cache called the Vector;
Row cache : Message Body Cache lines;
limited memory, priority vector cache, the actual result vector cache hit rate is 99%, row cache is 95%;
(2) fragment cache and page cache
user access in addition to twitter page (web channel), as well as mobile phones (API channel), while the proportion of the latter's total traffic 80% -90%.
addition mysql cache, cache API gravity will be in the channel.
The main phone screen, a screen is a screen of a message, it may be the entire page is divided into several local, each corresponding to a number of local / message, which is that fragment.
High popularity of the cache fragment of its pages, you can read its news release to improve efficiency, this is the fragment cache mission.
Author popularity, people will visit its home page, which is page cache mission.
Actual results, fragment cache hit rate of 95%, page cache is 40%.
Although the page cache hit rate, but because it is accessing the home page, the space it occupied is great, influence each other in order to prevent two kinds of cache, both cache needs to be deployed on different physical machines.
twitter The fragment cache and page cache are using memcached.
(3) http accelerator
cache problem web channels need to be addressed, after analysis, the pressure mainly from the web channel search.
When faced with unexpected events, readers can search for relevant information, and does not care about the authors of these information are those who are not of their own follow.
In order to reduce the pressure of the search, keyword search and search content can cache up here, twitter engineers use a varnish.
Interestingly, varnish usually deployed in web server layer, the first visit varnish, which is not related to the content, just visit web server;
twitter engineers puts varnish on the inner apache web server, because they believe varnish operations complex, fear of paralysis caused by the collapse of varnish system, it adopted this conservative deployment.
twitter no public varnish hit rate, they claimed, after the use of varnish, the whole point of load dropped by 50%.
(E) require isolation flood
twitter architecture is another major part of its message queue: isolating a user's operation, the peak flow rate smoothing.
When the restaurant is full, for new customers, although not serve, but not shut out, but now let them lounge to wait.
When a user accesses twitter, received him is apache web server, but apache could not receive an unlimited number of users.
January 20, 2009, Obama delivered his inaugural address, twitter traffic soared, then what to do.
Facing flood peak, how to ensure that the site does not rush collapse? Quickly accepted, but delayed service.
apache receipt of the request, forwarded to Mongrel, Mongrel responsible for the actual processing, apache the free hand to greet the next user.
However, the number of users able to receive apache is always limited, it is subject to the number of concurrent apache can accommodate the number of work processes, there is not a closer look inside the apache principle is as follows:
Figure 3: apache internal architecture
(Vi) data flow and control flow
rapid acceptance, delayed service, just a delaying tactic, to let users will not receive the 503 (service unavailable).
Flood real ability is reflected in the flood Flood two aspects:
(. 1) there is a huge Twitter memcached cluster can flood a large capacity;
(2) Twitter kestrel own message queue, as spillway drainage means, a control command is transmitted (drainage and channels);
when reaching the peak, twitter control data flow, the data to a plurality of machines to evacuate, to avoid stress concentration, resulting in paralysis of the system.
The following are examples twitter internal processes, assuming that there are two authors, through the browser message, a reader also read their messages through a browser.
FIG 4: twitter stream
(1) Log apache web server, apache assign a worker process for its services, log in, check the id, write cookie, etc.;
(2) upload new messages written, put forward the author id, messages, etc. to Mongrel, apache wait Mongrel reply, On the home page in order to update, the new update message written up;
(3) Mongrel after receiving the message, assign a msgid, and it kept on holding id and other vector memcached caching to go up;
at the same time, Mongrel let vector memcached to find the author who is follow, if there is no cache hit will go to the back-end mysql look, incorporating cache;
readers ids returned to the Mongrel, Mongrel put msgid SMS with the text cache to memcached Row;
(4) Mongrel notification kestrel message queue server, each author and reader there is a queue (not create);
Mongrel will msgid readers into the queue, as well as the author of the queue;
(5) a table Mongrel, it probably dealing with a particular queue id, it will return to the user id Add on the provision of information on the home page;
after (6) Mongrel is updated to the front of the home waiting for the apache, apache is back Browser.
(Vii) peak and cloud computing
not elaborate, when the flood peak could not carry, can only add machine.
Machine where they come from? The company's equipment rental platform cloud computing.
Of course, only you need to rent equipment at the peak, save money ah (question: twitter how to know when is the peak?).
(Viii) push and pull of compromise
can be seen, Mongrel workflow:
(1) the relevant ids into the vector memcached and row memecached even news release success, not responsible for posting the mysql database;
(2) the relevant msgid even if the message into the message queue kestrel push success;
Mongrel does not use any means to inform authors, readers and let them re-pull message.
The above work, reflecting the architectural design concept twitter "spin-off" of:
(1) a complete process will split into independent sub-processes work, a work by each service may be responsible (three-tier architecture itself is a spin-off );
collaboration (2)
twitter separate business processes, is an event-driven design, mainly in two aspects:
(1) separation Mongrel and mysql, the former does not directly intervene in the operation of mysql, memcached commissioned solely responsible;
(2) upload, Download logical separation: kestrel only to pass through the instruction queue;
