A few days ago because of Lu Han Guan Xiaotong events that led to the collapse of Sina microblogging few minutes. This happens more than once, but apparently not to deal with Sina, could not help but think of it flow dual 1.1 billion, Taobao will also add resources to address the prediction of high concurrency problems, but not so much so traffic in normal times , so it is impossible to always save resources, obviously cost a lot of ah. The Sina's approach is also temporarily on loan from Ali cloud server 1000, withstood the pressure concurrent, returned after a few hours. Taobao pairs 11 to practice and like, but Sina microblogging This is complicated by unpredictable, no one thought. . .
Temporary additional 1,000 servers, withstood the pressure, the database can not be done so fast temporary elastically stretchable, elastic stretch can do nothing more than Http Server, middleware and so on.
Analysis of a large peace of God (: https: link //www.zhihu.com/question/66346687) looked know, personally think it microblogging automatic expansion algorithm to do better, when a sudden increase in traffic, and it takes a lot of units when the server, not all algorithms to do, we need to manually confirm, otherwise the event of God horse bug, how much money it lost.
The following is Sina microblogging platform architecture, turn ~
Preface
Sina Weibo published monthly in March 2014 active users (MAU) has reached 143 million, 808 298 2014 New Year's micro-Bardon sent the first minute, such a huge user base and traffic, require high availability (HA) high concurrent access, low-latency powerful back-end system support.
Microblogging platform first-generation architecture is the LAMP architecture, the database is MyIsam, background using a php, cache Memcache.
With the growth of application scale, derived from the second-generation architecture for business functions modular, component-based and service, back-office systems from php replaced by Java, gradually formed SOA architecture, for a long time to support the micro-blog platform business development.
And after a long period of reconstruction, based on this line run, thinking and precipitation, the formation of a third-generation platform architecture system.
We look at a micro-Bo core business diagram (below), it is not very complicated? But this is no longer a simplified diagram of a simplified business, third-generation technology system is to guarantee fast, efficient and reliable release of new products and features on the microblogging core business.
Third-generation technology system
Twitter third generation technology platform system model using orthogonal decomposition method: in the horizontal direction, using a typical three hierarchical model, i.e., an interface layer, a service layer and resource layer; in the vertical direction is further subdivided into business architecture, technical architecture, monitoring platform and service management platform. Here is the overall platform architecture diagram:
As shown above, the whole orthogonal decomposition exploded 3 * 4 = 12 regions, each representing a horizontal dimension and a vertical dimension of the intersection, the core function points define the respective areas, such as the main area 5 complete the technical infrastructure services layer.
The following details the design principles of horizontal and vertical direction, in particular, will focus on technology components 4,5,6 and its role in the overall architecture of the system.
Hierarchical level
The horizontal dimension of the divide, Internet business background in the design of large and medium systems very basic, are reflected in each generation technology system platform. Here is a brief, explanation as to extend the vertical dimension of the subsequent groundwork:
Interface layer mainly implements the interface to interact with the Web page, mobile client, the definition of a uniform interface specification, platform core of three interface services are content (Feed) services, customer relationship services and communication services (single private letters, mass, group chat).
The core business of the service layer mainly modular, as a service, where the service is divided into two classes, a class of service for the atom, which is defined not dependent on any other service modules and services, such as short chain common services, service numbers are issued fall into this category. FIG isolated using lane, showing their independence. Another class of service combination, to complete the service by combining various atomic services and business logic, such as Feed services, communication services, in addition to their own business logic, also on short-chain, and Fa user service.
The main resource is stored data layer model, with generic cache resources Redis and Memcached, and persistent storage database MySQL, HBase, a distributed file system or TFS and Sina S3 service.
A characteristic of hierarchical level, dependency is from the top down, the underlying services depend on the upper, lower service does not depend on the top, to build a simple, direct dependencies.
Corresponding to the hierarchical model, the server Twitter system mainly includes three types: a headend (API provides interface services), the queue unit (uplink service logic processing, data write mainly) and storage (mc, mysql, mcq , redis, HBase, etc.).
Vertically extending technical architecture
With the development and optimization of business structure, research and development platform implements many outstanding middleware products, used to support the core business, which generated the middleware business is driven by, as more rich technology components to form a complete platform technology framework, greatly enhance the efficiency of product development and business operations platform stability.
Different from the horizontal upper lower-dependent relationship, the vertical direction to the technical framework for foundation support and the impact of business structure to both sides of the drive, monitoring platform, service management platform, Here's what one of the core components.
Interface Layer Web V4 framework
Interface to simplify and standardize the business framework interface development work, the common interface layer function packed into the frame, using the Spring aspect-oriented (AOP) design. Interface Jersey-based secondary development framework, based annotation defined interfaces (url, parameters), built-Auth, frequency control, access logs, downgrade, support interface layer monitoring platform and service management, as well as automated Bean-json / xml Serialization.
Service layer framework
Mainly related to the service layer framework RPC remote call and message queue framework, which is in the service layer microblogging platform the most widely used two frames.
MCQ message queue
Message Queue provides a first-in first-out communication mechanism within the platform, the most common scenario is the landing operation of asynchronous data written to the queue, the queue handler bulk read and write DB, asynchronous message queue mechanism to provide accelerated response time front-end machine, and secondly, the bulk of DB operation also indirectly improve the operational performance of DB, another scenario, platform, providing real-time data to search, big data, business operations through a message queue.
Internal microblogging platform heavy use of MCQ (SimpleQueue Service Over Memcache) Message Queue service, based on MemCache protocol, message data persistence writes BerkeleyDB, only get / set two orders, but also very easy to do monitor (stats queue), there is rich client library, online operation for many years, performance is many times higher than the general MQ.
Motan RPC framework
Microblogging Motan RPC service, the underlying communications network engine uses Netty framework, and Hessian protocol support serialization Java serialization, communication protocol support Motan, http, tcp, mc and so on, Motan within the framework of large-scale use, robustness in the system and service governance, there is a more mature technology solutions, the robust, based on Config configuration management service implements the High Availability and Load Balance policy (support flexible FailOver and FailFast HA policies, and Round Robin, LRU, Consistent Hash, etc. Load Balance strategy), service governance, generate a full service call chain data, service request performance data, response time (response time), QPS and standardization Error, Exception log information.
Resource Layer Framework
Very many resources framework layer, there are packages of MySQL and HBase Key-List DAL middleware, there are customized component count, there Proxy support distributed MC and Redis, the industry in these areas have more experience sharing, I here to share platform architecture object library with SSD Cache components.
Object Library
Convenient data object library supports object serialization and de-serialization of micro-Bo: When serialized, the object serialization JVM memory is written in HBase and generates a unique ObjectID, when you need to access the object, by ObjectID read, object library to support any type of object, support PB, JSON, binary serialization protocol, the largest microblogging microblogging application scenarios will be referenced in the videos, pictures, articles unified defined as an object, has defined dozens object type, and a standard abstract object metadata Schema, content objects uploaded to object storage system (Sina S3), the object metadata stored in Sina S3 download address.
SSDCache
With the popularity of SSD drives, making it an excellent IO performance is increasingly being used to replace traditional SATA and SAS disk, there are three common scenarios: 1) to replace the hard disk MySQL database currently no community for SSD optimized version of MySQL, even so, a direct upgrade SSD drives can also bring about 8 times the IOPS upgrade; 2) replace Redis hard to improve its performance; 3) used in the CDN to speed up the loading speed of static resources.
Twitter platform in the distributed cache SSD application scenario, the traditional Redis / MC + Mysql embodiment, extended Redis / MC + SSD Cache + Mysql embodiment, the Cache SSD is used as the L2 cache, the first reduced MC / Redis cost too small capacity issues, but also solve the access database DB bring pressure to penetrate.
Vertical monitoring and control services
With the size and business services become more complex, even if the business architect is difficult to accurately describe the dependencies between services, operation and maintenance management services become increasingly difficult, in this context, reference google's dapper and the twitter of zipkin, the platform implements its own large-scale distributed tracking system WatchMan.
WatchMan large distributed tracking system
As with other large and medium sized Internet applications, microblogging platform consists of numerous distributed components, each user by the HTTP browser or mobile client application server requests arrive, will go through a number of business system or system component, and leave footprints (footprint). But these scattered data for troubleshooting, or process optimization are limited help. For such a typical cross-process / inter-thread scene, collect and analyze summary of these logs is particularly important. On the other hand, collect performance data every footprint, according to the policy and do flow control various subsystems or downgrade, are also important factors microblogging platform to ensure high availability. To be able to do a complete call tracking link for each request; collect performance data for each service call on the link; you can keep track of all of Error and Exception system; by calculating the ratio of performance data and performance metrics (SLA) and then back to control flow (control flow) based on these objectives was born microblogging Watchman system.
A core of the system design principle is minimally invasive (non-invasivenss): As a non-service component, should be less invasive or not invasive as other business systems, maintaining transparency to the consumer, it can greatly reduce the burden on developers and access threshold. Based on this consideration, all the log collection points are located in the middleware technology framework, including the interface frame, RPC middleware framework and other resources.
WatchMan by the technical team to build a framework for application in all business scenarios, this wiki transport systems to improve monitoring platform, operation and maintenance of common services and the use of this system, the completion of distributed services management, including service expansion and volume reduction, service degradation, traffic is switched service publishing and gray.
end
Now, the technology platform framework play an increasingly important role in driving the technology platform upgrade, business development, system operation and maintenance services, this article is limited to space limitations, did not begin to reports, follow-up will continue to introduce the core design principles of middleware and system architecture.
This article first appeared in the "microblogging platform architecture" micro-channel public number, a small number of editorial refinements and adjust the time of publication.
About the Author
Wei Xiang Jun (@ Guardian to the military _ microblogging), graduated from Beijing University of Posts and Telecommunications, is currently microblogging platform architect, he has engaged in technology research and development work at Microsoft, Jinshan Yun, Sina Weibo, focus on system architecture design, audio and video communication systems, distributed file systems and data mining and other fields.
----------------
Disclaimer: This article is CSDN blogger "immortal water flooded" original article, follow the CC 4.0 BY-SA copyright agreement, reproduced, please attach the original source link and this statement.
Original link: https: //blog.csdn.net/stone_yw/article/details/78189485
