The beginning of the site architecture
Initially small volume of business, small traffic at this time of architecture, applications, databases, files are deployed on a single server, or even just rent some hosting space
1. applications, data, document separation
Applications, databases, files, deployed on each separate server, and configured in accordance with the use of different hardware servers, to achieve the best performance results.
2. Use the cache to improve site performance
Most sites have access to follow the 28 principles, namely 80% of the access requests and eventually landed 20% of the data, so we can cache hot data, to reduce the access path of hot data, to improve the user experience. Cache achieve common way is local cache, distributed cache. Of course, CDN, reverse proxy.
2.1 local cache local cache, as the name suggests is an application server to cache data locally, can be stored in memory can also be present document components. The local cache is characterized by fast, but because of the limited space of the local cache so the amount of data is limited. OSCache is commonly used in the local cache.
2.2 distributed cache distributed cache feature is that vast amounts of data can be cached, and extended very easily, often used in the portal site in the category, there is no local cache reasonably fast speed, commonly used in distributed cache is Memcached, Redis.
2.3 reverse proxy deployed in the engine room of the site, when a user requests access to reach the first reverse proxy server, the reverse proxy server will cache data back to the user, if there is no cache data will continue to get access to the application server, doing so reduces data acquisition costs. There is a reverse proxy Squid, Nginx.
2.4 CDN assume that our servers are deployed rooms in Hangzhou, Zhejiang for users who access is faster, and for users to access Beijing is slow, which is due in Zhejiang and Beijing belong to different developed Telecom and China Unicom China, Beijing users access via the Internet need to access the router through a longer path to the server to Hangzhou, the return path is the same, so the data transmission time is relatively long. In this case, often used to solve CDN, CDN content caching data to the operator's room, starting with the most recent acquisition of data when users access operators, thereby greatly reducing the network access path. More professional CDN operators blue flood, network places.
3. Use the + cluster server load balancing to improve application performance
Application server as a portal, it will take a lot of requests, we tend to share the number of requests by the application server cluster. In front of the application server load balancing server deployment scheduling a user request, according to distribution policy to distribute requests to the plurality of application server nodes.
Commonly used hardware load balancing technology has F5, they are generally more expensive than 15W. Software has LVS, Nginx, HAProxy. LVS is four (the transport layer) load balancing, according to the destination address and the port select internal server, and Nginx HAProxy are seven (application layer) load balancing can be selected according to the internal server message contents, and therefore better distribution path LVS and Nginx HAProxy, to higher performance, and is more HAProxy Nginx and configurable, as may be used for static and dynamic separation (the feature request message, selecting static resource server or application server).
4. Database Optimization
4.1 separate read and write and sub-library sub-table with the increase of users, the database has become the biggest bottleneck, improve database performance means is used to read and write as well as separate sub-library sub-table, separate read and write the name suggests is read into a database library and write library, data synchronization is achieved by standby function. Sub-library sub-table is divided into horizontal and vertical segmentation segmentation, segmentation level is large to a database table split, for example, the user table. Vertical segmentation is to cut according to different service points, such as user services, goods traffic related tables in different databases.
4.2 NoSql databases and search engines for queries and analysis of massive data, we use nosql database plus search engines can achieve better performance. Not all data should be placed relational data. Commonly used NOSQL have mongodb, hbase, redis, search engines have lucene, solr, elasticsearch.
5. The application server service splitter
With the expansion, application business has become very bloated, then we need to split the business application, such as Baidu divided into news, web pages, images, and other services. Each business application responsible for independent business operations. Achieved between the business communication or message via the shared database.
6. Use of distributed systems
Distributed File System 6.1 users increase day by day, more and more traffic, more and more documents produced, single file server can not meet the demand, then we need to support distributed file system. Commonly used distributed file system GFS, HDFS, TFS.
GFS (Google File System) can provide higher overall performance of the service to a large number of users
• suitable for deployment on low-cost commodity hardware
• provide fault tolerance
HDFS (Hadoop Distributed File System) can provide data access to high-throughput, ideal for use on large datasets
• run general-purpose hardware (commodityhardware)
• highly fault-tolerant
• suitable for deployment on an inexpensive machine
TFS (Taobao Flies System) aimed at the mass of unstructured data, providing highly reliable and high concurrent access to storage
• high scalability, high availability, high performance
• for Internet services
• Suitable for mass storage of small files
6.2 Distributed Applications will use various business services to some basic business services, such as customer service, order services, payment services, security services, which are essential elements to support various business applications. We will use these services extracted Division-service framework to build distributed services. Ali Dubbo is a good choice.
summary
Complete system architecture is as follows:
Note: Large site architecture is based on the business needs of continuous improvement, according to different business characteristics and will do specific design considerations, this article only describes some of the optimization techniques and means a conventional large-scale sites involved.