General electric business evolution:
Product Details page System Architecture Evolution
The first version of the
architecture
J2EE + Tomcat + MySQL
dynamic pages, each request must invoke multiple dependent services interface, query data from the database, and then rendered into HTML template by similar JSP technology, returns the final HTML page
architecture defects
each request is to access the database, performance is certainly bad
for each request must call a large number of dependent services, the instability of commodity-dependent service details page shows the jitter performance is often
the second version of
the architecture design
static pages technology
by MQ obtain data store listing page related to the change message
via Java Worker service the entire amount invoke all dependent services interface, query the database, access to the complete data constitute a product details page, and generate static HTML by velocity and other template techniques
static HTML page via rsync tool pushed directly to servers on multiple nginx, nginx on each server has the full amount of static HTML pages
nginx access to the product details page requests directly back to local static HTML pages
add a layer of load balancing in front of nginx server device, a request should hit any With the nginx server has full amount of static HTML pages can be returned
architecture defect
whole amount update problem
if a particular commodity classification, business and other information is changed
so that classification, shops, businesses following all the product details page will need to re-generate static HTML pages
update too slow problem
classification, more and more shops, businesses, commercial
load regenerate HTML increasingly high, rsync the whole amount of synchronous load all nginx higher and higher
Change from data to generate static HTML, and then synchronize the full amount to all nginx, time more slowly
expanding problem
because each product details page full amount must be synchronized to all the nginx, can not cause the system expansion, can not increase system capacity
architecture optimization
full update to resolve the issue amount
each Java Worker receive the change message for a dimension, a dimension not got me the whole amount and generates a complete HTML, but in accordance with the dimensions split to produce a change in the dimensions of HTML fragments
nginx to merge multiple HTML fragments by SSI html fragments then outputs a complete html
solve the expansion problem
for each item details page is not synchronized to the total amount of all nginx
but according to a trade route table id nginx, simultaneous access requests in the same layer nginx logical routing
update rate is too slow problem
adding more machine resources
and more room to deploy, each room to deploy a Java Worker + application Nginx, all rooms with a load balancing equipment, the whole process is completed within each room, not across the room
architecture optimized defect
update rate not fast enough questions
for each dimension item has an HTML fragment , Rsync push a lot of HTML fragments, the load is too high, poor performance
Nginx be merged SSI based on the mechanical hard drive, performance is poor
or there is a problem of the whole amount update
solves the classification, businesses, shops dimensions of change, as long as the incremental production than re small HTML snippets can be, do not regenerate whole amount associated with all product details page of HTML
However, if a page template to change or add a new page template or HTML fragments can cause hundreds of millions of commodities must be regenerated and rsync, for a few days to complete, can not respond to the demand
there is still capacity issues
nginx storage is limited, can not be unlimited storage of hundreds of millions, and the growth of the product details page of HTML files
if nginx storage limit is reached, need to remove part of the product details page HTML file into nginx could not find HTML, it is called the back-end interface to back dynamic pages architecture
dynamic architecture pages at high concurrent access, will depend on the system caused by excessive pressure, almost could not carry
a third version
required to support the needs of
rapidly responding to new template revision and the individual needs of a variety of page templates Join
page modularity, a change in the area of the page, as long as the update data in this region to
support high-performance access to
the level of support the expansion of scalability architecture
architecture design
system architecture design
depend on data services have changed to send a message to the MQ
heterogeneous data Worker service changes in the MQ message listener call-dependent services Interfaces, there is a change only pulling data can then redis data stored in
the data stored in the heterogeneous redis the Worker, atoms are raw data, including the basic information of goods, commodities extended attributes, other product information, product specifications, product classification, business information
data message sent to the MQ Worker isomers, data aggregation Worker MQ message intercepted
data aggregation data atom is removed from the Worker redis, the dimension storage after the polymerization according to redis in three dimensions including
the basic information dimensions: basic information, extended attributes
product description: PC version, mobile
Other information: Category, business information
nginx + lua, lua data read various dimensions of goods from redis, dynamic rendering by nginx to html templates, then output the final html
how to solve all the problems
update problem: no longer is generated and push html fragments, no longer need to synthesize html, update data directly to redis, then take the dynamic rendering, performance greatly enhance the
full amount update issues: data and template separation, data updating it to update data, the template updates directly push template to nginx, not We need to re-generate all html, go directly to dynamic rendering
capacity problem: nginx does not depend on the machine where the disk space to store a lot of html, data put redis, html templates to store, greatly reducing the space occupied, and redis clusters expandable