This chapter Jingdong product details page, for example, Jingdong product details page, although only a single page, but the source data aggregation is very much, except for some relatively high real-time requirements, such as price, inventory, service support asynchronous loading loaded via AJAX , the other in the back-end data are then assembled for data aggregation page template.
As shown, the main product page includes basic information commodity (basic information, a list of images, color / size relationship, extended attributes, specifications, packing lists, after-sale protection, etc.), product description, additional information (classification, brand, shop [ third-party seller], [third-party seller store] classification, similar related brands). More details here is not elaborated.
Jingdong hundreds of millions throughout the commodity, as if every dynamic access to content template assembly, enough to cause as many data sources can not meet the performance requirements; initial solution is to generate static pages, but the biggest problem of static pages: 1, can not be quickly page response requirements change; 2, it is difficult to do more online comparison test version. As two factors is sufficient to restrict the development of commodity diversification pages, so the technology is not very good program static.
Through analysis, the data are divided into four categories: basic information product page, product description (asynchronous loading), additional information (classification, brands, shops, etc.), other needs of real-time data display (price, inventory, etc.). While other information such as classification, brand shop is very small, you can put a small memory footprint Redis stores; and we can learn basic information goods static technology will do the aggregation of data storage, so that the benefits of data is atoms, while the template is variable at any time, absorb the advantages of static page aggregation, more than made up for the shortcomings of static version of the page; Another very serious problem is heavily dependent on these related systems, if they hang up or slow response to the commodity page hung up or slow to respond; we also introduce merchandise through AJAX techniques lazy loading (because it is the second screen, only when the user rolls the mouse to the screen when the display); and display real-time data to make asynchronous loading via AJAX technology; therefore we design can do the following:
接收商品变更消息，做商品基本信息的聚合，即从多个数据源获取商品相关信息如图片列表、颜色尺码、规格参数、扩展属性等等，聚合为一个大的JSON数据做成数据闭环，以key-value存储；因为是闭环，即使依赖的系统挂了我们商品页还是能继续服务的，对商品页不会造成任何影响；
接收商品介绍变更消息，存储商品介绍信息；
介绍其他信息变更消息，存储其他信息。
整个架构如下图所示：
技术选型
MQ可以使用如Apache ActiveMQ；
Worker/动态服务可以通过如Java技术实现；
RPC可以选择如alibaba Dubbo；
KV持久化存储可以选择SSDB（如果使用SSD盘则可以选择SSDB+RocksDB引擎）或者ARDB（LMDB引擎版）；
缓存使用Redis；
SSDB/Redis分片使用如Twemproxy，这样不管使用Java还是Nginx+Lua，它们都不关心分片逻辑；
前端模板拼装使用Nginx+Lua；
数据集群数据存储的机器可以采用RAID技术或者主从模式防止单点故障；
因为数据变更不频繁，可以考虑SSD替代机械硬盘
核心流程
首先我们监听商品数据变更消息；
接收到消息后，数据聚合Worker通过RPC调用相关系统获取所有要展示的数据，此处获取数据的来源可能非常多而且响应速度完全受制于这些系统，可能耗时几百毫秒甚至上秒的时间；
将数据聚合为JSON串存储到相关数据集群；
前端Nginx通过Lua获取相关集群的数据进行展示；商品页需要获取基本信息+其他信息进行模板拼装，即拼装模板仅需要两次调用（另外因为其他信息数据量少且对一致性要求不高，因此我们完全可以缓存到Nginx本地全局内存，这样可以减少远程调用提高性能）；当页面滚动到商品介绍页面时异步调用商品介绍服务获取数据；
如果从聚合的SSDB集群/Redis中获取不到相关数据；则回源到动态服务通过RPC调用相关系统获取所有要展示的数据返回（此处可以做限流处理，因为如果大量请求过来的话可能导致服务雪崩，需要采取保护措施），此处的逻辑和数据聚合Worker完全一样；然后发送MQ通知数据变更，这样下次访问时就可以从聚合的SSDB集群/Redis中获取数据了。