High concurrent high availability, O2O, micro-services architecture with learning website https://www.itkc8.co m
Thank you very much http://www.cnblogs.com/skyblog/p/5044486.html
About architecture, I believe that is not the more complex the better, but on the contrary, is the last word too simple to mention here now. It is also popular micro service to the cause, take a look at the market, there have service architecture: EJB, SCA, Dubbo, etc., advanced services than micro, micro services than perfect, but none of them have caught on so micro-services, because they are too complex. Simple is high-tech, Apple phone is said to have a special team to study how to make the user more simple operation. Large companies are developed by a small company, if the company feels a little time and effort would not choose the framework, and the development of small companies to large companies at the beginning of the process of technology selection, generally accompanied by continuous optimization of process systems while constantly optimize tend not to choose to develop technologies and frameworks, but the original foundation improvements, and maybe that is the essence of popular simple framework.
Suppose we need to design a "Internet +" Insurance platform for ultra-high volume of business insurance agency business. Assuming this insurance online insurance agency business scale of 20 million registered users, stores and franchisee sales 20,000, an amount of 200 million one-year warranty (China Ping total subscribers reached 167 million, with more than 798,000 life insurance sales staff about 246 000 full-time employees as of June 30, 2015, the Group's total assets amounted to 4.63 trillion yuan, attributable to shareholders' equity was 3,311.90 billion. at present, the Internet leader Zhong An Insurance insurance, mainly operating in microfinance Since backed by Alibaba, daily sales of up to 100 million insurance policy, but others find it difficult to copy Zhong An insurance mode). So we take the big Internet companies and public safety trade-offs to consider this insurance the insurance O2O platform.
l needs analysis
Reference insurance business related documents (incomplete documentation), to obtain the following core demand matrix (because it involves too many features, just take a big function points).
| classification |
Features |
quality |
constraint |
| E-commerce (B2C) |
Products (search, display and other details) |
Timely response, security, robustness, ease of use |
Various types of insurance, treatment may be different |
|
|
Buy (submit orders, payments) |
Timely response, security, robustness, ease of use |
Various types of insurance, treatment may be different |
|
|
User Center (my policy, my claims, etc.) |
Timely response, security, robustness, ease of use |
Various types of insurance, treatment may be different |
| Management agents (franchisees management) |
Insurance auto insurance (Inquiry, recorded single, payment) |
Timely response, robustness, scalability |
Various types of insurance, treatment may be different |
|
|
Non-insurance auto insurance (Inquiry, recorded single, payment) |
Timely response, robustness, scalability |
Various types of insurance, treatment may be different |
|
|
Query Policy |
Timely response, robustness, scalability |
|
|
|
Document management |
Timely response, robustness, scalability |
|
|
|
My Account (my policy, commission settlement) |
Timely response, safety, reliability, ease of use |
Various types of insurance, treatment may be different |
| Docket Management |
Docket Entry |
Timely response, robustness, scalability |
Various types of insurance, treatment may be different |
|
|
Collect claims data |
Timely response, robustness, scalability |
|
|
|
Transfer files |
Timely response, robustness, scalability |
|
|
|
Tracking files |
Timely response, robustness, scalability |
|
| Customer Management |
Customer information maintenance |
Timely response, robustness, scalability |
Upload a large file |
|
|
Customer Activity Management |
Timely response, robustness, scalability |
|
|
|
Opportunity Management |
Timely response, robustness, scalability |
|
|
|
My Platform (news, events, business opportunities) |
Timely response, robustness, scalability |
|
| Insurance adjuster |
Auto insurance to assess the damage process of tracking help |
Timely response, robustness, scalability |
Various types of insurance, treatment may be different |
|
|
Chuxian assist people injured |
Timely response, robustness, scalability |
Various types of insurance, treatment may be different |
|
|
Legal Aid Services |
Timely response, robustness, scalability |
Various types of insurance, treatment may be different |
O2O concept from the point of view: " O2O namely Online To Offline, also about opportunities at the line of business and the Internet combined together, make the Internet a front office trading of the line so you can use the service offline to online Showmanship consumption. can be screened using online services, and settlement of transactions online, quickly reach the scale of the model is the most important features are: promotion effect can be found, each transaction can be tracked (Baidu Encyclopedia) ", not every kind of service are for O2O. Commodity sales is not suitable O2O, because you can buy at the store do not need to line directly from the shop. But under Insurance Services does need to combine offline and online, if it is pure line will not be able to meet customer demand for services. O2O the line services can be franchisees, agents, may also be outlets.
The above requirements are made in accordance with the user's perspective, although the use of the word "system", but the system here is an abstract concept that may include software systems and personnel, system, etc.. The above requirements can be divided into three categories, one is purely for end-user online services: e-commerce sites (B2C); one is specifically for the agent services: an agent system; the rest is some public service, it is possible to e-commerce sites and agents of the system will be used for some services. In addition the insurance business of the biggest problems is more insurance, handling different types of insurance are very different, with the difference between ordinary large e-commerce sites, such as Lynx, all goods are the same the next single, the same service (mainly It is to express this), and even insurance products online at B2C website also different single operation, short insurance, life insurance, auto insurance, etc., not to mention there are a lot of insurance service, these service more different. Traditional insurance companies in dealing with this in general, will do more than one system, such as life insurance system, insurance system and so on.
l Systems Analysis
Installation business scale analysis we mentioned earlier (assume) some of the more important performance requirements:
a) products: continue on the line of insurance products not currently
b) B2C website daily visits: 50 million PV
c) B2C product purchase concurrent peak: 2000 TPS
d) operation and maintenance system at the same time online: 10000 (20000 Total salesperson or agent)
e) operation and maintenance system of concurrent peak: 2000 TPS
f) Short Insurance Order: Single 185 million per year
g) Long Insurance Order: 500 million per year
h) auto insurance Order: 10 million per year
i) information files: one million new single-year
Daily visits 50 million and product concurrent 2000 TPS is our assumption, customer information and files information with orders data changes and changes in the front, although we assume that the total produce 200 million orders each year, but according to the type of insurance, short risk (travel insurance, disability insurance) produced a significant 90% of the orders, which require special handling. In addition to auto insurance, and long risk (mainly life insurance, etc.), whether insured or service are significantly different, so it requires special handling.
So we follow the above requirements, system analysis, first by a large responsibilities will be divided the same duties as a service. With this performance requirements and above all functional requirements need to add a "quality" feature, that is, "high concurrency" high concurrency will affect all designs. Also, if you want quality characteristics of the Internet platform for the insurance row a sequence, most importantly, scalability, security, and more because of the type of insurance and different approach, in addition, high concurrency and reliability will directly affect the function of achieve, but did not affect large scalability. In-depth analysis of the key technologies to achieve listed after each function responsibilities, as follows:
| Demand Class |
Implementation requirements |
Subsystems and services to achieve |
Hardware and Software Technology |
| Client |
B2C e-commerce site |
B2C Web Client |
Cluster deployment, caching, distributed cache, search engine technology, static |
| B2C e-commerce site mobile client |
B2C App client |
|
|
| Agent Management |
Web proxy clients |
Cluster deployment, caching, distributed cache, search engine technology, static |
|
| Agent to manage the mobile client |
Attorney Client App |
|
|
| Process management files |
Web client files processing |
Cluster deployment, distributed cache |
|
| Customer relationship management |
Customer Management Web Client |
Cluster deployment, distributed cache |
|
| Insurance Appraisal Management |
Insurance adjuster Web Client |
Cluster deployment, distributed cache |
|
| Operation and maintenance of product management |
Web Client Product Management |
Cluster deployment, distributed cache |
|
| Statements and financial statistics |
Financial statements and statistics Web Client |
Cluster deployment, distributed cache |
|
| Public Service |
Operation and maintenance of product management, Web Access front-end products |
product service |
Cluster deployment, distributed cache |
| E-commerce order management or agent |
Order Services |
Cluster deployment, distributed cache |
|
| Such as e-commerce or agents involved in financial operations |
Ledger Service |
Cluster deployment, distributed cache |
|
| Statements and financial statistics |
Reporting Services |
Cluster deployment, distributed cache |
|
| Business Services |
B2C电子商务网站及手机客户端个人账户 |
B2C个人账户服务 |
集群部署、分布式缓存 |
| 代理人管理 |
代理人管理服务 |
集群部署、分布式缓存 |
|
| 案卷处理管理 |
案卷处理管理服务 |
集群部署、分布式缓存 |
|
| 客户管理 |
客户管理服务 |
集群部署、分布式缓存 |
|
| 保险公估管理 |
保险公估管理服务 |
集群部署、分布式缓存 |
|
| 短险开放式接入 |
开放式接入平台服务 |
集群部署、分布式缓存 |
|
| 工具性服务 |
保险公司产品对接 |
产品对接服务 |
集群部署、消息队列 |
| 在线支付 |
第三方支付服务 |
集群部署 |
|
| 短信邮件通知 |
通知服务 |
集群部署、消息队列 |
|
| 性能监控 |
日志采集服务 |
集群部署、消息队列 |
|
| 文件服务器 |
文件服务 |
集群部署、消息队列 |
|
| 服务授权与审计 |
服务授权与审计服务 |
集群部署 |
|
| 分布式事务管理 |
分布式事务管理服务 |
集群部署 |
|
| 定时任务管理 |
定时任务服务 |
集群部署 |
各个子系统及模块的关系如下图。
其中订单服务、产品服务、财务服务、工具服务为基础服务，其它各个业务模块的服务会调用这些基础服务。各个业务模块的服务都是根据业务领域进行划分的，同一业务领域下实现技术不同会被划分为两个服务，比如产品展示和订单原本属于同一个大的领域，但其因为实现技术和质量要求不同需要划分为两个服务。因为短险接入量大，而且大部分是跟第三方合作接入，因此设计短险接入公共接口服务平台处理大量短险订单。
l 存储及缓存架构
对于大型的高并发系统来讲，最重要的当属数据的架构。我们在前面也提到过，web系统业务处理模块本身就可以集群部署，当用户出现高并发时最先遇到的瓶颈就是数据库访问的瓶颈。这也是我们说数据架构最为重要的原因。其实web系统是典型的“计算机信息系统”，也就是说一切以数据（信息）为基础，所有的功能都是围绕着数据来的。这也是我们在这里说数据是web系统最重要的，很多公司在做少用户量web系统时直接设计好数据库就可以开发了。
按照上面划分的业务领域我们设计多个数据库，技术选项包括“是否读写分离”、“是否水平切分”及“路由键”。其中路由键是指在进行水平切分后，我们使用那个“标识”去查询数据库，一般来说会使用该业务领域聚合根对象的主键作为路由键。
| 数据库 |
是否读写分离 |
是否水平切分 |
水平切分路由键 |
| 产品数据库 |
是 |
|
|
| 订单数据库 |
|
是 |
客户ID |
| 公共数据库（元数据、公共数据） |
是 |
|
|
| 客户管理数据库 |
|
是 |
客户ID |
| 案卷管理数据库 |
|
是 |
客户ID |
| 代理人服务数据库 |
|
是 |
代理人ID |
| B2C电子商务个人账户数据库 |
|
是 |
客户ID |
| 保险公估数据库 |
|
是 |
客户ID |
| 工具数据库（短信、支付、文件） |
|
是 |
客户ID |
| 报表数据库 |
是 |
|
|
| 日志采集服务数据库 |
|
是 |
日志ID |
除工具数据库外，其它的数据库的划分很容易理解。工具数据库的数据也大都跟客户或说用户有关，比如“产品对接服务”，产品对接服务是指用户在购买了保单后，系统会自动对接到具体的保险公司接口去上传保单信息和下载保单，所以水平切分数据库时可以采用用户ID作为路由键。“在线支付”和“通知服务”也是类似，都是保存用户相关的数据，在线支付服务保存的是用户在线支付的流水，通知服务保存的是发送给某用户的短信或邮件。
另外在数据架构中我们也可以看到一个规律，就是使用了水平分库的存储结构就不能再使用读写分离，原因是防止存储单元过度泛滥，因为使用了水平分库之后，本身也要为数据库建立多个备份库，这个时候如果再用读写分离需要建立一套只读库，数据库的数量将增加一倍。使用了分库后我们可以把热点数据存储在分布式缓存中以起到读写分离类似的作用。
另外缓存也是存储技术的一种，而且非常重要。从日常生活中我们也可以看到这一点。比如你要去购买一台电脑，你会发现二级缓存和内存大的价格高出很多，如果你的显卡显存巨大，那将是顶级配置，发烧级配置。手机也是一样，内存大的手机速度明显快，价格也高，如果内存和持久化存储都高，那也是顶级配置。对于web系统也是一样，有些大型web系统只要缓存处理的好，数据库不需要分库就可以承载亿级的用户，比如维基百科、新浪微博等。也因此，不管是电子商务网站还是互联网+大型应用，都会在它们的架构中看到缓存大量使用的情况。
从整个web系统的架构来看，缓存在两个层面大量使用，分别是展示层和逻辑层，展示层通常使用高速的页面缓存，逻辑层通常使用高并发的分布式缓存。当然有些分布式缓存工具既可以在逻辑层使用也可以在显示层使用。
| 系统 |
是否使用CDN |
是否使用高速缓存服务器（varnish） |
是否使用分布式缓存（redis） |
| B2C电子商务网站 |
是 |
是 |
|
| 报表及财务统计Web端 |
|
是 |
|
| B2C个人账户服务 |
|
|
是 |
| 代理人管理服务 |
|
|
是 |
| 案卷处理管理服务 |
|
|
是 |
| 客户管理服务 |
|
|
是 |
| 保险公估管理服务 |
|
|
是 |
| B2C个人账户服务 |
|
|
是 |
| 短险公共平台服务 |
|
|
是 |
l 逻辑架构
在给出系统总体的逻辑架构前，我们先看看系统前端和服务层直接的关系。前端是客户端，可以有多个客户端，也可以有多种客户端，比如手机端（APP、WAP、微信）等。客户端和服务之间的架构是典型的MVC架构，V是客户端，C就是spring mvc或servlet开发的控制层，对于Web应用V和C在开发角度是一个工程，剩下的M就是服务层。这是对于web系统来说的，对于app或者使用html直接实现的客户端，或者是.net实现的桌面客户端，由于这些客户端是单独开发的，没有控制层，因此需要增加一层“API 网关”作为这些客户端的控制层。
图中的服务组件就是指各个业务服务，这些服务可以看成是组件的概念。通常前端界面一个界面中需要的数据通常不仅仅来自于同一个服务，即使是来自于同一个服务，那也来自于很多不同的接口，servlet或api 网关作为控制层，所起到的作用就是组合接口、组合数据，并处理接口间的事务性。另外服务组件也是分层的，图中并没有展现，一般可以分为3层，从低到高依次是工具性服务组件、基础业务层服务组件、业务层服务组件。前端界面的请求按照从高到底向下传递和处理请求。
按照职责、通用性、技术特性综合考虑和计量，逻辑架构设计如下图：
十几个子系统分别分布在服务层、控制层、表现层（典型的三层架构）。实体层和接口访问层虽然属于“层”，但它们并不单独发布，而是使用Jar包类库的方式提供给其它服务调用，是逻辑上的层。服务组件的构成大都是按照业务领域划分的，只有一个除外，就是“B2C网站”，B2C网站由于是面向终端用户的高并发电子商务网站，为了处理高并发，我们将其拆分为两个业务领域（也可以拆分成多个业务领域，看实际并发量），分别是用户账户和产品。用户浏览产品并购买，这是电子商务网站最基本的两个领域。其中产品浏览等功能由更底层的产品服务提供，用户账户功能由会员服务提供。还有一些功能，比如推荐商品可能是由其它服务提供，这些可以在控制层直接组合这些服务。
l 服务架构
服务框架采用典型的“服务注册表”模式，注册表使用redis。服务组件在启动时将自己注册进服务注册表，web服务器或api 网关在访问服务时查询服务注册表得到服务的uri，然后调用某服务接口。
淘宝的dubbo使用的是zookeeper作为服务注册表，之所以使用zookeeper，主要是使用它的负载均衡的功能。笔者认为restful接口没有必要使用zookeeper做负载均衡，可以使用nginx（负载均衡服务器都可以），所以没必要选用动态的zookeeper作为注册表，而是使用“redis+心跳监测”机制（redis也可以换为LDAP等）来完成服务注册和监控失效服务的功能。这个方案至少比dubbo简单几个数量级，简单就是硬道理。
在注册服务时只需要注册nginx服务器的IP以及服务描述信息即可。反观dubbo还要注册接口进注册表，笔者认为这个没必要，因为调用一个服务接口的充分必要条件就是知道服务器的IP即可。至于调用什么服务接口，肯定在代码里已经写死，目标服务器必定存在这个服务接口。将服务接口注册进服务注册表如果是为了监控审计服务的使用情况，那这个功能使用访问日志来实现能做的更好。
总体的分布式拓补结构如下图：
对于服务集群来讲，看上去像是一个大哥（nginx）带有众多小弟的结构。访问某服务群组只需要访问其nginx服务器即可，这里nginx均采用高可用方案，防止单台nginx出现问题。至于高层服务调用底层服务也是直接访问其服务器组的nginx。这个执行的流程其实和日常生活中的概念很像，如公司内部的执行流也是如此：老板分配任务给部门经理，部门经理分配任务给主管，主管分配任务给具体的个人（某单台服务器）。领导们起到的作用也是负载均衡和监控，负载均衡就是把任务可以分配给多个人同时执行（并且某个执行失败自动切换），监控就不必说了就是监控任务完成情况。在我们的方案里，会专门有个服务去监控所有服务器的执行情况，很简单的服务就可以做到。
l 关于分布式事务
如果研究一下EJB就会发现，EJB和微服务的架构基本相同，甚至所有面向服务（SCA、SOA等等）的架构都相差不大。很多人反对EJB，并不是EJB不够强大，而是它不够简单，它给项目带来的复杂性甚至超过了项目本身（这也是笔者不建议使用dubbo框架的原因）。曾有人说过：你要么把事情做的尽可能简单，让人挑不出毛病；要么把事情做的尽可能复杂，让人找不出毛病，EJB就是后者。EJB分布式事务机制实现的很好，可惜的是这种“一刀切”的事务机制，大大降低了web系统的性能，所以几乎所有面向互联网的应用都极少使用分布式事务，也就不会采用EJB。互联网应用本身事务性操作并不多，一些新闻、博客之类的网站甚至都不需要事务。另外一个方面，即使出现一个或两个分布式事务应用场景，也可以通过其它手段解决，比如事件机制等等。
在之前的文章中我们也提到过对于分布式事务最佳的策略是尽量避免。如果避免不了将按以下方式实现。
1） 将分布式事务性操作封装在一个服务中，这个服务使用XA或链式分布式事务管理。
2） 可以使用事件机制协调事务管理（具体做法就是事务失败后发失败事件到可持久化的消息队列，然后需回滚操作的接口监控此事件并执行回滚）。
3） 使用自定义分布式事务管理器管理分布式事务。
笔者设想的自定义分布式事务管理器主要是封装了流程及分布式事务相关功能，笔者将在其它文章专门讨论。如图所示，假设有一个事务需要依次调用ABCDE五个接口，我们首先调用分布式事务管理接口创建这条“流程”的实例，实例中五个接口分别对应五个状态，调用成功后将该接口对应的状态设置为“成功”，反之就是“失败”，流程处理结束后，分布式事务检查状态，然后按照一定的策略调用失败接口的反向操作接口去回滚数据（前提条件也是参与分布式事务操作的接口要开发反向操作接口）。这既是一个简单的流程引擎，也是一个分布式事务协调处理装置，具体是否有必要做的复杂（比如处理并行流程），还要看实际环境下分布式事务的情况，但笔者认为互联网前端应用使用简单流程应该足以应付。
l 开发架构
系统所需的工程，“[ ]”里面表示工程的名称。
从图中不难看出，我们将运维相关前端界面合并为一个前端系统，总体来讲前端只有3种，B2C前端、APP前端、运维前端。把运维前端合并为一个项目有利于加快前期的开发、部署的效率，在后期如果某子系统功能界面太多，可以将前端系统独立，比如公估系统、客户管理系统等，独立后的系统需要使用单点登录，这样就可以在各个系统之间免登陆切换。
开发环境：
编码：UTF-8
工具：Myeclipse 10
SVN：Site-1.8.22
Web服务器：Tomcat7
JDK: JDK1.7、 Java EE 5
开发环境：Maven 3
高并发高可、O2O、微服务架构用学习网站 https://www.itkc8.com
开发技术选型：
表现层：Bootstrap+Html+Jquery
MVC框架：Spring MVC 3.2
安全框架：Spring security 3.2
Rest接口实现：Spring boot
持久层：Mybatis3.2
分布式缓存：Redis
数据库：MySql 5.6