Practical application of micro Services Architecture [upgrade] .net core business platform (core-grpc)

I. Introduction

This article was originally continue to share IdentityServer4the relevant article, because before Bo Friends ask me about 微服务related issues, I would skip IdentityServer4the sharing, 微服务the relevant technical learning and sharing. 微服务I shared directory is placed in the beginning of April to share a series of articles, here on the first pass through the lower, pre-arranged 微服务opening article applications 电商系统升级之微服务架构的应用.
This blog and the number of public adhere to the thinking of architecture to share technology, share more than just a simple Demo how to use .

Two, Scene

Under the first to review my articles on the application Asp.Net Core IdentityServer4 authorized centers in actual combat , the electricity supplier by the architecture 单体式架构upgrade to Split多网关架构

Before you upgrade

After the upgrade:

However, after the upgrade problem again, due to the increase before the agency and the business 授权中心and 支付网关split out separately, which makes the company's business orders turned a few times, this time the entire electricity supplier system reached a bottleneck, if we do not find a solution You have to program the system is down.

2.1 Problems and Solutions

Through research and analysis of technical problems, the bottleneck causing the problem mainly in the following reasons, just need to solve the following problem you can get a great performance boost

Surge orders per day, resulting in much the order data, however, the entire electrical system data stored in a supplier database, and is 单表, 单数据库(not separated read and write), in order that the data continuously surge.
Related businesses need to rely order inquiries, orders that caused the collapse of a database query slow
The whole business system to achieve electrical connections bottleneck (already distributed deployment, the server will consume more pay more funding and reach the best value for money)

In order to solve the above problems once and for all, through research and technology, decided to make the following order business upgrade:

Split orders independent micro-services (This section focuses on sharing)
Using ESdata migration (divided by year, and to read and write separation, there is not focus on how, next time to learn and share with you)
Increase 分布式缓存(this is not the focus of follow-up come to learn and share with you)

After Chart upgraded as follows:

Chart description:

On behalf of the right of same color or previous electricity supplier system 单体式架构, for a split single business architecture, business process in which a mixed layer of distributed cache processing
The left is the micro-architecture services, this upgrade is the split architecture, which has a database and migrate data from the original database split into ES cluster, and a separate read and write.
Order service free to expansion into distributed services, dynamic expansion of support services and server through a number of tools.
The right of the follow-up business can also be split, split into different business services.
Subsequent upgrades may also be considered a related aspect the message queue, etc., not contemplated architecture diagram (used in the subsequent return to share the upgrade of the related art, there is a return to the core of the article 微服务)

Third, micro-Services Overview

微服务The concepts I will not say more, or less on the first micro-service advantages and disadvantages brought about by simple overview.

3.1 Advantages of micro-services

The large, complex applications can deliver sustained and continuous deployment: Continuous Delivery and Continuous Deployment is part of DevOps, DevOps is a fast, frequent and reliable software delivery practices. Efficient DevOps organizations often deploy software faced with fewer problems and failure to production. DevOps tools Docker, Kubernets, Jenkins, Gitand so on.
Each service is relatively small and easy to maintain: micro Services Architecture Application monomer is small compared to the more developers understand the logic code service easier. Small code base, packing, start the service fast.
Services can be deployed independently: Each service can be deployed independently of the other services
It can be independently extended services: services can be extended independently, whether the X-axis extended Cloning of example, the Z-axis or flow partitioning. In addition, each service can be deployed on top of the hardware to fit their needs
Micro-service architecture can achieve autonomy team: the team can be split according to the service. Each team has its own micro-responsible services, without concern that they are not responsible for service.
Easier to test and adopt new technologies: Finally, micro-services can eliminate long-term reliance on a technology stack. Because the service is smaller, the use of programming languages and techniques to rewrite the replacement of a service becomes possible, this also means that, after a failed attempt on a new technology, you can directly discard this part of the work will not apply to the whole band to the risk of failure.
Better fault tolerance: micro-services architecture can also achieve fault isolation replacement. For example, a service triggered a fatal error will not affect other services. Other services are still operating normally.
Service can be independently expansion: For the whole structure, it can choose to expand and load-related business, conducted through random expansion of the relevant technical tools Dynamic

3.2 micro inferior services

Split and service definition is a challenge: the use of micro-service architecture bear the brunt of the problem is that there is no one specific, well-defined algorithm can complete the split work services. As with software development, split, and the definition of service is more like an art. Worse, if the service system appeared split deviation is likely to build a distributed application monomer; contains a lot of tight coupling between each service, but must be deployed together the so-called distributed systems. This will put the drawbacks of both monomers architecture and micro-architecture services rolled into one.
Distributed systems due to the complex nature, the development, testing and deployment becomes more difficult: Another problem with micro-services architecture is to create additional developer must deal with the complexity of distributed systems. Services must be inter-process communication. This is much more complex than a simple method call.
More needs to be carefully coordinated development teams when deploying services across multiple functions: the use of micro-services architecture Another challenge is the need to more carefully coordinate the development team when deploying services across multiple functions. Must develop a release plan, sorted according to the service dependencies. This is now under way to deploy multiple components of different monomers architecture.
Developers need to think in the end you should use the micro-service architecture at what stage of the application: Another problem with micro-service architecture is decided to start using this architecture in which stage of the application life cycle.
Problem of cross-service data: in monomer applications, all data in one database, while the micro-services architecture, each service has its own database, you want to acquire, operate other data services, only through the service provides an API call, thus bringing the issue is a problem, the process of communication, if it comes to business, you also need to manage the affairs Saga, increasing the difficulty of development.

3.3 microService split principle

When it comes to 单体架构splitting, it is not free to split, is to have certain principles, is the advantage of a good split, the split is not chaos. The following is my access to information as well as my experience summed up the principle of split

1, single responsibility, high cohesion and low coupling
2, the micro-size service moderate
3, consider the team structure
4, in order to cut into the business model
5, an evolved split
6, the annular avoid dependence and dependence bidirectional
7, DDD (consider using 领域驱动设计to carry out the design of the underlying service, follow-up analyzes articles about the design of the individual)

Fourth, the micro-service combat

Well, here we have some understanding of the micro-services, do not continue a detailed overview of the concept of something related, directly below the line and the code, so that everyone familiar with the application of micro-services. Here I use 莫堇蕈the open source on github micro-services framework , Frame Source Address: https://github.com/overtly/core-grpc ( I strongly recommend that this framework is now more mature company for production environments )

In order to better maintain open source projects and technical exchanges, deliberately created an exchange group, group number: 1083147206 Those interested to join the open source exchange

4.1 `core-grpc`Micro Services Framework advantages:

Consul achieve integration and service discovery mechanisms such as registration and health checks
Real-time monitoring service status
Multi-node polling mechanism
Failover, pull into the blacklist
Support Framework and .Net Core two frameworks
Based Grpc micro service
Deployment Support Environment Variables

4.2 combat

Creating `Jlion.NetCore.OrderService`orders microService

We vs2019create a console application to select the frame .Net Core 3.1 named Jlion.NetCore.OrderServicelater referred to 订单服务, after we have created through nugetthe introduction of package core-grpcmicro-services framework, as follows:

the current core-grpcmicro-services framework, the latest official release version is 1.0.3

Cites core-grpcWe also need to install a tool VS RPC Menu, this tool is also a big God provided free of charge, the picture is as follows:

Because Microsoft's official download more slowly, I am here to share network disk Baidu, Baidu network disk download address is as follows:

Links: https://pan.baidu.com/s/1twpmA4_aErrsg-m0ICmOPw extraction code: cshs

If not installed by downloading vs integrated installation, the download is complete and needs to close vs 2019 related to normal installation.

VS RPC Menu tool as follows:

The client code generation support for Grpc and Thrift
us again in 订单服务creating a project in OrderRequest.protoa file, this is Grpcthe grammar, syntax do not understand the students can click gRPC official Chinese version of the document _V1.0 learning, address: HTTP: // DOC .oschina.net / grpc? t = 56831

OrderRequest.protocode show as below:

syntax = "proto3";
package Jlion.NetCore.OrderService.Service.Grpc;


//定义订单查找参数实体
message OrderSearchRequest{
    string OrderId = 1; //定义订单ID
    string Name = 2;
}

//定义订单实体
message OrderRepsonse{
    string OrderId = 1;
    string Name = 2;
    double Amount = 3;
    int32 Count = 4;
    string Time = 5;
}

//定义订单查找列表
message OrderSearchResponse{
    bool Success = 1;
    string ErrorMsg = 2;
    repeated OrderRepsonse Data = 3;
}

The above is the definition of several news entity,
we then create the JlionOrderService.protofollowing code:

syntax = "proto3";
package Jlion.NetCore.OrderService.Service.Grpc;

import "OrderRequest.proto";

service JlionOrderService{
    rpc Order_Search(OrderSearchRequest) returns (OrderSearchResponse){} 
}

The above code can see the top of package Jlion.NetCore.OrderService.Service.Grpcthe code, which is a statement after the package name is generated after the code name space, this is very important .
At the same time defines the JlionOrderServiceservice entrance, and defines a method of searching an order Order_Search, here we have completed a small part of.

Generate client code

Then the JlionOrderService.protofile inside right "choice Grpc Code Generation" Grpc code automatically survival of micro-service client code.
Survival tool has the following features:

Survival Grpc client code
Grpc compiler (not used)
Grpc package (commonly used to publish client dll to nuget server)
You can also generate the code and packetizes Thrift

Create a `Jlion.NetCore.OrderService.Grpc`class library

Just by the tool to generate Grpcclient code direct copy of Jlion.NetCore.OrderService.Grpcthis library (package must be consistent with the code of the above statement Grpc) hereinafter referred to 订单服务客户端, and need to Nugetadd packages Overt.Core.Grpcdependency, the code structure is as follows:

Jlion.NetCore.OrderService.Grpclibrary has been constructed, and now let Jlion.NetCore.OrderServiceservice reference Jlion.NetCore.OrderService.Grpclibrary

`订单服务`Achieve their`IHostedService`

Create a HostServiceclass that inherits IHostedServicecode is as follows:

public class HostedService : IHostedService
{
    readonly ILogger _logger;
    readonly JlionOrderServiceBase _grpcServImpl;
    public HostedService(
        ILogger<HostedService> logger,
        JlionOrderServiceBase grpcService)
    {
        _logger = logger;
        _grpcServImpl = grpcService;
    }

    //服务的启动机相关配置
    public Task StartAsync(CancellationToken cancellationToken)
    {
        return Task.Factory.StartNew(() =>
        {
            var channelOptions = new List<ChannelOption>()
            {
                 new ChannelOption(ChannelOptions.MaxReceiveMessageLength, int.MaxValue),
                 new ChannelOption(ChannelOptions.MaxSendMessageLength, int.MaxValue),
            };
            GrpcServiceManager.Start(BindService(_grpcServImpl), channelOptions: channelOptions, whenException: (ex) =>
            {
                _logger.LogError(ex, $"{typeof(HostedService).Namespace.Replace(".", "")}开启失败");
                throw ex;
            });
            System.Console.WriteLine("服务已经启动");
            _logger.LogInformation($"{nameof(Jlion.NetCore.OrderService.Service).Replace(".", "")}开启成功");
        }, cancellationToken);
    }

    //服务的停止
    public Task StopAsync(CancellationToken cancellationToken)
    {
        return Task.Factory.StartNew(() =>
        {
            GrpcServiceManager.Stop();

            _logger.LogInformation($"{typeof(HostedService).Namespace.Replace(".", "")}停止成功");
        }, cancellationToken);
    }
 }

The above code is mainly created a host computer and realized the StartAsyncservice and start StopAsyncservice stop method.
We've created HostedServiccethe code again created previously defined Grpcmethod of service implementation class JlionOrderServiceImpl, as follows:

public partial class JlionOrderServiceImpl : JlionOrderServiceBase
{
    private readonly ILogger _logger;
    private readonly IServiceProvider _serviceProvider;

    public JlionOrderServiceImpl(ILogger<JlionOrderServiceImpl> logger, IServiceProvider provider)
    {
        _logger = logger;
        _serviceProvider = provider;
    }

    public override async Task<OrderSearchResponse> Order_Search(OrderSearchRequest request, ServerCallContext context)
    {
        //TODO 从底层ES中查找订单数据，
        //可以设计成DDD 方式来进行ES的操作，这里我就为了演示直接硬编码了

        var response = new OrderSearchResponse();
        try
        {
            response.Data.Add(new OrderRepsonse()
            {
                Amount = 100.00,
                Count = 10,
                Name = "订单名称测试",
                OrderId = DateTime.Now.ToString("yyyyMMddHHmmss"),
                Time = DateTime.Now.ToString()
            });

            response.Data.Add(new OrderRepsonse()
            {
                Amount = 200.00,
                Count = 10,
                Name = "订单名称测试2",
                OrderId = DateTime.Now.ToString("yyyyMMddHHmmss"),
                Time = DateTime.Now.ToString()
            });

            response.Data.Add(new OrderRepsonse()
            {
                Amount = 300.00,
                Count = 10,
                Name = "订单名称测试2",
                OrderId = DateTime.Now.ToString("yyyyMMddHHmmss"),
                Time = DateTime.Now.ToString()
            });
            response.Success = true;
        }
        catch (Exception ex)
        {
            response.ErrorMsg = ex.Message;
            _logger.LogWarning("异常");
        }
        return response;
    }
 }

Modify Programthe code, and to HostedServiceand JlionOrderServiceImplpoured into the container, as follows:

 class Program
 {
    static void Main(string[] args)
    {
        var host = new HostBuilder()
           .UseConsoleLifetime() //使用控制台生命周期
           .ConfigureAppConfiguration((context, configuration) =>
           {
               configuration
               .AddJsonFile("appsettings.json", optional: true)
               .AddEnvironmentVariables();
           })
           .ConfigureLogging(logger =>
           {
               logger.AddFilter("Microsoft", LogLevel.Critical)
                     .AddFilter("System", LogLevel.Critical);
           })
           .ConfigureServices(ConfigureServices)
           .Build();

        AppDomain.CurrentDomain.UnhandledException += (sender, e) =>
        {
            var logFactory = host.Services.GetService<ILoggerFactory>();
            var logger = logFactory.CreateLogger<Program>();
            logger.LogError(e.ExceptionObject as Exception, $"UnhandledException");
        };

        host.Run();
    }

    /// <summary>
    /// 通用DI注入
    /// </summary>
    /// <param name="context"></param>
    /// <param name="services"></param>
    private static void ConfigureServices(HostBuilderContext context, IServiceCollection services)
    {
        //HostedService 单例注入到DI 中
        services.AddSingleton<IHostedService, HostedService>();
        services.AddTransient<JlionOrderServiceBase, JlionOrderServiceImpl>();
    }
 }

To here simply 微服务already been encoded, but also the lack of two configuration files, we created appsettings.jsonprofiles, and consulsettings.jsonservice registration found in the configuration file
consulsettings.jsonconfiguration file as follows:

{
  "ConsulServer": {
    "Service": {
      "Address": "127.0.0.1：8500"// 你的Consul 服务注册及发现配置地址
    }
  }
}

The above address configuration is just a simple example, here I assume my Consulservice address is 127.0.0.1:8500 services start at will be registered with this address.

appsettings.jsonConfiguration files are as follows:

{
  "GrpcServer": {
    "Service": {
      "Name": "JlionOrderService",
      "Port": 10001,
      "HostEnv": "serviceaddress",
      "Consul": {
        "Path": "dllconfigs/consulsettings.json"
      }
    }
  }
}

I am here to serve the listening port 10001, registration back to Consulthe port will also see the
official full configuration file as follows:

{
  "GrpcServer": {
    "Service": {
      "Name": "OvertGrpcServiceApp",                    // 服务名称使用服务名称去除点：OvertGrpcServiceApp
      "Host": "service.g.lan",                          // 专用注册的域名 （可选）格式：ip[:port=default]
      "HostEnv": "serviceaddress",                      // 获取注册地址的环境变量名字（可选，优先）环境变量值格式：ip[:port=default]
      "Port": 10001,                                    // 端口自定义
      "Consul": {
        "Path": "dllconfigs/consulsettings.json"        // Consul路径
      }
    }
  }
}

Well, 订单服务have all been completed, 订单服务the overall structure of the service is as follows:

Well, we here at the command line to start JlionOrderServicethe service, you can build in a production environment Dockerinside the container

Before we can build a good look at my Consulservice, open the management interface, as shown:

images can be found just started the service has registered to go in, but there are a health check failed, mainly due to the server can not access my local of 订单服务all health checks can not pass. You can set up your locally Consulfor testing services.

I come back to open a local service configurations by the port number 10001 into 10002, and then view the Consulmanagement interface, as shown below:

found to have registered two services, port numbers are 10001 and 10002, which can be customized tools and services to automatically add the shelf services, distributed services are complete.
Here 订单服务start has been completely successful, we need next is a client that is above architecture diagram 电商业务网关or 支付网关so on to keep 订单服务communication up.

Create an order gateway (with order service for communication)

Before you create an order I put the above Gateway 订单服务客户端publish library to my nuget package, there is not demonstrated. I tested the package name published JlionOrderServiceDemonuget official search can find. You can also search directly added to your Demo for testing.
I created Asp.Net Core 3.1 framework by VS 2019 WebApinamed Jlion.NetCore.OrderApiServicebelow referred to 订单网关服务
now I posted earlier 微服务client dependencies JlionOrderServiceDemoadded to 订单网关服务the following chart:

Now 订单网关服务add the OrderControllerapi controller, as follows:

namespace Jlion.NetCore.OrderApiService.Controllers
{
    [Route("[controller]")]
    [ApiController]
    public class OrderController : ControllerBase
    {
        private readonly IGrpcClient<OrderService.Service.Grpc.JlionOrderService.JlionOrderServiceClient> _orderService;
        public OrderController (IGrpcClient<OrderService.Service.Grpc.JlionOrderService.JlionOrderServiceClient> orderService)
        {
            _orderService = orderService;
        }
     
        [HttpGet("getlist")]
        public async Task<List<OrderRepsonse>> GetList()
        {
            var respData =await _orderService.Client.Order_SearchAsync(new OrderService.Service.Grpc.OrderSearchRequest()
            {
                Name = "test",
                OrderId = "",
            });

            if ((respData?.Data?.Count ?? 0) <= 0)
            {
                return new List<OrderRepsonse>();
            }

            return respData.Data.ToList();
        }
    }
}

Injected via the constructor code OrderServiceand provides an GetListinterface methods. Next, we also need to OrderService.Service.Grpc.JlionOrderServiceinject into the container, as follows:

public void ConfigureServices(IServiceCollection services)
{
     services.AddControllers();

     //注册Grpc 客户端，具体可以查看源代码
     services.AddGrpcClient();
}

Now the entire 订单网关服务project structure below:

There are two most important project configuration dllconfig//Jlion.NetCore.OrderService.Grpc.dll.jsonand consulsettings.jsonthey are doing it? We were content to see me both local configuration
Jlion.NetCore.OrderService.Grpc.dll.jsonconfiguration is as follows:

{
   "GrpcClient": {
       "Service": {
        "Name": "JlionOrderService",  // 服务名称与服务端保持一致
        "MaxRetry": 0,                // 最大可重试次数，默认不重试
        "Discovery": {
            "Consul": {              // Consul集群,集群优先原则
                "Path": "dllconfigs/consulsettings.json"
            }，
            "EndPoints": [           // 单点模式
                {
                    "Host": "127.0.0.1",
                    "Port": 10001
             }]
            }
        }
    }
}

Jlion.NetCore.OrderService.Grpc.dll.jsonConfiguration is to tell 订单网关服务and 订单服务what to communicate, and communication among some of the parameters. To test my local use single mode, without the use of Consul mode
consulsettings.jsonconfiguration is as follows:

{
  "ConsulServer": {
    "Service": {
      "Address": "127.0.0.1:8500"
    }
  }
}

Have not found the same configuration and the configuration of the server before, it is to tell 订单网关服务(clients caller) and 订单服务server-side service discovery cluster address, if the above configuration is a single-point mode if this configuration will not work.

Here 订单网关服务(client-side call) coding is completed, we started it:

I'm here fixed 5003 port, now a perfect start, we visited orders interface look successful. Access results as shown below:

Micro service perfect run successfully.

The above construction of micro-services is still relatively trouble, the official provides a more rapid way to build micro-services you need, do not need to write that code above, all of those codes to build your micro-services through the template, there is a need to learn to be click on
micro-services project to build templates tutorial
tutorial address: https://www.cnblogs.com/jlion/p/12494525.html

Article Demo code has been filed on github, code address: https://github.com/a312586670/IdentityServerDemo
Micro Services Framework open source project Address: https://github.com/overtly/core-grpc