We have discussed the cluster composed of Nginx+tomcat, which is already a very flexible cluster technology, but when our system encounters a larger bottleneck, the single-point server of all applications can no longer meet our needs. At this time, we have to consider Another kind of content that we are familiar with is distributed, and the currently popular Dubbo framework.
1. Background
Before we needed to call other people's interfaces remotely, we did this:
Problems we encountered:
(1) When there are more and more services, service URL configuration management becomes very difficult, and the single point pressure of F5 hardware load balancer is also increasing.
At this time, a service registry is needed to dynamically register and discover services, so that the location of the service is transparent.
And by obtaining the address list of service providers on the consumer side to achieve soft load balancing and failover, reduce the dependence on the F5 hardware load balancer, and also reduce part of the cost.
(2) With further development, the dependencies between services become complicated and misunderstood, and it is not even clear which application should be started before which application, and architects cannot fully describe the architectural relationship of the application.
At this time, it is necessary to automatically draw a dependency relationship diagram between applications to help architects clean up the relationship.
(3) Then, the call volume of the service is increasing, and the capacity problem of the service is exposed. How many machines does this service need? When should the machine be added?
In order to solve these problems, the first step is to count the current daily call volume and response time of the service as a reference indicator for capacity planning.
Secondly, it is necessary to dynamically adjust the weight. Online, increase the weight of a certain machine all the time, and record the change of the response time in the process of increasing, until the response time reaches the threshold, record the number of visits at this time, and then use This number of visits is multiplied by the number of machines to invert the total capacity.
To solve these problems, what Dubbo did for us:
Load balancing:
This is called soft load balancing!
Now let's get in touch with this excellent framework:
Introduction
The structure is shown in the figure:
Node role description:
Provider: The service provider that exposes the service.
Consumer: The service consumer that invokes the remote service.
Registry: A registry for service registration and discovery.
Monitor: The monitoring center that counts the invocation times and invocation time of the service.
Container: The service runs the container.
Description of the calling relationship:
0. The service container is responsible for starting, loading, and running the service provider.
1. When the service provider starts, it registers the service it provides with the registry.
2. When the service consumer starts, it subscribes to the registration center for the services it needs.
3. The registry returns the service provider address list to the consumer. If there is a change, the registry will push the change data to the consumer based on the persistent connection.
4. The service consumer, from the provider address list, selects a provider to call based on the soft load balancing algorithm, and if the call fails, select another provider to call.
5. Service consumers and providers accumulate the number of calls and call times in memory, and regularly send statistical data to the monitoring center every minute.
Third: Dubbo integrates with Zookeeper and SpringMVC
Step 1: Install Zookeeper on Linux
Zookeeper is the registration center of the Dubbo service. Dubbo's original database-based registration center did not use Zookeeper. Zookeeper is a distributed service framework. It is a tree-shaped directory service data storage, which can manage data in clusters. It can be very good here. As the registry of the Dubbo service, Dubbo can be deployed in clusters with Zookeeper. When the provider experiences abnormal shutdowns such as power failure, the Zookeeper registry can automatically delete the provider information, and when the provider restarts, it can automatically restore the registration data, and Subscription request. We first install Zookeeper on linux. We install the simplest single point, and the cluster is more troublesome.
You need to install JdK first, download it from Oracle's Java website, the installation is very simple, so I won't go into details.
Stand-alone installation is very simple, just get the Zookeeper zip package and extract it to a directory such as: C:\zookeeper-3.4.5\, the startup script of Zookeeper is in the bin directory, and the startup script under Windows is zkServer.cmd.
Before you execute the startup script, there are still several basic configuration items that need to be configured. The Zookeeper configuration file is in the conf directory, which contains zoo_sample.cfg and log4j.properties. All you need to do is to rename zoo_sample.cfg It is zoo.cfg, because Zookeeper will look for this file as the default configuration file when it starts. The following is a detailed introduction to the meaning of each configuration item in this configuration file.
<span style="font-size:18px;"># The number of milliseconds of each tick tickTime =2000 # The number of ticks that the initial # synchronization phase can take initLimit = 10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=C:\\zookeeper-3.4.5\\data dataLogDir=C:\\zookeeper-3.4.5\\log # the port at which the clients will connect clientPort=2181 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature #autopurge.purgeInterval=1</span>
- tickTime: This time is used as the time interval for maintaining heartbeats between Zookeeper servers or between clients and servers, that is, a heartbeat will be sent every tickTime.
- dataDir: As the name suggests, it is the directory where Zookeeper saves data. By default, Zookeeper also saves log files for writing data in this directory.
- dataLogDir: As the name suggests, it is the directory where Zookeeper saves log files
- clientPort: This port is the port through which the client connects to the Zookeeper server. Zookeeper will listen to this port and accept client access requests.
When these configuration items are configured, you can start Zookeeper now. After startup, you need to check whether Zookeeper is already in service. You can use the netstat –ano command to check whether the clientPort port number you configured is listening for the service.
Step 2: Configure the management page of dubbo-admin to facilitate us to manage the page
(1) Download the dubbo-admin-2.4.1.war package, deploy it in the tomcat of windows, first put the dubbo-admin-2.4.1 under the webapps/ROOT of tomcat, and then decompress it
(2) Then go to webapps/ROOT/WEB-INF, there is a dubbo.properties file, which points to Zookeeper, using Zookeeper's registration center, as shown in the figure:
<span style="font-size:18px;">dubbo.registry.address=zookeeper://127.0.0.1:2181 dubbo.admin.root.password=root dubbo.admin.guest.password=guest
</span>
(3) Then start the tomcat service, user name and password: root, and access the service, display the login page, indicating that the dubbo-admin deployment is successful, as shown in the figure:
The third step: the integration of SpringMVC and Dubbo, the Maven management project used here
First: We first develop service registration, which is to provide services. The project structure is shown in the figure:
(1) A service interface has been added to the test-maven-api project, and the code is as follows:
public interface TestRegistryService { public String hello(String name); }
(2) test-maven-console adds the jar packages of Dubbo and Zookeeper to pom.xml, and references the jar package of test-maven-api. The code is as follows:
<span style="font-size:18px;"><dependency>
<groupId>cn.test</groupId>
<artifactId>test-maven-api</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>dubbo</artifactId>
<version>2.5.3</version>
</dependency>
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
<version>3.4.6</version>
</dependency>
<dependency>
<groupId>com.github.sgroschupf</groupId>
<artifactId>zkclient</artifactId>
<version>0.1</version>
</dependency></span>
(3) test-maven-console implements specific services, the code is as follows:
@Service("testRegistryService")
public class TestRegistryServiceImpl implements TestRegistryService {
public String hello(String name) {
return "hello"+name;
}
}
(4) Our service and implementation are complete. At this time, we need to expose the service. The code is as follows:
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:jee="http://www.springframework.org/schema/jee" xmlns:tx="http://www.springframework.org/schema/tx" <span style="color:#cc0000;">xmlns:dubbo="http://code.alibabatech.com/schema/dubbo"</span> xmlns:context="http://www.springframework.org/schema/context" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-3.1.xsd <span style="color:#990000;">http://code.alibabatech.com/schema/dubbo http://code.alibabatech.com/schema/dubbo/dubbo.xsd</span> http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd" default-lazy-init="false" > <!-- provider application name information, this is equivalent to a name, our dubbo management page is more clear which application is exposed --> <dubbo:application name="dubbo_provider"></dubbo:application> <!-- Use the zookeeper registry to expose the service address --> <dubbo:registry address="zookeeper://127.0.0.1:2181" check="false" subscribe="false" register=""></dubbo:registry> <!-- Service interface to be exposed --> <dubbo:service interface="cn.test.dubbo.registry.service.TestRegistryService" ref="testRegistryService" /> </beans>
illustrate:
Description of some attributes of the dubbo:registry tag:
1) Whether register registers the service with this registry center, if it is set to false, it will only subscribe, not register.
2) Check whether an error is reported when the registration center does not exist.
3) Whether subscribe subscribes services to this registry center, if it is set to false, it will only register, not subscribe.
4) timeout registration center request timeout time (milliseconds).
5) The address can be configured in the Zookeeper cluster, and multiple addresses can be separated by commas, etc.
Some attribute descriptions of the dubbo:service tag:
1) The path of the interface service interface
2) ref refers to the ID of the bean of the corresponding implementation class
3) The registry registers with the specified registry and is used in multiple registries. The value is the id attribute of <dubbo:registry>. Multiple registry IDs are separated by commas. If you do not want to register the service to any registry, you can set the value set to N/A
4) register defaults to true, whether the service of this protocol is registered to the registry.
(5) Start the project, and then we display the exposed services on the Dubbo management page, but it shows that there is no consumer yet, because we have not yet implemented the consumer service, as shown in the figure:
Second: We are developing service consumers, that is, calling services. We are creating a new consumer project:
(1) The pom.xml of test-maven-server-console introduces the jar package of Dubbo and Zookeeper, and the jar package of test-maven-api. Because the jar package of test-maven-api is introduced, we call it locally in the project call the same. code show as below:
<span style="font-size:18px;"><dependency>
<groupId>cn.test</groupId>
<artifactId>test-maven-api</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>dubbo</artifactId>
<version>2.5.3</version>
</dependency>
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
<version>3.4.6</version>
</dependency>
<dependency>
<groupId>com.github.sgroschupf</groupId>
<artifactId>zkclient</artifactId>
<version>0.1</version>
</dependency></span>
(2) The specific implementation of the test-maven-server-console project, the code is as follows:
@Controller public class IndexController { @Autowired private TestRegistryService testRegistryService; @RequestMapping("/hello") public String index(Model model){ String name=testRegistryService.hello("zz"); System.out.println("xx=="+name); return ""; } }
(3) The address we want to reference, the code is as follows:
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:jee="http://www.springframework.org/schema/jee" xmlns:tx="http://www.springframework.org/schema/tx" <span style="background-color: rgb(255, 255, 255);"><span style="color:#990000;">xmlns:dubbo="http://code.alibabatech.com/schema/dubbo"</span></span> xmlns:context="http://www.springframework.org/schema/context" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-3.1.xsd <span style="color:#990000;">http://code.alibabatech.com/schema/dubbo http://code.alibabatech.com/schema/dubbo/dubbo.xsd</span> http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd" default-lazy-init="false" > <dubbo:application name="dubbo_consumer"></dubbo:application> <!-- Use the zookeeper registry to expose the service address --> <dubbo:registry address="zookeeper://192.168.74.129:2181" check="false"></dubbo:registry> <!-- Service to be referenced --> <dubbo:reference interface="cn.test.dubbo.registry.service.TestRegistryService" id="testRegistryService"></dubbo:reference> </beans>
illustrate:
Description of some properties of dubbo:reference:
1) Service interface called by interface
2) check checks whether the provider exists when starting, true reports an error, false ignores
3) registry obtains the service list from the specified registry registration, used in multiple registries, the value is the id attribute of <dubbo:registry>, and multiple registry IDs are separated by commas
4) loadbalance Load balancing strategy, optional values: random, roundrobin, leastactive, respectively: random, round-robin, least active calls
(4) When the project is started, on the Dubbo management page, you can see consumers, as shown in the figure:
(5) Then access the consumer project, the Controller layer can call the specific implementation of the service like calling the local, as shown in the figure:
