Cloud testing, these key points you do not get to
HTTPS: // www.cnblogs.com/mypm/p/10852656.html?tdsourcetag=s_pcqq_aiomsg Sticky sessionAffinity there is not really the best development model of cloud native.
REVIEW, start talking about cloud technology, then talk about the cloud morphology, in addition to conventional functional tests, the cloud of the test, but also there are a few must get to the hard core indicators, and finally explain the key points were hard What is the core indicators, and how to test it. This is a thought-provoking questions, hope that all people get to test them, and do more than post mentioned that extract more check point.
First look at the general trend of the rise of the cloud
Moment oracle cut across China R & D Center, is ignited, a key reason is that, by the oracle cloud computing and other emerging technologies impact their business grow weak or even decline, so pay more attention to cost control, so that strategic personnel changes, the cloud make more computing resources; today, Ali goes beyond Microsoft Azure, second only to the Amazon AWS cloud computing world's second largest company, 10 years ago in the case of Baidu and Tencent are not optimistic about, Ma believes that cloud computing is future annual investment of 100 million, without knowing these years re-energized, so Ma voted Ali cloud Wang Jian 10 years, Ma said to really put the final in 1000 only made things to do! But then the whole of China no company is willing to invest the cloud of this "illusory" in the cause, the most difficult times, 80% of engineers for various reasons, left the cloud Ali, Dr. Wang Jian ten years ago, in the eyes of many people It is a truly a "liar."
Early cloud vendors, cloud resources are mainly provided by the emphasis on iaas layer, currently with the further development of cloud computing, from iaas, to paas, serverleess plus container technology, cloud vendors have become standard products, cloud technology is an irreversible trend It has been accepted by the public, of course, subject to some restrictions on non-technical factors or some historical baggage restrictions, mixed cloud, public cloud cloudy and a long-term existence. More and more companies to service in the cloud, small companies may just simply move to the cloud service (that is, the earlier this deployment, move to cloud), in fact this is not true cloud computing, disaster recovery course ability to enhance the quality, SaaS vendor, or large companies, will be on the cloud vendors pass platform to build a layer of glue, integrating PAAS and SAAS form their own cloud management platform, transparent achieve elasticity calculation, the lateral expansion and visualized O & M monitoring and other non-functional management needs.
These hard-core test cloud of non-functional key points must be measured must get to
These non-functional hard core and key characteristics of cloud computing has a causal relationship, but also service and reliability, measurement and governance are closely related services. In simple terms you want to ensure the reliability, availability, manageability of your cloud service, you must have some non-functional indicators of the cloud, because then you business functions easy to use, unreliable, the user is not guaranteed follows eight hard core point is the cloud of non-functional indicator, a subsequent one and then were told, the hard core points is defined and how to test these points, in particular the first, second, third, three an indispensable, otherwise your software just move to the cloud, does not have the characteristics of any cloud computing, that is a false cloud.
The first elastic, also known as auto-scaling;
2, stateless service;
3, multi-tenant support;
4, failover / separator;
5, limit protection services;
6, application security;
7, call chain tracking,
8, visualization services management (observable, automatic health check service elegant off, etc.)
One example will be described with hard core 8 points and test methods or means
The figure is one of aws, and build a VPC via VPN and local services make up a hybrid cloud network architecture,
Create four subnets, each for deployment: tenant registration Web, database, Web Invoicing program, Invoicing App program; in order to achieve high availability, according to each type of subnets available each district to create a, b two a.
After successful registration of tenants, call CloudFormation interfaces, cloud Invoicing business systems automatic deployment. (Actually physically isolated)
Go back to the previous eight hard core indicators
The first elastic, also known as automatic retractable
Speaking of flexibility, start often said that the cloud server, ecs talking about, it is the abbreviation Elastic Compute Service, which is a processing capacity of elastically scalable computing server, which is from IAAS (more partial hardware resources: CPU, memory, disk, network) layer to achieve resilience, which addresses the underlying hardware or elastic resources; actual cloud services from software layer now of view, but also has its own elasticity is elastic or paas saas software layer, for example, , according to the time required scale or other strategies (based traffic, or the hardware resources of a particular baseline), as the peak of the registration service, a 10-node support to ensure peak concurrency, off-peak, and then decrease registered service node, freeing up computing resources to do other things. In the cloud, cluster nodes to achieve the expansion is a very simple matter, you can configure the expansion strategy. When tested, it must be elastic as a Point the Check , how to verify that the definition can understand from the front, will not repeat it here. Now the question is, the dynamic expansion of the service, how to make a cluster-aware, and it can be used, see item 2 below
2, stateless service;
Connect a problem, dynamic expansion of the service, how to make a cluster-aware and usable? Let cluster aware there are two methods, namely, expansion of service at the time of the cross, to the ELB (load balancing) Dynamic registration (if the extension of the node, need to manually configure and reboot the load evenly related components, this is not called elasticity) , or by registering components to achieve service discovery, which has a lot of mature technology, there is not much explanation, the point is, new horizontal expansion out of service, plus after a cluster, let him be able to provide services, which requires the service is stateless (such as session session, or other context, does not depend on where the service is the main container, such as tomcat, jetty, weblogic, iis, etc.), or a when the request is routed to the service node of the new expansion, because the problem might be for the session or context, resulting in business service is not available. When tested, it must serve stateless as a check point, if the service implementation side, through by the load balancing "stick" strategy to achieve the session sharing, then there is a big problem, while reducing service node, or some service node is hung, before subsequent client requests for these services section of the service over to another node, session will be lost, the usual practice is located above the session or in containers under external service thread ( addition tomcat, jetty, weblogic, iis, etc.), as in memcache, in Redis.
How to verify that no checkpoints service it? Assumed to be a Web program by closing a single node, and then restart check whether the same session. This test can be verified in a non-cloud environment, using only a node to a test (non-clustered mode), for example, to log into a page, and ready to do some operations that depend on the session, this time, to stop function was about to operate, first stopped this node, and then restart the node ,, after the restart before logging on the page, and then do the operation for this before, have not prompted to re-login, or session expired, the successful operation can be verified; of course, also be used to verify the two nodes (trunked mode), on each node, physiognomy put two pages, in the above, print out the session id, and the name of the node, the first request for this page, remember session id and node name under printing, turn off any node, then refresh this page to see the same print session id is not, as such, but node name has changed, indicating that the service is stateless. There are other ways, but to stimulate here, to provide some ideas. Such as plain background API, as long as the check authentication information has expired, or whether it is the same authentication information.
3, multi-tenant support;
Since it is a service on the cloud, must require a different customer (tenant / unit / organization) can be used, and independently of each other, multi-tenant, there are usually two ways isolation, isolation logic and physical isolation; logical isolation means that everyone a system which was only added to a field in a table in the database layer, the data relevant to tenants; physical isolation, which each individual portion of a tenant system. When tested, it must be multi-tenant support as a Point the Check , test methods, by way of illustration of the isolation, not to know such as authentication, multi-tenant support, and in what ways does not support isolation. In the above example, the tenant successful registration, call CloudFormation interfaces, cloud Invoicing business systems automatic deployment, which is actually physically isolated.
4, failover / separator;
Cloud services, there will be a failure when, in order to protect the political influence of the failure to produce a minimum, must have a strategy to deal with failure, failover can be divided into two levels ,, one IAAS layer, a business service is its own fault transfer. The entire system is down or faulty, using the mirror, automatically re-instantiate an instance, but the network properties unchanged, which is IAAS level, appears in the PAAS, and actually there is no distinction previously, corresponds, under the traditional way, to quickly enable redundant or standby server, system, or hardware to take over their work; the other is a layer of software, business service system, when the service is not available to support the retry logic, while supporting retry, requires keeping idempotence (simply put, do the same operation on the same data, and do a do N times, the result is the same), or on the wrong service isolation, isolation will not lead to snow jump effect, or reduce the use of service Shi's fault. In short, when tested, have to failover / isolation as a check point, transfer IAAS layer, just to confirm to cloud vendors, basically cloud vendors have achieved this level, troubleshooting software side, the need to isolation strategies to perform the appropriate tests, depending on the specific details of the application and then detailed investigation, the main point is not given a statement to this test.
5, limit protection services;
Since it is a public cloud, it is intended for all of your customers, in some cases, visits will explosion, or malicious access attacks, then the reliability of the service, the implicit characterization must be guaranteed, by concurrent access / limit requests or requests within a time window to protect the system speed, the rate limit is reached once the service may be denied or queued, so that access to the service does not cause excessive collapse, which is limiting.
Test methods, by concurrent pressure measurement, or increase the amount to the limit supported by the system, the system because there is not much traffic crashes. When tested, it must serve current limit protection as a check point
6, application security;
Put service on the cloud, for the entire Internet, in addition to deal with malicious attacks, but also to prevent the server from being hijacked, but also to ensure data security, and internal unauthorized access and so on, security is a very professional one direction, time of testing, but also must be secure as a check point, testers in this regard can only do some routine safety tests, such as SQL injection, cross-site whether XSS attack, sensitive information transmitted in the clear, accessible through the API to verify whether some level of deposit, etc. high of business, but also two-way authentication, and even real-name authentication, which he will need to ask a professional security firm to conduct a comprehensive safety testing, they can scan the security vulnerabilities and insecure factors existing system, and gives good corrective recommendations .
7, call chain tracking,
This depends on whether the service is a distributed application distributed application, the presence of each system call, forming a call chain, typically a request, will lead to component A, B calls, component, B component invokes C, C calls D and possibly longer call chain, to be honest, the call chain track, a little biased in favor of operation and maintenance, at the time of testing, distributed environment, without the aid of the call chain to track some problems simply can not locate, for example, a request wrong, wrong is actually a node on which the call chain, or a function is very slow, slow in which, without the aid of tracking the call chain, you can not do anything with the program, even if let to develop its own break point, too, must dead, plus a distributed cluster, the same service, each call, call chain may not be the same. Applications on the cloud, usually distributed with, as a tester, it is necessary to trace the call chain as a the Check Point , can be distributed under the scenario proposed positioning is more accurate, more professional problems, not just at the table as the BUG on. Open source tracking call chain has zipkin, pinpoint, skywalking and so on. Call chain tracking a need to develop over there to integration, testing here to get to this point and will be used. Below is a screen shot with the last example zipkin
8, visualization services management (observable, automatic health check service elegant off, etc.)
Service management, operation and maintenance is biased stuff, his own definition, you can own Baidu; Here, I simply explain the scene up, about the meaning of governance services, your service in the cloud, reliability should be protected, the main is to prevent, can not be thrown off balance, the real problem happened, fryer, need visual way, the observed state services, health status, traffic conditions, response speed, concurrency, resource usage, etc., and according to some , automatic or semi-automatic way to start the elastic expansion, or take isolation, fuse and other measures to ensure the availability of services. In devOps popular now, and testers to rely on a little luck Victoria is not a bad thing, the test will provide more inspiration and bring more testing means. Service on the cloud. When the service is closed elegant, Incidentally, to turn off a service, the service is run in conjunction with the thread into related businesses is turned off, which means these business operations bound to fail, as opposed to closed elegant service, refer to before closing he would refuse to enter a new request comes in, at the same time to complete all the current business, was shut down, a little like a bank window, do not accept the business, but to handle the business currently being processed. Testers as a check point, can be used to verify the level of the cloud service implementation level, but also for reliability testing services related to generation test means / methods, but also get to this point.
In conclusion, the cloud is to test the application, in addition to its own business function test, but also test above 8 non-hard-core business functions mentioned points, especially in the first 3:00, it is to distinguish between true cloud, cloud false most critical point; clouds of you definitely can not reverse the trend, the test person should be related to the concept of time into, we need to keep up with development. Of course, limited individual level, knowledge is limited, there may be biased after discussion, welcome brick and additions. itest testing technology team, has been concerned about the testing of new technologies, new frontier, and to itest open-source software as a testing ground to achieve management philosophy, itest, is a convergence of 10 years of experience, process-driven test of open source test management software, we tested people develop their own test management software, reflect our feelings testing is the most understand the test person open source test management software rookie; Itest revenue team members made from have feelings for software testing, interested in open source, and enthusiasm spread to share our test ideas group members. ( Process-driven open source test management software Rookie official website )
REVIEW, start talking about cloud technology, then talk about the cloud morphology, in addition to conventional functional tests, the cloud of the test, but also there are a few must get to the hard core indicators, and finally explain the key points were hard What is the core indicators, and how to test it. This is a thought-provoking questions, hope that all people get to test them, and do more than post mentioned that extract more check point.
First look at the general trend of the rise of the cloud
Moment oracle cut across China R & D Center, is ignited, a key reason is that, by the oracle cloud computing and other emerging technologies impact their business grow weak or even decline, so pay more attention to cost control, so that strategic personnel changes, the cloud make more computing resources; today, Ali goes beyond Microsoft Azure, second only to the Amazon AWS cloud computing world's second largest company, 10 years ago in the case of Baidu and Tencent are not optimistic about, Ma believes that cloud computing is future annual investment of 100 million, without knowing these years re-energized, so Ma voted Ali cloud Wang Jian 10 years, Ma said to really put the final in 1000 only made things to do! But then the whole of China no company is willing to invest the cloud of this "illusory" in the cause, the most difficult times, 80% of engineers for various reasons, left the cloud Ali, Dr. Wang Jian ten years ago, in the eyes of many people It is a truly a "liar."
Early cloud vendors, cloud resources are mainly provided by the emphasis on iaas layer, currently with the further development of cloud computing, from iaas, to paas, serverleess plus container technology, cloud vendors have become standard products, cloud technology is an irreversible trend It has been accepted by the public, of course, subject to some restrictions on non-technical factors or some historical baggage restrictions, mixed cloud, public cloud cloudy and a long-term existence. More and more companies to service in the cloud, small companies may just simply move to the cloud service (that is, the earlier this deployment, move to cloud), in fact this is not true cloud computing, disaster recovery course ability to enhance the quality, SaaS vendor, or large companies, will be on the cloud vendors pass platform to build a layer of glue, integrating PAAS and SAAS form their own cloud management platform, transparent achieve elasticity calculation, the lateral expansion and visualized O & M monitoring and other non-functional management needs.
These hard-core test cloud of non-functional key points must be measured must get to
These non-functional hard core and key characteristics of cloud computing has a causal relationship, but also service and reliability, measurement and governance are closely related services. In simple terms you want to ensure the reliability, availability, manageability of your cloud service, you must have some non-functional indicators of the cloud, because then you business functions easy to use, unreliable, the user is not guaranteed follows eight hard core point is the cloud of non-functional indicator, a subsequent one and then were told, the hard core points is defined and how to test these points, in particular the first, second, third, three an indispensable, otherwise your software just move to the cloud, does not have the characteristics of any cloud computing, that is a false cloud.
The first elastic, also known as auto-scaling;
2, stateless service;
3, multi-tenant support;
4, failover / separator;
5, limit protection services;
6, application security;
7, call chain tracking,
8, visualization services management (observable, automatic health check service elegant off, etc.)
One example will be described with hard core 8 points and test methods or means
The figure is one of aws, and build a VPC via VPN and local services make up a hybrid cloud network architecture,
Create four subnets, each for deployment: tenant registration Web, database, Web Invoicing program, Invoicing App program; in order to achieve high availability, according to each type of subnets available each district to create a, b two a.
After successful registration of tenants, call CloudFormation interfaces, cloud Invoicing business systems automatic deployment. (Actually physically isolated)
Go back to the previous eight hard core indicators
The first elastic, also known as automatic retractable
Speaking of flexibility, start often said that the cloud server, ecs talking about, it is the abbreviation Elastic Compute Service, which is a processing capacity of elastically scalable computing server, which is from IAAS (more partial hardware resources: CPU, memory, disk, network) layer to achieve resilience, which addresses the underlying hardware or elastic resources; actual cloud services from software layer now of view, but also has its own elasticity is elastic or paas saas software layer, for example, , according to the time required scale or other strategies (based traffic, or the hardware resources of a particular baseline), as the peak of the registration service, a 10-node support to ensure peak concurrency, off-peak, and then decrease registered service node, freeing up computing resources to do other things. In the cloud, cluster nodes to achieve the expansion is a very simple matter, you can configure the expansion strategy. When tested, it must be elastic as a Point the Check , how to verify that the definition can understand from the front, will not repeat it here. Now the question is, the dynamic expansion of the service, how to make a cluster-aware, and it can be used, see item 2 below
2, stateless service;
Connect a problem, dynamic expansion of the service, how to make a cluster-aware and usable? Let cluster aware there are two methods, namely, expansion of service at the time of the cross, to the ELB (load balancing) Dynamic registration (if the extension of the node, need to manually configure and reboot the load evenly related components, this is not called elasticity) , or by registering components to achieve service discovery, which has a lot of mature technology, there is not much explanation, the point is, new horizontal expansion out of service, plus after a cluster, let him be able to provide services, which requires the service is stateless (such as session session, or other context, does not depend on where the service is the main container, such as tomcat, jetty, weblogic, iis, etc.), or a when the request is routed to the service node of the new expansion, because the problem might be for the session or context, resulting in business service is not available. When tested, it must serve stateless as a check point, if the service implementation side, through by the load balancing "stick" strategy to achieve the session sharing, then there is a big problem, while reducing service node, or some service node is hung, before subsequent client requests for these services section of the service over to another node, session will be lost, the usual practice is located above the session or in containers under external service thread ( addition tomcat, jetty, weblogic, iis, etc.), as in memcache, in Redis.
How to verify that no checkpoints service it? Assumed to be a Web program by closing a single node, and then restart check whether the same session. This test can be verified in a non-cloud environment, using only a node to a test (non-clustered mode), for example, to log into a page, and ready to do some operations that depend on the session, this time, to stop function was about to operate, first stopped this node, and then restart the node ,, after the restart before logging on the page, and then do the operation for this before, have not prompted to re-login, or session expired, the successful operation can be verified; of course, also be used to verify the two nodes (trunked mode), on each node, physiognomy put two pages, in the above, print out the session id, and the name of the node, the first request for this page, remember session id and node name under printing, turn off any node, then refresh this page to see the same print session id is not, as such, but node name has changed, indicating that the service is stateless. There are other ways, but to stimulate here, to provide some ideas. Such as plain background API, as long as the check authentication information has expired, or whether it is the same authentication information.
3, multi-tenant support;
Since it is a service on the cloud, must require a different customer (tenant / unit / organization) can be used, and independently of each other, multi-tenant, there are usually two ways isolation, isolation logic and physical isolation; logical isolation means that everyone a system which was only added to a field in a table in the database layer, the data relevant to tenants; physical isolation, which each individual portion of a tenant system. When tested, it must be multi-tenant support as a Point the Check , test methods, by way of illustration of the isolation, not to know such as authentication, multi-tenant support, and in what ways does not support isolation. In the above example, the tenant successful registration, call CloudFormation interfaces, cloud Invoicing business systems automatic deployment, which is actually physically isolated.
4, failover / separator;
Cloud services, there will be a failure when, in order to protect the political influence of the failure to produce a minimum, must have a strategy to deal with failure, failover can be divided into two levels ,, one IAAS layer, a business service is its own fault transfer. The entire system is down or faulty, using the mirror, automatically re-instantiate an instance, but the network properties unchanged, which is IAAS level, appears in the PAAS, and actually there is no distinction previously, corresponds, under the traditional way, to quickly enable redundant or standby server, system, or hardware to take over their work; the other is a layer of software, business service system, when the service is not available to support the retry logic, while supporting retry, requires keeping idempotence (simply put, do the same operation on the same data, and do a do N times, the result is the same), or on the wrong service isolation, isolation will not lead to snow jump effect, or reduce the use of service Shi's fault. In short, when tested, have to failover / isolation as a check point, transfer IAAS layer, just to confirm to cloud vendors, basically cloud vendors have achieved this level, troubleshooting software side, the need to isolation strategies to perform the appropriate tests, depending on the specific details of the application and then detailed investigation, the main point is not given a statement to this test.
5, limit protection services;
Since it is a public cloud, it is intended for all of your customers, in some cases, visits will explosion, or malicious access attacks, then the reliability of the service, the implicit characterization must be guaranteed, by concurrent access / limit requests or requests within a time window to protect the system speed, the rate limit is reached once the service may be denied or queued, so that access to the service does not cause excessive collapse, which is limiting.
Test methods, by concurrent pressure measurement, or increase the amount to the limit supported by the system, the system because there is not much traffic crashes. When tested, it must serve current limit protection as a check point
6, application security;
Put service on the cloud, for the entire Internet, in addition to deal with malicious attacks, but also to prevent the server from being hijacked, but also to ensure data security, and internal unauthorized access and so on, security is a very professional one direction, time of testing, but also must be secure as a check point, testers in this regard can only do some routine safety tests, such as SQL injection, cross-site whether XSS attack, sensitive information transmitted in the clear, accessible through the API to verify whether some level of deposit, etc. high of business, but also two-way authentication, and even real-name authentication, which he will need to ask a professional security firm to conduct a comprehensive safety testing, they can scan the security vulnerabilities and insecure factors existing system, and gives good corrective recommendations .
7, call chain tracking,
This depends on whether the service is a distributed application distributed application, the presence of each system call, forming a call chain, typically a request, will lead to component A, B calls, component, B component invokes C, C calls D and possibly longer call chain, to be honest, the call chain track, a little biased in favor of operation and maintenance, at the time of testing, distributed environment, without the aid of the call chain to track some problems simply can not locate, for example, a request wrong, wrong is actually a node on which the call chain, or a function is very slow, slow in which, without the aid of tracking the call chain, you can not do anything with the program, even if let to develop its own break point, too, must dead, plus a distributed cluster, the same service, each call, call chain may not be the same. Applications on the cloud, usually distributed with, as a tester, it is necessary to trace the call chain as a the Check Point , can be distributed under the scenario proposed positioning is more accurate, more professional problems, not just at the table as the BUG on. Open source tracking call chain has zipkin, pinpoint, skywalking and so on. Call chain tracking a need to develop over there to integration, testing here to get to this point and will be used. Below is a screen shot with the last example zipkin
8, visualization services management (observable, automatic health check service elegant off, etc.)
Service management, operation and maintenance is biased stuff, his own definition, you can own Baidu; Here, I simply explain the scene up, about the meaning of governance services, your service in the cloud, reliability should be protected, the main is to prevent, can not be thrown off balance, the real problem happened, fryer, need visual way, the observed state services, health status, traffic conditions, response speed, concurrency, resource usage, etc., and according to some , automatic or semi-automatic way to start the elastic expansion, or take isolation, fuse and other measures to ensure the availability of services. In devOps popular now, and testers to rely on a little luck Victoria is not a bad thing, the test will provide more inspiration and bring more testing means. Service on the cloud. When the service is closed elegant, Incidentally, to turn off a service, the service is run in conjunction with the thread into related businesses is turned off, which means these business operations bound to fail, as opposed to closed elegant service, refer to before closing he would refuse to enter a new request comes in, at the same time to complete all the current business, was shut down, a little like a bank window, do not accept the business, but to handle the business currently being processed. Testers as a check point, can be used to verify the level of the cloud service implementation level, but also for reliability testing services related to generation test means / methods, but also get to this point.
In conclusion, the cloud is to test the application, in addition to its own business function test, but also test above 8 non-hard-core business functions mentioned points, especially in the first 3:00, it is to distinguish between true cloud, cloud false most critical point; clouds of you definitely can not reverse the trend, the test person should be related to the concept of time into, we need to keep up with development. Of course, limited individual level, knowledge is limited, there may be biased after discussion, welcome brick and additions. itest testing technology team, has been concerned about the testing of new technologies, new frontier, and to itest open-source software as a testing ground to achieve management philosophy, itest, is a convergence of 10 years of experience, process-driven test of open source test management software, we tested people develop their own test management software, reflect our feelings testing is the most understand the test person open source test management software rookie; Itest revenue team members made from have feelings for software testing, interested in open source, and enthusiasm spread to share our test ideas group members. ( Process-driven open source test management software Rookie official website )