An increase in the number of services to a certain extent, the possibility of the problem is growing, now means a variety of common architecture, high availability means are designed to enhance the usability of the system, to provide users with a better experience. The full link is equivalent to the pressure measuring service do a physical exam to understand the current state of the system
Definition: online and real-based business scenarios, by simulating a user request massive to pressure testing the entire link.
purpose:
- Verify that the new features of stability
- Verify the stability and flexibility at the peak flow and services
- Online service for more accurate capacity assessment
- Find bottlenecks and optimize system
Pressure test standard limit
- load does not exceed (machine Audit * 0.6)
- NIC NIC traffic does not exceed the capacity of 0.6, more than words could delay large
- Request timed out no more than one hundred thousandth of the total amount requested
- QPS is not less than 85% of the estimated, or need to optimize, or give a reasonable explanation
Pressure sensing scheme
It is a more realistic simulation of the environment, the pressure sensing line machines and machine the same configuration, deployment line modeled machine deployment. Using the measured pressure data as real data lines.
Option One
Multiplexing online environment pressure measured in the low peak period, such as three in the morning, playback read request, write request can not be measured pressure, because write requests cause data pollution.
Local pressure measurement may be used everyday environment, or with online environment:
-
Daily environment: low demand, if you want a real effect, and online services can build exactly the same facilities, the disadvantage is the high cost
-
Online Environment: completely online environment, measuring compressive capacity of the machine, the flow rate gradually dispensed onto fewer servers, more than 10 minutes was observed, until the server process limit.
- We need strong pressure measurement platform
- Three-dimensional monitoring system
- Service management platform
- Major companies can participate in the full link pressure measurement system, at the end of the reference in the text.
Traffic multiplexing tool: TCPcopy
Option II
Option One is hard pressed to carry out measurements of the entire cluster, easily sweeping, unable to assess the true performance of the system. If you want the full link voltage measured:
-
Construct real data as possible
-
Pressure test online real environment
-
Core Technology
-
Pressure sensing identification passthrough
- Cross Thread : The InheritableThreadLocal parent thread a ThreadLocal variables passed to the child thread, ensuring the delivery pressure measured logo
- Cross Process : stored in the Header of the request, do some identity.
-
Pressure measurement service isolation, not because of pressure measurement affect the normal service
- The business needs to create a pressure sensing line grouping of the entire link, isolate a group of machines for measuring pressure, split at the entrance of the request, the request may be
-
Pressure measurement data isolation, without affecting the real data
-
Using shadow tables for data isolation, using the same online database, only when the write data to write data to another one "shadow table".
-
At last
Appendix Internet giant gave a lot of real cases, we can learn together
reference
- Link-pressure measurement platform (Quake) in group practice in the United States
- Practice measured pressure drops full link
- Like full link has actual measured pressure
- Ali Middleware - Link-pressure measuring arms
- Pressure sensing full link jingdong
- Pressure sensing full link logical thinking
- What links hungry full pressure test
Reproduced in: https: //juejin.im/post/5d08502fe51d4576bc1a0e01