The importance of safety in production
Production safety is our bottom line, with the continuous expansion of business, a wide range of services change, fault lines began to question and frequent; these faults to the customer and the company has brought immeasurable loss, not only affect us in the industry word of mouth, it will affect the trust of our customers.
We need to think, why people use your service? If your service fails, the user what kind of repercussions? How do we stop the failure, reduce the impact on customers, maintain our values?
How to improve the safety in production
Work Safety will promote the availability of our services, to minimize line accident rate; from what we should improve production safety, and to protect our customers, protect our students do produce mainly concentrated in the general direction? 技术, 架构, 系统, 工具, 组织, 机制, 文化on.
- In the beginning of system design, we will do as much as possible
系统规划, to do架构优化, to simplify the architecture; the same functions of the system as much as possible merge merge (系统改造), reduce the cost of learning systems; - All systems must have disaster recovery capabilities (
容灾建设) - All of the code, when the line must be to ensure the quality (
编码质量)
After some data analysis, 80% of the line failure is due to changes caused, so the change of control must be cut to reduce a major fault line; they must be careful and cautious when changing online us;
- Change control: We must change strong controls on all lines
- Gray capacity: All updates and on-line system must have the ability gradation
- Monitoring Optimization: All systems have a comprehensive monitoring coverage of all monitored alarms must converge, not let the major key alarm is drowned out
- Contingency plans: for all system failures and the scene of some of the problems occur or might occur, they should have contingency plans;
- System Walkthrough: not enough light has contingency plans, contingency plans must be regularly for these exercises to ensure the reliability of the plan to ensure that when problems can quickly be executed
- System inspections: regular inspections of the system to do, identify potential problems, such as: load, performance ...
- Line pressure test: the system periodically pressure test, the verification system function, but also play an active role on the capacity planning system
Standardize two aspects of the above (design, on-line), we have been able to deal with failure and the risk of line, but this way the method has not been effective implementation and control, may spirits high, we execute it.
So we have 安全生产组织such a department or organization to ensure that these norms and mechanisms work well in production by the organization to ensure that our technology architecture and tools can truly fall, to solve the real fault line by means of security technology;
With the assurance organization, at some point we have to regulate from our operations:
- Specifications: specification change, gradation specification
- Red line: For safety norms do the red line, if not performed in accordance with the specifications area, it is possible to trigger the red line,
- Accountability: red trigger, resulting in failure, we need to have accountability standards (P, D, E ...), these standards can be derived from the business availability, safety production strategy, such as failure affects
The concept of safe production
- Failure-oriented design: we were designing the system, we must take into account the circumstances and extreme system problems that may arise and have the appropriate disposal options and planning.
- No gray, no change: all changes must be able to on-line through the gray environment and our batch release, but there must be a separation between batches released
- Change tricks 3: gray can be observed, the rollback