|0x00 Speaking from a meeting
The author recently participated in an offline exchange meeting. Not only some leaders were invited to participate, but there were also many front-line R&D engineers. Everyone discussed some specific issues with each other. The discussion process was very interesting. First of all, let the leader speak. The leader made a statement on the overall situation of the company. At the same time, he made a summary of the next management focus. After the leader gave a speech, the frontline R&D engineers in the venue looked at each other, naturally. Following the leadership's speech ideas, group discussions were conducted on the implementation of some specific strategies of the company. After the discussion, a student raised a question. The original intention of our meeting was to solve specific problems, but the meeting was over, as if we did not discuss any specific issues.
Why did you bring up this matter? Because it involves a very critical issue: the information asymmetry between the leader and the frontline. Now the company is an organization with very clear job responsibilities. For leaders, considering the company’s development strategy is essential work, but when the content of the essential work is passed on to frontline employees, although everyone can understand what the leader means, It seems that when it is implemented in work, it is always a little bit worse, because what is passed down is just an instruction, but no one tells you what to do.
The key to the solution lies in the middle management team. However, due to the development model of the Internet, people who are promoted from the front line to the management position are often those with outstanding business or technical capabilities, and the entire promotion process hardly revolves around management capabilities. Therefore, become this group of professionals. After entering the middle level, most people find it difficult to adapt to the change of identity. For the management of the team, either fish or pretend, and the middle level with good management skills often spend less time in the professional field. As a result, a middle-level professional with good professional skills makes people very irritable, and a middle-level manager with good management skills is often not so convincing. It is difficult to be a good middle-level manager.
But since the reality is unreasonable, there will be opportunities for change, and this kind of opportunity depends on whether the current frontline students can be keenly grasped. The author previously wrote an article "Number One is a State of Mind, Not a Rank", which mentioned that everyone should have a mentality of one to do the most common things. In fact, in most work scenarios, it is really unnecessary to be able to do your own job well and to be nosy. But if you want to cross the "middle-level trap", learn to find a balance between management ability, understanding of company strategy, professional ability, and landing ability, and lay the foundation for yourself to move to a higher level, these "messages" are not a waste of time. The behavior is an opportunity to exercise one’s EQ.
In any field, investing 10,000 hours will make a career.
|0x01 Why model reconstruction is proposed
The author has a data background. In the field of data warehouse, there is often a need for "modeling". In fact, in the field of software engineering, the design of "models" is also very important. But in addition to "modeling", there is another problem that will accompany it, which is "model reconstruction". Why does "model reconstruction" occur? Most people don't have in-depth discussion and research. Most of them are done as a job or as a scheduling requirement, without really exploring the reasons behind "model reconstruction".
Under the influence of the mentality of the number one position, let us look at the background that "model reconstruction" usually faces:
- Incomprehensibility: Many people will have a mentality of "incomprehension" after taking over a new business, such as why there are so few commented documents, what is the reason for such a design, the code specification is really bad... unless it is For long-term investments, most businesses will give up certain standardization and stability in order to improve the efficiency of research and development in the early stages of development. Although experienced architects or modelers will develop a big picture of the business framework at the beginning of the business, as time goes by, various unexpected situations will inevitably occur, which makes us have to fight the original architecture. Patch, and continue for a while, the model will not understand.
- Total errors: Another reason for model refactoring is that there are always problems in the business. Whether it is inaccurate data calculations or always reporting errors in software, it is always at a certain stage of architecture development that problems will be exposed intensively. The original intention of model reconstruction is not necessarily for the sake of stability, but if there is a problem with stability, the model will definitely be reconstructed.
- Slow running: This one is for the discussion of data business, because there is no model that can be designed very well in the early stage of business development. A good model must be accumulated with experience in the process of continuous business development. As far as the data model is concerned, when the original model cannot continue to support business development at a high speed, it is necessary to reconstruct it.
- No value: This point will be ignored by most front-line engineers, but when reporting their work results, they are often used to "whip the corpse". Yes, how is the data model designed? In many cases, it is not just whether it is stable enough, or whether it will go wrong, but what value this model can bring to our business, even through this model, How can it drive business development?
Simply put, "model reconstruction" is just compensation for past "technical debt". But the core element of the reconstruction process is "how technology adds value to the business", not just quality, stability, and efficiency.
|0x02 What to consider for model reconstruction
Regarding whether the model design is good or not, you can refer to the previous article: "How the data model is good or bad". This module will talk about some more advanced theories, namely "structural thinking".
For a front-line engineer, after working hard for a whole year, what he fears most is not the hard work, but the result of his own hard work is not recognized by the boss, or the result of the output is of little significance to the business. And since we are going to refactor the model, we must learn to reflect on our past defects in structured thinking, such as reporting whether our work can be systematically discussed, and whether the things we have done are systematic and a learning process Is there an important point as a starting point, etc. If you use a more grounded way to explain structural thinking, you must have a "routine" in doing things. Don't underestimate the routines, because the technology or ability can not directly bring out the results, and the routines are the fastest way to achieve goals and produce value.
Most people don't care about what you do, only what results you produce.
"Routine" is not entirely a summary of past experience. It is a bit like machine learning. It refers to the ability to predict the future course of action by decomposing the central elements based on the accumulation of past experience, that is, "behavior methodology". For example, 5W2H is a "methodology" that helps us analyze problems. If you can speak out about what you are doing from the seven aspects of Why, Who, When, Where, What, How, How much, I believe you have a very in-depth understanding of the business. Use your full strength to prove yourself and report well.
In fact, 5W2H is a kind of structural thinking, and structured is the simplest, clearest and most effective means of summarizing experience.
Next, let's talk about it from a theoretical level. The first is to establish the central idea. Any refactoring of the model will definitely focus on solving a certain business direction, usually from two aspects:
- What is the main goal of this business direction?
- Why should I design this model?
If the business direction is a relatively mature and stable model, such as e-commerce, then its first appeal must be to apply mature design solutions in the industry, and to stabilize the output time and ensure design quality as much as possible. And you are responsible, most likely because you are most familiar with this aspect of business, or you have worked in related fields before, and you can quickly implement the experience of your predecessors.
For example, using SWOT analysis, we can quickly analyze the current situation.
For many small companies, being able to use the mature models of large companies to make the business run quickly is a kind of technological value-added. For large companies, business needs more innovative exploration. The root of the contradiction between R&D and business lies in the balance of supply and demand, and the improvement of efficiency is a kind of value-added technology. Of course, technical personnel can also directly put forward valuable ideas. In fields such as Internet advertising, business growth often comes from the spontaneous contributions of technical students. At this time, we must pay attention to cultivating organizational culture so that students in different fields can have a seat. The opportunity to solve problems together, to find the point of technological value-added in the collision of ideas.
When we have the central idea, we must disassemble the structure of the sub-categories, and according to a certain order, conduct a reasonable analysis or reorganization of the sub-categories to find the best plan. If you go back to the topic just now, it's probably the disassembly sequence:
At this time, I will report to the boss and the business side again, and follow the dismantled sequence, and carry out a structured narrative. What can be said clearly in half an hour can be reported in 5 minutes.
After dismantling until here, we return to the model reconstruction itself. In the traditional data warehouse hierarchical structure, the ODS layer is the guarantee of data output time, and monitoring is often done at this layer; the CDM layer is the basis for the rapid development of business, if there are no special circumstances, the CDM layer should be high Stable, if the public layer is frequently modified, something must be wrong with the model design; the ADS layer is the guarantee for the business to run, and the data can be processed in any dimension to support flexible use downstream.
Subdivided further, is the application of high cohesion and low coupling in the SQL development process. For example, GRASP method, refer to the previous article "Architecture Methodology".
|0xFF Solve the complexity dilemma
However, structural thinking is not a panacea. In many cases, if the design is improper, it will fall into a "complexity dilemma".
Since SQL development is not like software engineering, object-oriented design ideas can be used flexibly, and dimensional models are more like the application of process-oriented ideas in the data field. Therefore, in some very complex business scenarios, a large number of CASE WHEN statements will appear . At the same time, many hard-coded configurations cannot be configured through a public Constant class, and making a modifiable dimension table is against the specification. Therefore, the rapid development of business lags behind, and many ADS layer data will fall into a complex dilemma. , Not only is it not easy for others to understand, but it is not easy to locate the problem by yourself. You have to look through the source code to know the original design details.
Therefore, for the governance of complex businesses, it is not a clear and straightforward methodology, which tests the accumulation of experience of engineers.
But everything has some experience that can be used for reference. For example, in the face of complex difficulties, you can try to gradually dismantle it from the following process, and use structural thinking to try to govern:
Business understanding -> domain modeling -> process decomposition -> multidimensional analysis.
Business understanding is the most important thing to dismantle the complex dilemma. For example, e-commerce has been developing for so many years. Everyone knows the meaning of words such as SKU and ORDER. However, in some relatively new fields, such as new manufacturing and the pharmaceutical industry, it is not so easy to figure out the essence of a thing. At this time, you must be familiar with the business you are doing, what is the current situation, what is the direction of industry improvement, What are the pros and cons of different solutions? This is very helpful for figuring out possible pits in the future.
For domain modeling, refer to the previous article: "A Brief Talk on Domain Modeling".
Refer to the previous article for process decomposition, but here we must emphasize that in the process of process decomposition, we must figure out which parts are stable and which are constantly changing. In many cases, for the convenience of development, a lot of business logic is often mixed in the public layer. Although the downstream is cool, once it encounters business changes, or the scene is inappropriate, and then go back and modify it, it will be very difficult. Uncomfortable. Therefore, although redundancy is necessary, it must be redundant.
Finally, there is multi-dimensional analysis. In most cases, the system is complicated because of the interdependence, mutual reference, and interrelation between business lines. If an analysis framework has a single scenario and no branch processes, there will be no governance problems. Therefore, only by drawing more pictures, drawing good pictures, drawing good pictures, and drawing the business process clearly, can we know where the focus of improvement is.
