|0x00 Software industry sees data
From the perspective of the traditional software industry, looking at the way Internet people engage in data, it feels like a step backwards.
In the past, many software development models, such as waterfall, spiral, agile, etc., were based on the needs of users as the starting point. A large-scale project was broken down into sub-projects in an iterative manner, and results were achieved for each specific unit. Testing to achieve the purpose of rapid development. It can be said that the use of project management methods to make requirements can carry out more precise control over the quality and cycle of output results. But not everyone will develop in a unified way, so the concept of "design mode" was put forward in the follow-up, which is used to guide the concept of development that is difficult to standardize. In the course of thirty years of practice, this set of theories has withstood the test of time, and it is still in the traditional software development model and is very versatile.
Past experience tells us that the significance of technology lies in the ability to realize the "superimposability" of experience and technology based on the results of predecessors. That is, our results must be "reusable".
However, under the Internet model, Data Warehouse has been doing it for many years. In addition to mature dimensional modeling and layered models, we have been in circles. How to conduct demand research, how to design the implementation plan, how to control the quality of the development process, how to verify the results at the delivery stage, and how to ensure quality control after going online. These series of processes are not guaranteed by a systematic theory. Of course, in the stage of rapid business development, meeting demand is the first priority, but when unicorns continue to grow and eventually form a super large-scale company, the past model will not apply to all scenarios. For example, if an Internet company wants to do some industry digitization and enterprise digitization work, they will suddenly find that their model is no longer applicable. The most typical example is the structuring of artificial data, which is difficult to define clearly. Therefore, we have to learn from traditional software development and repair and patch on the basis of existing models. It looks very much like "regression of the times".
|0x01 Speed is the meaning of change
However, on closer inspection, there are still some essential differences. For example, in traditional software development, there is a concept of delivery cycle, and it is normal to use months as the delivery cycle. But in the current enterprise data center, this speed has to be reduced to weeks or even days.
From month to day, it seems that the result has not changed, but the process is completely different.
As a military fan, what I talk about most is the bomber model H-6K. As a model designed by the former Soviet Union in 1955, we have been using its aerodynamic shape to this day, so many people complain about this model as "too old-fashioned". But its core subsystems, such as engine, radar, and electronic level, have been completely replaced with the latest technological products. It can be said that except for the appearance, the H6K is no longer a product of the 1955s.
The same is true for the data center. From the first day of its birth, the thinking of Internet people to do data is to "fast". From the perspective of corporate digital capabilities, the Internet still cannot get rid of the database + backend + frontend model. People in the past, looking at the current products, subconsciously think that this is still a product of the old era, it is nothing more than a change in technology selection. But in fact, whether Hive goes to interactive or ETL goes to batch integration, the technical foundation of the product is completely different. Even the automated BI reporting tool, its data source already supports dozens of different data sources, not limited to a database.
Therefore, the idea of doing data for enterprises is still the old one, but the connotation is completely different.
|0x02 Fast means more than speed
Fast, not only refers to the fast development speed, but also refers to the focus on core things based on a set of mature models, by reducing the investment in irrelevant things, and increasing the demand development speed in disguise.
Imagine that a traditional requirements development cycle is about 21 days, involving PRD design, requirements review, functional module development, testing, joint debugging, and launch. Although everyone can accumulate experience in work experience and collaboration processes, the speed of development will not be substantially improved, because technology is not accumulated.
So what if we put our energy into the development of tools? Putting aside the cost of implementation, assuming that we currently have complete report development work, complete process development tools and complete data models, then the product can even be directly based on the existing mature data model without designing PRD, just in the report tool and It may only take 1-2 days to build a prototype on the process tool, and this process does not even require development intervention. If the function of the tool is not perfect, then the technology complements the tool capability. In this way, everything that the development students do can superimpose production efficiency, and ultimately achieve a qualitative change in development speed.
The essence of data warehouse and data analysis model is to accumulate past experience through automated processes. Later generations can achieve a leap in development efficiency on the basis of the predecessors. Of course, if the scene becomes more complicated, it is another matter.
So can the traditional software industry achieve this speed? can. However, its connotation is not for "fast", but for "continuous and stable delivery." There are differences in design ideas, so the ways and means of realization are naturally different.
|0xFF Stock market competition requires more speed
Some people may ask, now that the market has entered stock competition, there will not be so many fast-growing businesses, so the speed at which we make demand will gradually decline.
My personal understanding is that if the core of Internet thinking is "user thinking" or "fast iteration", then the essence of "fast" will not change. Because trial and error is necessary, and a lot of trial and error is necessary. In this process, "speed" determines everything.
