Author | Mingming Ruyue Editor | He Miao
Listing | 《Opening》（ID：CSDNnews）
In software engineering, automated programming has long been considered the holy grail. Since the birth of compilers, the goal of automatic program generation is to translate high-level languages close to natural language into low-level languages, and the ultimate goal is to automatically generate programs according to the requirements described in natural language. Although this question has been explored in academic circles for fifty or sixty years, it is only in recent years that significant progress has been made.
With the advent of GPT-4, automated programming ushers in new possibilities. In the future, human-machine collaboration to improve development efficiency will become the mainstream development trend. The powerful capabilities of AI make many programmers worry about being replaced, but the industry believes that AI is not a replacement for programmers, but a tool to help them improve productivity. Which programmers will be hit harder by this AI revolution? Who is the beneficiary of this technological revolution? How will it change the way programming interacts?
With all kinds of questions, "Let's Talk: Will AI Programming Replace Programmers?" Which is better? "During the live broadcast, we invited Jiang Tao, the founder and chairman of CSDN with more than 30 years of programming experience, and the founding partner of Geekbang Venture Capital; Peng Xin, deputy dean of the School of Computer Science and Technology, Fudan University; Huawei Cloud Intelligent Software Wang Qianxiang, the chief research and development expert; Wu Hengkui, the founder and chief scientist of Supersymmetry, gathered online to discuss in depth and bring some guidance to programmers in the AI era.
Among them, the exciting preview:
Jiang Tao: People who are greatly affected by AI can be classified into three categories: one is programmers whose jobs will be replaced; the other is people whose skills will be enhanced; the other is practitioners in the new blue ocean market;
Peng Xin: Advances in technology are gradually raising the boundary between simple labor and complex labor, and less creative development activities are easily replaced. Humans still play an important role in the last mile of software development. It takes people to find and fix the last 1% of problems and take the blame;
Wang Qianxiang: How to describe questions and prompt words may become a field worthy of research, and may even develop a new discipline;
Wu Hengkui: AI models will have a huge impact on the improvement of human intelligence, which will have a major impact on human society in addition to productivity improvements.
How to establish automation programming standards?
ChatGPT demonstrates strong programming ability, but how to measure the strength of AI programming ability. CSDN once proposed five levels of automated programming:
The first level (C1): Autocomplete based on the current line of code.
Second level (C2): When writing code, the AI can predict the next line of code.
The third level (C3): Generate code based on natural language; complete programming language translation function based on natural language.
Fourth level (C4): highly automatic programming. Projects and annotations can be generated based on natural language, modules and annotations can be generated based on natural language, functions and annotations can be generated based on natural language, functions, modules, and project granularity automatic test generation; mainstream programming languages can be translated correctly; the next line of code can be generated based on the current line of code ; Code debugging (bug location and correct correction suggestions); Automatic completion based on the current line of code; Code inspection (natural language prompts).
Fifth level (C5): Fully automatic programming. Can generate system and annotations based on natural language; generate projects and annotations based on natural language; generate modules and annotations based on natural language; generate functions and annotations based on natural language; generate automated tests at the granularity of functions, modules, projects, and systems; best in all programming languages Mutual translation; generate the next line of code based on the current line of code; code debugging (bug location and automatic correction); automatic completion based on the current line of code; code inspection (accurately prompting problems in natural language); automatic optimal error correction of the code.
Yan Hui: What do you think about the standard of automated programming?
Jiang Tao : GPT-4 or GPT brings a new ability to programming - the ability to use precise language to describe requirements and convert them into codes, modules, or mature systems. This ability is not very obvious in China, but it has become very popular abroad. It is called Prompt Engineering (Prompt Engineering). Some friends in Silicon Valley told me that many programmers may face challenges. The historical changes are huge, requiring new technical requirements and capabilities, and new modules also require new skills. GPT-4 is an assistant, but this assistant will replace part of the existing work, so there is no need for so many assistant programmers.
Peng Xin : These levels basically reflect the development process of intelligent programming. Traditional IDEs have simple code completion. 3-5 years ago, it was very difficult to generate complete functions, but starting with Copilot, code generation has been achieved. The shock that ChatGPT brought to me is that it can not only give the code, but also tell you the logic of the code, and even change the code, such as adding a cache at the right location and explaining why you chose to add the cache at this location.
From the perspective of programmers' long-term engineering development, such as engineering maintenance, code understanding is still very important. It is precisely because of this that humans still play an important role in the last mile of software development. It takes humans to find and fix the last 1% of problems and take the blame . In the future, intelligent programming can be further enriched. In addition to generating a large amount of code, it can also provide functions such as explaining and repairing defects.
Wang Qianxiang : It is meaningful to publish this evaluation standard. And whether the assessment is reasonable is crucial, otherwise it will lose credibility. In order to ensure the reasonableness of the evaluation, you can refer to the practice of others, such as GPT-4. There are a lot of mature evaluation data and leaderboards in the NLP field, but the lack of evaluation standards in the code field is a problem that needs to be solved. The fifth page of the GPT-4 report lists a large number of evaluation data, such as SAT exams and law exams. You can learn from these evaluation criteria to build your own evaluation system, the first line of which talks about Codeforces evaluation. AlphaCode, the artificial intelligence developed by DeepMind, participated in 10 programming competitions held by Codeforces, and achieved the top 54.3% results, beating 46% of the participants. In contrast, GPT-4's rating in Codeforces is only in the bottom 5%, indicating that its programming ability is far inferior to AlphaCode. In the early days, the ability of Codex was evaluated using a test set called HumanEval, which contained 164 hand-written items, and this method was meaningful for evaluating human programming ability.
If you want to evaluate the programming ability of AI, you can first look at how human programming ability is evaluated. This is a basic point . Of course, this method also has some shortcomings. The competition questions will not use those very complicated libraries, but in practical applications, many public and private libraries will be used. In this regard, we have developed a new evaluation method, CoderEval, which is now open-sourced. It sets standards for evaluating AI capabilities from multiple aspects. We welcome your valuable suggestions.
Wu Hengkui : The evaluation standard released by CSDN has five grades from C1 to C5. This is an evaluation standard that programmers rely on when looking for programming aids. It can better evaluate the helpful role of the model in work. It is a very good direction . The academic standards mentioned by Mr. Wang are more extensive and applicable to various fields. The benchmark (standard or data set) made by Huawei is also a very good example. As an evaluation benchmark in academia, it has been widely used in article publishing and large-scale model testing. C1 to C5 can also be used in combination with evaluation methods in academia, and perhaps more integration points can be found later. In the financial field, we have also proposed a Chinese financial downstream task evaluation standard, and domestic attention to this type of evaluation standard is relatively low.
New technologies bring anxiety or opportunity?
Yan Hui: When a new technology emerges, it may bring changes to the whole society or the type of work. The emergence of muskets in history led to the disappearance of the knight class, civilians became warriors, and the entire social regime also changed. When new technologies such as artificial intelligence GPT emerge, the powerful capabilities it exhibits bring many challenges to programmers. Which type of programmers is more challenged? Which category is less challenging?
Peng Xin : I have discussed this issue with many colleagues. From the perspective of academia, opportunities outweigh anxiety. It's exciting and refreshing to see a breakthrough in artificial intelligence in the field of AI programming after a few years of silence.
Historically, human technological innovations have often changed the boundary between simple labor and complex labor. Brooks pointed out as early as when he wrote "The Mythical Man-Month" that software development needs to deal with essential difficulties and accidental difficulties, and there are simple labor and complex labor. The essential problem is to clarify the problem, such as decomposing the problem of large-scale software and giving an appropriate design. It is relatively simple to translate the problem into achievable code, this part is relatively simple and easy to be replaced. This change started more than ten years ago, and some companies began to use low-code development to save time. Although this change was subtle and did not cause much shock, it also replaced the positions of some junior programmers.
In contrast, why does ChatGPT have such a deep impact? Because their appearance makes this trend more obvious and has a greater impact on people. The advancement of technology is gradually raising the boundary between simple labor and complex labor, and less creative development activities are easily replaced .
At an industry conference, I once put forward a slogan: " The advancement of artificial intelligence can help reduce repetitive labor and thinking. The biggest waste in software development is the waste of knowledge and the waste of repetitive thinking ." In fact, many problems and achievements have been considered and solved by predecessors, but they have not been brought together before. The emergence of artificial intelligence technology makes up for this deficiency, allowing us to better use and share existing knowledge and knowledge. results, thereby reducing repetitive thinking and labor.
In general, I think AI can replace some relatively simple programming tasks, especially those that do not involve complex designs, and where the requirements are already very clear. But humans still need to confirm and test the correctness of the code and make changes. And the workload and number of people will definitely be less than before.
Wang Qianxiang : I think there are three types of programmers who are less challenged.
The first category is those who are not afraid to take the blame . If something goes wrong with the program written by AI, the responsibility ultimately rests with the human. If you are someone who is willing to take responsibility and take the blame, then you probably won't lose your job. AI models are based on statistics, and there are always mistakes, but the probability is different.
The second category is programmers who write code for non-generic systems . Today's software is rarely independent, and all need to call other systems or interfaces. A lot of software needs to be built on existing systems. If your system is non-universal, or you have special underlying knowledge, you are not easily replaceable. Because the model generated by AI may not have seen these special situations and cannot generate suitable code.
The third category is programmers who do architecture . Requirements are always changing, old requirements can be met with AI-generated code, but new requirements require new architectures. At this time, AI may not have seen such an architecture and cannot generate suitable code. Therefore, it is relatively safe to be a post of architecture.
Wu Hengkui : In the short term, it is very unlikely that programmers will be eliminated on a large scale. AI mainly helps programmers improve productivity. In the actual industrial production process, it is very difficult to truly realize end-to-end automatic programming, such as using a large model to completely develop a product . Although OpenAI has shown some cases of writing websites and games in natural language, and GPT-4 has demonstrated examples of generating websites graphically, achieving end-to-end automatic programming is still just a demo.
Its role should be to greatly improve the productivity and efficiency of programmers. It can find relevant information more quickly and accurately, and automatically complete codes to help programmers improve productivity. In the long run, this kind of AI tool can help many people who don't have enough time to become professional programmers to master programming tools faster to solve problems. This means introducing more people into the programming community, turning it into the level of half a programmer, thus greatly improving the production of the entire society.
In the future, when very large programming projects can be written with large models, will a large number of programmers lose their jobs? We need to reconsider this issue.
Jiang Tao : We can classify the people affected by it into three categories: one is programmers whose jobs will be replaced; the other is people whose skills will be enhanced; and the other is practitioners in the new blue ocean market.
The first job losses will be those working on single models . It's not necessarily that the career they worked for is gone, but that their work goals are gone. ChatGPT can already hang many single models, and its performance in areas such as translation is far better than single models such as Google Translate and DeepL. This is a sweeping advance that has had a huge impact on single-model research. As a result, many people need to change work tasks, learn new knowledge or use new tools.
The second type of people who are prone to unemployment are those whose needs can be clearly defined, such as employees of outsourcing companies . There are two reasons: first, if outsourcing companies adopt AI tools, their work efficiency will be improved; second, after customers use AI tools, their efficiency will also increase. Because their tasks are relatively clear, outsourced programmers will be hit hard.
Thirdly, AI will also bring new opportunities, such as promote engineering (prompt word engineering) . If the GPT large-scale model is compared to an operating system, the exposed API is to input a string of text prompts, and then get an output. But the words aren't as fixed as API programming used to be, which is an interesting change. As Hengkui said, if we want to complete a task from end to end, there is still a lack of middle-level tools, that is, promote engineering, which is one of the new opportunities brought by AI. These new opportunities stem from GPT being an "operating system" that requires a richer ecosystem and a variety of third-party tools and applications. In this process, more new development opportunities can be discovered and seized. We can expect the emergence of so-called GPT-native apps. Just like the early development process of mobile applications, the initial simple application has gradually evolved into an application that can give full play to the advantages of large models.
Who will be the beneficiaries of this technological revolution?
Yan Hui : In this technological revolution, some programmers will be affected, so who will be the beneficiary?
Peng Xin : The answer is obvious. Of course, the boss will benefit first, because technological progress can reduce costs. The second is the programmers who have not been eliminated.
With the emergence of large models, many application tools will be developed. The large model can integrate the general knowledge on the Internet and the specific domain knowledge within the enterprise to form a unified intelligence center and provide a unified portal for developers. Programmers who are not eliminated will benefit from these technological advances.
Those with less experience but flexible minds will also be beneficiaries . Take an architect as an example. An excellent architect usually has many years of development experience, has dealt with different types of systems, and has faced various challenges. It may take many years for a novice to learn these experiences. If a person is inexperienced, but good at interacting with the large model, can accurately grasp the prompts of the large model, is good at expressing questions to the large model, digests and absorbs the answers of the large model, combines these information with the context of the current project, and quickly forms specific solutions, then the capabilities of such people will increase rapidly. This is just a possibility, not necessarily entirely plausible.
Wu Hengkui : The vigorous development of large models may bring great changes to the future education field, especially the popularization of computer use and programming knowledge. More people who originally thought that computers were out of reach can master this tool faster. This educational change will affect all fields, from education to industry. Especially in the industrial world, the cost-effectiveness of large models to improve productivity will be obvious.
Jiang Tao: Those who will benefit the most will be the first batch of companies to adopt this AI technology, but it remains to be seen which industry or type of work will change the fastest . Science and technology are changing with each passing day, join early and benefit early. Although many people may be anxious because of this, more importantly, it opens up a new blue ocean. In this new blue ocean, it is not clear what kind of results can be bred. In the short term, finding an entry point can benefit, but whether this benefit will be long-term remains to be questioned. Many parents may feel anxious and worry about their children's future, but I suggest that parents let their children use these technologies as early as possible so as to benefit early.
Coding education will no longer follow the old paradigm
Yan Hui: What impact will this AI technology revolution have on the learning methods of programming education? Will there be a different learning model in the field of programming?
Peng Xin : In terms of programming learning, the level of thinking should be gradually improved. 20 years ago, people would start with grammar and explain how to write programs. In the future, ordinary students may not need to start with data structures, algorithms, and basic programming grammar as they do now. This part may be taken over by machines or AI. Programmers are more tasked to express their intentions clearly. Computational thinking may become Become a general ability and be strengthened . In a specific field, considering aspects such as resource constraints and time complexity constraints, excellent programmers are still needed to complete. Therefore, we still need to train some excellent programmers in the current way.
Wang Qianxiang : Indeed, the proportion of programming education courses needs to be adjusted. Some older knowledge, such as assembly, may need to be compressed, while new knowledge requires intensive learning.
Future programming may rely more on prompt words, and needs to be clearly expressed. How to describe problems and prompt words may become a field worthy of research, and a new discipline may even be developed to study how to use a language that is closer to natural language Describe the problem in a way that makes large AI models easier to understand.
Wu Hengkui : My point of view is different from yours. This view comes from a Twitter discussion between OpenAI CEO Sam Altman and Chief Scientist Ilya Sutskever. We believe that the prompt (prompt word) is only a transitional stage, because the large AI model has not yet fully understood the limitations of natural language, and the future development direction is definitely moving towards a more powerful natural language understanding ability .
Regarding the issue of education, we believe that AI models will have a huge impact on the improvement of human intelligence, which will have a major impact on human society in addition to productivity improvements. The improvement of intelligence level may trigger a huge change in future education, and the next generation of children will use computers as the most basic general education skills and knowledge . With the help of large-scale AI models, existing educational resources and human intelligence resources can be greatly supplemented, thus triggering major changes in the field of education.
The way of interaction changes accordingly, expanding the boundaries of programming
Yan Hui : Although computer technology has undergone great changes, the purpose of writing programs is still to solve people's basic needs. What changes may there be in the needs and application scenarios of programming solutions in the future?
Peng Xin : Software development ultimately serves the needs and scenarios of the real world. AI programming does not change this. Code generation is just a sub-node in the decomposition of software development tasks.
AI is better at generating some relatively simple code, and AI may be of limited assistance when dealing with more complex software development tasks. From the perspective of academic research, we hope that future tools can be based on large-scale AI models, through intelligence on independent issues, and connect various parts in series to achieve more seamless, more automated, and more systematic intelligence programming .
The fusion of computing and humans, machines, and things parallels to a certain extent the development of intelligence in large models. In the future, software programming objects will no longer be limited to the data structure and storage inside the computer, but will cover various devices connected to the Internet of Everything, such as coffee machines, doors and windows of buildings, sensors, voice speakers, and robots. These devices can be packaged through APIs to enable application innovation. We pay attention to changes in the final application form, especially changes on the device.
An influential article in Meta proposed to let the big model "long hand", that is, let the intelligence of the big model be transformed into practical actions. Large models can be incorporated into real-world problem solving, for example in conjunction with robotics. There are already studies exploring how to use large models to better program robot behavior.
The increasing digitization of the physical world and the idea that software defines everything has become a reality. Things in the physical world are programmable through software packaging and network access, which greatly expands the programming space. In the future, this kind of integration will undergo some chemical reactions and bring about huge changes .
Jiang Tao : In the past, Microsoft has been advocating natural human-computer interface. Now this concept has evolved into LUI (Language User Interface), and language-based applications will gradually rise. New types of computing devices and scenarios that are more suitable for LUI interfaces may emerge in the future. The front-end interface needs to better combine LUI with the user's intent and goals to create more vertical scenarios and new applications .
Just as Didi Dache uses the positioning capabilities of mobile phones to generate huge applications, the capabilities of large models such as GPT may give birth to killer applications in the future. Microsoft has integrated GPT into Office, which is of great significance for improving information productivity and digital productivity. Its potential is still to be tapped.
Wu Hengkui : The greatest achievements in the current information society are applications such as search engines and Douyin. The development of large models not only focuses on information processing, but also on simulating human intelligence. Where exactly are the biggest or killer apps for large models? An important direction is the continuous approaching or surpassing of human intelligence by large models . Because all industrial production and applications are based on science, if the big model surpasses humans in understanding and discovering science, it will go deep into industrial production in various industries.
The combination of large models and robots will have a substantial impact in the physical world, perhaps far greater than recommendation engines and information retrieval. Because most of the production is still carried out in the physical world at present, if the large model can grasp the knowledge required for industrial production more deeply, it will enter the field of industrial production and directly affect various industries, such as manufacturing and metal processing. With the help of this high-intensity intelligence system, human society should establish a knowledge-centered society and get rid of an information-centered society. In the future, everyone can master the knowledge of all disciplines . I think the development of AGI can bring an opportunity to revolutionize human society.
