AI Programming and R&D Effectiveness Forum Excerpts from Notes

After the big model came out, 90% of the skills seemed to have lost value, but due to leverage, the remaining 10% of the skills were magnified 1,000 times. Kent Beck

Table of contents

Areas covered by LLM

Technologies involved in LLM

Key issues and technical challenges

Challenges facing businesses

Is AI programming a silver bullet for software engineering?

Areas covered by LLM

NLP/ML
Prompt project
knowledge engineering
multi-task scheduling
Algorithm optimization

Technologies involved in LLM

Algorithms: deep learning, reinforcement learning, transfer learning
Large language model pre-training technology, including building a pre-training system
Efficient parameter fine-tuning technology
Data governance technology, including pre-training data and fine-tuning data
Model scaling techniques
Security, trustworthiness and robustness

Key issues and technical challenges

Key technology	Specific content	category
Chinese-friendly code generation	Under the premise of ensuring model performance, enhance the understanding of Chinese semantics and meet the same code generation capabilities using Chinese/English descriptions.	Model optimization
Prompt	Determine the completeness and rationality of the task description entered by the user, and clarify the intention through interaction to improve the accuracy of code generation. Prompt combines accurate context to expand the perceptual field of model generation.	Model optimization
AI Agent Exploration	Combined with pre-trained models and under Agent logic, accurate generation and reasoning of complex tasks are completed. Combined with the memory module, search enhancement is performed to reduce model illusion and improve answer accuracy.	Model optimization Engineering optimization
Experience evaluation and optimization	Construct evaluation indicators/means that are objective and close to real projects	Engineering optimization
Model online learning	Fine-tune large online models based on explicit and implicit feedback from users to achieve real-time updates of online models.	Model optimization
Low cost SFT	How to achieve rapid and low-cost construction of training/validation data sets for various R&D scenarios, as well as model training and automatic verification deployment Model parameter plug-in and other technologies enable low-cost training and data isolation	Model optimization
Post-processing	Check and fix compilation and run errors of the generated code according to the project context Combined with unit testing, fix logical errors in the generated program	Model optimization Engineering optimization
Model quantification	Under the premise of ensuring that the accuracy does not drop too much, the model is quantified and the end-side measurement computing power is supported to achieve model inference.	Engineering optimization

Challenges facing businesses

Huge investment in computing hardware
1. > Taking 2012 as a watershed, the demand for computing power of AI models has increased by 300,000 times in just 6 years.
2. >GPU hardware performance doubles every year.
The cost of building computing power is high
1. >Today’s AI load requires a single rack to have a power supply capacity of 20kw-40kw, which may increase to 80kw in the future.
2. >Most companies currently have a single rack power supply capacity of around 7 to 15kw.
Diverse computing power requirements
1. >Single-machine single-card, single-machine multiple-card, multi-machine multiple-card training resources require diverse resource occupancy rates and high utilization rates
2. >There are a large number of developers, high occupancy rate of development environment resources, and low utilization rate

Is AI programming a silver bullet for software engineering?

It may be that the current AI programming is quite mature. In a limited time, to solve a programming problem, professional contestants can provide a perfect solution code within 15 minutes with the assistance of AI. But one thing you need to realize is that for projects with high call depth code, such as C++, the function call depth of the project can go as deep as 10 or even 18 layers. Limited by the target field of view, it will be difficult to achieve the expected results using AI programming at this time, which is also a problem that needs to be solved in the future.

This is a question from multiple angles, because now, to a certain extent, many Internet companies have begun to use AI programming; but many practitioners are skeptical about AI programming and have questions about the reliability of coding.

Perhaps one day in the future, AI programming will really reach a very high level and be recognized and applied by most people in the industry. At this time we can look back and say, This is indeed a silver bullet, a universal solution, and a huge change in the industry.