TL；DR。
In brief, this article explores large model-driven standardization of software engineering practices and how requirements and designs can be normalized into a DSL format. In this way, AI can be written more automatically and efficiently.
With the application of large language models in software development more and more widely, traditional software engineering practices began to be re-focused and mentioned. In areas such as writing clear documentation, doing code reviews and unit testing, we can see that LLM (Large Language Model) can bring a lot of improvement. In some other fields, such as auxiliary interface design, auxiliary architecture design, and architecture governance, we see that people have more and more attempts.
Regardless of whether it is architecture design or interface design, it finally depends on the expression of requirements. The standardized design from requirements to models is exactly the way that classical software engineering (such as UML, Unified Modeling Language ) is particularly good at. In the past, we were exhausted writing detailed requirements, and LLM just helped us to a certain extent.
Does this mean a revival of classic software engineering? And do we need a new generation of software engineering?
Introduction: Looking back on the past
Have you tried to provide a detailed requirement or API interface to let ChatGPT draw PlantUML (an open source software that supports UML implementation) diagram?
Introduction 1: The programming era of craftsmen
When I was just learning programming, the second-hand computer used in school was still DOS with white letters on a blue background. Because the computer was very slow at that time, and the brain was not bright, the teacher would ask you to write and draw on paper and pen for a small function, and then input it into the computer. When I was in college, I was engaged in embedded development. To write an OS with only task scheduling, I had to do a wave of theoretical verification before throwing it on the chip and running. Today, designing a web system and framework starts with PoC verification, and writing a function starts with testing.
In my impression, among implementation methods at different levels and in different fields, I seldom achieve complete consistency between design and implementation, so I have always been unfavorable to classic software engineering methods.
In web application development, we have been pursuing a quick verification method, such as:
Create the application template, and its hello, world.
A successful run of the first unit test.
Successful verification of the first CI/CD.
Local desktop inspection of the first feature.
End-to-end delivery of the first function.
The MVP of the product goes live.
In the pursuit of stable infrastructure, such as the embedded field running on resource-constrained devices, because users also expect Internet-like delivery methods, they are also subject to a series of shocks. Like the car software we thought in the past, we have also developed an OTA upgrade method to appear and prevent you from breaking down due to operating system upgrades on the road.
Introduction 2: Software Engineering Methods: Thinking about LLM + Classical Software Engineering Design
When working, I started to get used to the standard agile software development process, which means that in the development project, I need (generated by ChatGPT):
Identify needs: Work with stakeholders to identify and document needs. This is usually achieved through means such as user stories, use cases and requirements specifications.
Planning and Design: Determine the project's scope, timing, and resources, and create a project plan. At this stage, the team also needs to design the architecture and technical solutions of the system.
Perform iterative development: Agile methods use iterative development, with each iteration typically lasting 1-4 weeks. Teams create working software based on requirements and designs, and demonstrate and review at the end of each iteration.
Continuous testing and integration: During the iteration and development process, the team needs to continuously test the software and perform continuous integration. This helps ensure software quality and stability, and reduces defects and technical debt.
Delivery and Deployment: At the end of each iteration, the team delivers working software, which is deployed. This way, stakeholders can learn about the functionality and features of the software early on and provide feedback.
Evaluation and Continuous Improvement: The team needs to regularly evaluate the process and results for continuous improvement. This includes reviewing iterations, assessing team performance, and taking action to resolve issues.
In agile software development, we would emphasize: working software over detailed documentation . And now we have a new team member: LLM, this new team member needs documentation , detailed documentation , and more detailed documentation . (PS: Of course, there are various ways to document, for example, code information is also part of the document.)
Primer 3: LLM as Copilot
Subsequently, we attempted to introduce LLM in the field of software development afterward. After a series of internal brainstorming, we think it will play different roles at different stages of maturity:
Phase 1: LLM as Copilot. The professional division of labor in software engineering is not changed, but each professional technology is enhanced, and the AI-based R&D tool platform assists engineers in completing tasks and affects individual work.
Phase 2: LLM as Co-Integrator. Synergy across R&D responsibilities and roles, the AI-based R&D tool platform solves the problem of communication between different roles to improve efficiency and affect role interaction.
Phase 3: LLM as Co-Facilitator. Affects the division of roles in the software R&D process, and the AI-based R&D tool platform assists in decision-making. Aids in planning, forecasting, and coordinating efforts to influence organizational decision-making.
Back to classic software engineering development, why are we unwilling to draw UML diagrams? First, it is only suitable for reference; second, the learning cost is not low. Simply put, the price/performance ratio is too low.
And it happens that LLM can solve these two problems to a certain extent. LLM can be used as a Copilot to solve "I'm too lazy to do" and "I do it repeatedly". For example, you can let it generate UML, although it is not so reliable. But it can be used by changing it.
And with the basis of "you can use it after changing it", the rest of the things become very simple.
Building Contexts Dynamically: The Heart of LLM + Software Engineering
After exploring a series of practices and application development of LLM + software engineering, we set out to build ArchGuard Co-mate to guide software architecture design and software architecture governance.
The bottleneck behind it is: how to dynamically build the context needed for software development?
Tacit knowledge: why ChatGPT cannot generate satisfactory API?
Most people may have tried to let ChatGPT generate a RESTful API, and modified one prompt after another, such as:
You are a software architect, please generate all APIs for the blog entity. Use the form to return, format: method, path, request parameters, return parameters, status code.
Then, ChatGPT starts to generate the API, and at this time you find that the API it generates may not conform to the internal API specification. So, we try to talk to it to generate a more accurate API, however, it may not be as fast as you modify it directly. Or, we can provide it with a refined API specification, such as we designed in Co-mate:
您是一个软件架构师，请生成 博客 entity 的所有 API。要求：
1. 使用表格返回，格式：方法、路径、请求参数、返回参数、状态码。
2. 你需要参考 API 规范生成。
API 规范如下：
###
rest_api {
uri_construction {
pattern("/api\\/[a-zA-Z0-9]+\\/v[0-9]+\\/[a-zA-Z0-9\\/\\-]+")
example("/api/petstore/v1/pets/dogs")
}
http_action("GET", "POST", "PUT", "DELETE")
status_code(200, 201, 202, 204, 400, 401, 403, 404, 500, 502, 503, 504)
security(
"""
Token Based Authentication (Recommended) Ideally, microservices should be stateless so the service instances can be scaled out easily and the client requests can be routed to multiple independent service providers. A token based authentication mechanism should be used instead of session based authentication
""".trimIndent()
)
}
###Then, as we provide more and more context, ChatGPT can finally work like you. Although, at this time, the time to generate the API combined with ChatGPT has far exceeded the time to design the API by hand - because we have always needed the time required to provide context, we have been making knowledge explicit.
LLMs are like "new grads": what context do grads need?
That's when you realize: "Oh, LLM is like a new grad on our team". You need to teach him a series of knowledge:
Functionality of the system: What does the core system do? What are its main functional modules? How do these modules provide value to users?
Advantages of the system: Compared with other similar systems, what are the unique advantages and characteristics of the core system? How does it perform? Does it provide a better user experience?
System application scenarios: What application scenarios does the core system apply to? What industries or fields can it help?
System technical architecture: What is the technical architecture of the core system? What techniques and tools does it use? How scalable and flexible is it?
User groups of the system: Who are the main user groups of the core system? What are their needs and behavior patterns?
By the way, I gave the graduate a bunch of documents and asked him to spend two days reading them. Then, you start to let him realize a certain function so that he can practice. In the end, you find that 9 out of 10 graduates cannot write code that meets the requirements. There is another one that can be written because he has practiced in this team.
Think about it, when we realize the function of an API, we need:
When designing an API, you need to follow the specifications of the API specification design.
When writing tests, you need to follow best testing practices.
When implementing the API code, it is necessary to follow the specification of the code specification.
When writing commit messages, you need to follow the conditional API specification.
……
That's right, every small step we need a precise spec to write code that meets the requirements. In most teams, this is tacit knowledge, or maintained by outdated documentation. (PS: So, in fact, even team members who have worked for several years may not necessarily be able to write code that conforms to the specification)
Therefore, we advocate the use of pair programming to share knowledge, so that newcomers to the team can get started faster.
LLM-Driven Standardization of Software Engineering Practices
Now, after going full circle, let's get back to the topic of the article. We regard LLM as a newcomer to a team. It needs to know the context of the team in order to assist us to complete the tool more efficiently.
When building the architecture governance platform ArchGuard, we implemented it around the idea of a three-state architecture (design state, development state, and running state). For software, it is quite similar. We will design the architecture based on the initial requirements , that is, the design state architecture. When it is implemented, it is based on a refined architecture that responds to market changes, that is, a development state architecture. If we want to have fast market responsiveness, we tend to balance the time spent on the two parts, so often the two will not be exactly the same.
Under the Implementation Architecture: Standardizing the Implementation Process
I believe that most people have never used GitHub Copilot to write code, but most of them have used ChatGPT to write code. I think everyone will come to a conclusion: when we give a sufficiently precise context, AI can produce very accurate code, although there is still a certain degree of randomness. (PS: Of course, the second conclusion is still the one mentioned earlier: if I had given enough precise context, then I would have finished writing.)
Therefore, in order to make AI write code more automatically, we need to explore the implementation of process standardization , such as:
Obtain requirements from the requirements management system and conduct requirements analysis.
Combining source code and requirements system, select the most suitable entry for change (such as Controller in Java)
Give the requirements and Controller to AI analysis to realize the code of the code.
Automatically complete other parts of the code step by step according to the Controller (under implementation...)
……
Under the current software development process, we can only let LLM simulate the current process work. This is the original intention of creating AutoDev, use ChatGPT to decompose requirements, write the process of analyzing requirements into the tool, so that ChatGPT can analyze individual requirements, and automatically write code based on this.
At this time, we will find another problem: ChatGPT lacks a global view. It only got a single requirement, acting like a newcomer. It also requires more design and specification-related information.
Under the Design Architecture: Specification DSLization
As a practitioner of AI + software engineering, I don't believe that documentation can help LLM solve this problem. Because documentation is always behind, lacks human maintenance, and cannot be automated.
Therefore, what we are exploring in Co-mate is to standardize DSL, that is, to carry out secondary encapsulation on the basis of the original ArchGuard standard code. That is, you can let LLM generate the design according to the DSL, and you can also check whether the generated design meets the specification through the DSL.
For example, in Co-mate's Foundation Spec, we can check the naming in the following way:
naming {
class_level {
style("CamelCase")
pattern(".*") { name shouldNotBe contains("$") }
}
function_level {
style("CamelCase")
pattern(".*") { name shouldNotBe contains("$") }
}
}In the generated code, it can also be provided as the context of LLM. Since it's a DSL, not a document, it can be dynamically pulled out as part of the context.
A new DSL for classic software engineering
In the past, our industry has accumulated a series of DSLs, such as a large number of ADL (Architecture Design Language), UML (Unified Modeling Language), BDD languages (such as Cucumber), and so on.
Cucumber is behind Gherkin is an interesting DSL, especially suitable for combining with LLM. It is also in line with the article "[Language interface: Exploring a new generation of API design with large model-first architecture] ( https://www.phodal.com/blog/language-api-llm-first-api/)" mentioned A new generation of streaming (Streaming) DSL format. as follows:
Feature: OKR协作与管理
用户可以创建和管理OKR，跟踪目标和关键结果的进展。
Scenario: 创建OKR
Given 用户已登录到OKR协作与管理系统
When 用户进入系统主界面
And 用户选择创建OKR
And 用户填写目标和关键结果的详细信息
And 用户设置时间周期和权重
And 用户点击保存按钮
Then 系统应成功创建并保存OKRTherefore, we can format the requirements in the above way.
But we have encountered another problem, how to express the more macro needs?
So, I found UML from the classic engineering method. I still "believe" that many people have tried to let LLM generate PlantUML to assist in architectural design. Although there is a certain probability that the generated UML will not work or be accurate, I think it is quite fun.
Because, I have not been good at writing the standard UML, so I do not like it. And because most back-end developers have written Gradle configuration, I think it is similar to Kotlin DSL, which is easier to understand and modify:
caseflow("MovieTicketBooking", defaultRole = "User") {
// activity's should consider all user activities
activity("AccountManage") {
// task part should include all user tasks under the activity
task("UserRegistration") {
// you should list key steps in the story
story = listOf("Register with email", "Register with phone")
}
task("UserLogin") {
story += "Login to the website"
}
}
activity("MovieSelection") {}
// ...
activity("PaymentCancel") {
task("ConfirmCancel") {
role = "Admin" // if some task is role-specific, you can specify it here
//...
}
}
}I've been trying to balance use cases with user stories and try to bring them together to provide a dynamic context for future code generation.
so what?
In order to better apply LLM to the software development process, then we need:
Build standardization of the software development process to instrumentalize it.
Document specification DSL, coding, and dynamic provision to reduce the cost of AI thinking.
Closing the classic software engineering approach, delivered in a new bottle.
And this will take a long time.