In that article " LLM as Co-integrator: Reshape interaction between teams and continuously improve information alignment ", we said that in order to better use AIGC to improve performance, our second stage should be: let LLM do some collaborative work , such as: building multi-scenario knowledge questions and answers, reducing knowledge retrieval costs, designing team APIs, and creating intelligent assistants.
Therefore, we set out to build CoUnit to explore LLM as a team interface to get closer to the collaboration between teams. We referred to the design of the Bleep semantic search engine and used Rust, Sentence Transformer, and Onnx to build localized , offline , low-cost vectorization and semantic search capabilities. GitHub: https://github.com/unit-mesh/ co-unit .
Watch the video demo first:
We have integrated CoUnit in AutoDev, just update your AutoDev plug-in to version 1.2.1.
Why CoUnit？
In the past few months, a large number of enterprises and teams have been building their own intelligent customer service, Q&A AI and so on. Back to software development, we may also need a similar question-answering AI, at least it will be faster and more friendly than FAQ. But how to make such an AI question answering application more valuable?
1. Thinking about value from the perspective of team topology
We should think about:
Which roles spend the most time working on integration?
What knowledge (documentation, API) can be partially viewed?
Which teams contain the bulk of the knowledge the organization needs?
Going back to our previous exploration of AI4SDCL (AI applied to software development), for teams with a large amount of integration work, only the ability to build LLM as Integrator can help this team release productivity. Therefore, combined with the idea of team topology, we will find that enabling teams and platform teams are the most in need of this type of tools.
Empowerment team : composed of experts in specific technical fields (such as DevOps, continuous delivery, automated testing, etc.) or product fields, empowering product-oriented teams to provide advice and support on tools, practices, frameworks, technology stacks, etc. For example, "technical consulting team" is one of them. In China, there are Huawei's software coaches, Tencent's agile coaches, etc.
Platform team : Deliver a highly autonomous working model to product-oriented teams. They provide self-service APIs, tools, knowledge, services and support to development teams, typically various infrastructure platforms, such as cloud native-related technology platforms with infrastructure code.
Focusing on these two types of teams, building cross-team knowledge question and answer AI can better assist these teams.
2. Invisible Team API
In real software development, we have always faced dependency problems between teams. In addition to documentation, wikis, etc., there are also go-live schedule and priority issues, which will slow down the delivery process to a certain extent. In particular, in order to solve the dependency problem between teams, developers need to agree on a time for pull-through meetings, which will also slow down the team's delivery rhythm. In addition to this, we can provide the clearest possible access to information and teams to minimize the cognitive load on others.
For this reason, in the book "Team Topology", it is advocated to build a Team API to solve the above problems, that is, to provide documents, wikis, and non-sensitive code, as well as roadmaps, communication preferences, and so on. These things are very suitable to be solved in conjunction with LLM, that is, LLM is used to analyze the needs of other teams and answer users based on information such as knowledge bases and open APIs.
By the way, different teams may prefer to work on different time scales (such as bi-weekly iterations, when to release and go online, etc.), and these can be considered to be solved in the next AI4SDLC stage.
What is CoUnit?
simply put:
CoUnit, a virtual team interface (Team API) based on LLM, combines LLM with intelligent team docking and collaboration through vectorized documents, knowledge bases, SDKs and APIs.
Since HuggingFace uses the Rust language to build a series of AI infrastructure, and the open source Bleep code search engine uses the Rust language, we also use Rust to build a local semantic code search service - locally with the help of Sentence Transformers Vectorize and search without an Internet connection.
Based on CoUnit, you can:
Semantic code search
Semantic OpenAPI search
Semantic HTTP API search
Semantic document search
……
Of course, more types can be added in the future. In terms of technical difficulty, these are not the main problems, and it is not difficult at all to realize the function. The difficulty is, how to improve the accuracy of search? , such as: How to combine Chinese and domain specific nouns for search?
How does CoUnit work?
Before using CoUnit, we need to prepare a series of original data, such as code API and so on. Therefore, we implemented an ArchGuard server API in CoUnit, which can directly accept data uploaded by ArchGuard and store it in the vector database. Later, we can use it in CoUnit clients, such as AutoDev.
The overall working process is as follows:
The existing process is relatively complex
Considering a series of code-related elements of structured data, such as documents, architectures, API definitions, etc., we do not need to use OpenAI for vectorization. We can use Sentence Transformers to achieve semantic text similarity, semantic search and other functions .
CoUnit working process example
The interaction process used is as follows:
Submit user tasks to CoUnit through API, such as: Merchant order query .
The API will return the corresponding prompt, and the caller will call the corresponding LLM to generate the analysis result.
The following is an example of the analysis results:
{
"domain": "merchant",
"query": "merchant: query merchant order",
"natureLangQuery": "商户申请单查询",
"hypotheticalDocument": "GET /api/merchant/order/query?order_id=(order_id)"
}4. Based on the above analysis results, call CoUnit's analysis engine respectively, namely: English language query, native natural language, and hypothetical document query.
5. The application analysis determines the next operation.
summary
For LLM collaboration cross-platform collaboration, CoUnit provides more of a new possibility. If you are also interested in developing AI applications with Rust, welcome to join us: https://github.com/unit-mesh/ co-unit.
