ChatGLM large model application construction & Prompt project

• Refer to "[GLM Course] ChatGLM Large Model Application Construction & Prompt Project"
• Zhipu Qingyan , Zhipu AI development platform , ChatGLM blog

1. Introduction to large models

1.1 Basic knowledge of large models

1. The large model is also a probabilistic model.
   The large model is based on previously learned knowledge and generates subsequent content with the highest probability based on the input questions. Given a reasonable question, large models generate reasonable answers.
2. Hallucination problem . Large models do not judge whether the input is reasonable, but only generate content based on probability, which sometimes leads to unreasonable answers.
   For example, I asked the big model to help me write a story about Lin Daiyu uprooting a weeping willow. Everyone knows that this is an unreasonable plot and is simply impossible. But for a large model, it will not judge whether it is reasonable or not, but will only generate content based on the maximum probability. This is what we call illusion, or serious nonsense.

3. Base model and command model

  The base model is similar to the learning stage of knowledge, which is used to understand various kinds of knowledge. The instruction model is similar to a professional training phase (question review), which is aligned with human cognition and enables the model to perform well in a specific field.

  This can be compared to the process of learning English. Reading textbooks is the foundation, but before taking the exam, you need to brush up on questions and understand how to answer questions in specific fields. The task of instruction engineering is to provide this training and guidance for the model.

1.2 Large model classification

It's important to understand what a large model can accomplish. Large model work can be divided into three main categories:

1. Text generation: information from less to more:
   This involves having large models expand the information provided into more. For example, WPS's AI cloud document can generate weekly work reports based on the provided daily work information fragments.

2. Text extraction: from more to less information:
   This category involves text extraction, which is extracting useful content from large amounts of information. For example, it can extract key information from a resume, or understand instructions and translate the user's words into specific actions. In a sales scenario, it can analyze multiple rounds of conversations to understand user preferences or needs for products.

3. Transformation of text information:
   The most typical example of transformation of text information is translation, and large models can provide excellent translation effects. Another example is code generation, such as domestic CodeGeeX2 and foreign Copilot can generate code. In addition, self-service analysis (NL2SQL) in the field of data analysis can convert natural language into SQL queries for faster data analysis.

  These job categories demonstrate the versatility of large models that can support automated and intelligent tasks in a variety of domains and scenarios, improving efficiency and convenience.

2. How to build large model applications

2.1 To B. To CDifferences in scene applications

When applying large language models, different scenarios require different strategies and principles. The following is a summary of application principles for different scenarios:

To C scenario (for consumers):

1. All-round assistant: The large model acts as an all-round assistant in To C scenarios. Users can obtain new content by constantly making requests.

2. Continuously improving instructions: In this scenario, the refining of instructions is an iterative process. Users can start with simple instructions and then gradually refine them based on feedback from the model to obtain more accurate output.

3. Flexibility: In To C scenarios, the output of the model needs to adapt to the different needs and ways of thinking of different users. Therefore, flexibility and personalization are key.

4. Adjustable content: Users have a certain degree of freedom in the final output content, because manual intervention and modification are usually required to ensure that specific needs are met.

To B scenario (for enterprises):

1. Scenario-based design: In To B scenarios, we usually design large-model applications for specific scenarios. This requires product managers and engineering teams to think together about how to guide users to provide effective input.

2. Engineering instructions: In this scenario, instructions usually need to have engineering characteristics to adapt to the determined scenario. This includes stable input and evaluation sets to ensure model stability and usability.

3. System interaction: To B scenarios usually require interaction with other systems, so the output content must be in a stable format to enable interaction with downstream systems or business goals.

  In the To C scenario, the large model is more like a flexible assistant, while in the To B scenario, it serves more as a specialized tool that serves specific scenarios and needs. Users enter key content according to the prompt format (APP) given by the product manager, and guide users to complete effective input. With large models, product managers can focus more on product positioning and design, and guide users to make effective input. Based on these instructions, the instruction engineer develops an APP that allows the large model to execute instructions more stably (based on a stable input evaluation set, to make an engineered large model solution and obtain business-usable results).

2.2 Application principles of large models in To B scenarios

1. Embed into business processes

  In order to effectively apply a large model, the first task is to embed it into the enterprise's business processes. For example, suppose a company needs to conduct a large number of recruitment interviews. When evaluating a candidate, you typically look at their resume and reviews from past interviewers. In this case, a button can be added to the recruitment system to allow the large model to summarize the content of the previous interviewer with one click, and then output the candidate's strengths and weaknesses in table form. Large models excel at text understanding and generation, so this is a very useful feature to embed into business processes.

2. Unleash the potential of large models

   Large models have strong potential for text understanding, generation and some basic reasoning. Therefore, these potentials of large models should be fully utilized when applying them within the enterprise. Product managers and business experts need to understand the capabilities of large models, and then find areas where large models can be applied from the company's existing business systems. This may include automatic text summarization, information extraction, natural language reasoning, etc.

3. Re-break down business goals

  A large model is not a magical wishing well, you can't just tell it a goal and expect it to do everything. Instead, we need to redesign a set of processes to solve specific problems. Here are some examples:

1. Problem background: The boss of the tourism industry needs to build a store robot that can perform itinerary planning.

2. Decomposition process: In order to meet business needs, we decompose trip planning into four steps to clarify the tasks and goals of the model.

   a. Step 1 - Role playing and information collection: First, the model plays a role, interacts with the user in natural language, and collects information about travel needs, such as destinations, travel preferences, number of travelers, etc. This is a natural in-store robot interaction that provides a good user experience.

   b. Step 2 - Information extraction: Next, the model extracts key information from previous conversations, such as budget, number of people traveling, specific family members, and destination list. This information is used in subsequent steps for trip planning.

   c. Step three - itinerary planning: The model can use the extracted information to make requests to external services (such as Ctrip) or internal itinerary planning systems to generate itinerary planning suggestions. Large models can also plan their own itinerary through fine-tuning training.

   d. Step 4 - Generate a road book: Finally, the robot generates a highly readable and elegant road book based on the user's family situation and attraction information. The model receives instructions and generates text that meets the user's needs based on daily travel plans, attraction introductions, and family members. In this way, we provide users with valuable itinerary suggestions.

  This example highlights how business goals can be refactored into a series of tasks to clarify tasks and guidance for large models. This approach helps ensure that the model effectively meets specific business needs.

2.3 Large model application skills

2.3.1 Large model application skills

When designing and managing large model applications, there are some key tips that can help improve performance and user experience:

1. Manage user input:

  For example, in the case where a user wants a large model to help write a song, the user usually provides information such as theme, style, etc. In order to effectively interact with user input, multiple templates around different themes can be prepared in advance, and then the user input is combined with the adapted template into a structured instruction. This design allows product managers to guide users to provide effective input through interaction, and also helps command engineers better collect corpus, develop commands, and fine-tune commands.

3. Make good use of JSON to interact with the system:

  In the example below, a large model needs to control devices in the car based on user input. To achieve this, product managers need to design universal commands that can control multiple devices such as air conditioners, seats, and windows. These instructions need to include specific operating actions and preconditions, and design a stable output structure so that the large model can effectively convert human language into instructions that the device can operate .

3. Refine the scene and split the instructions:

  Break down the tasks of the large model into specific scenarios and problem types so that the large model can focus more on solving a certain type of problem. For example, users may need large models to answer subject questions, such as mathematics or Chinese. In order to improve the effect, these questions can be classified first, and then instructions can be designed for each classification to indicate the professionalism of the large model in a specific field (for example, prompt the model, you are now a professional Chinese teacher...). In addition, some CoT examples can be prepared for each domain to help the large model answer questions better and further improve the quality of the output.

4. Make good use of external systems:

   Taking the medical consultation scenario as an example, we hope that the large model can make a preliminary disease diagnosis. In Internet medical scenarios, the time and cost of communicating with doctors remotely are relatively high. If the large model can collect effective patient information first, it can save a lot of time and cost, but this may require a lot of professional medical knowledge, such as what to do with respiratory diseases. How to diagnose and diagnose digestive tract diseases. If the large model is divergent and asked, the effect may not be good.

   In order to solve this problem, external control systems can be used to combine the consultation process with the medical knowledge graph to provide information in a stable and effective way and ensure that the problem is controllable.

  In addition, the information from multiple rounds of consultation sessions needs to be refined and collected. This can also be done through an external system (another instruction) to continuously extract the dialogue information, and then compare it with the disease diagnosis knowledge map to ensure that questions are not missed or repeated.

5. Handle multitasking and multi-turn conversations:

  In the English sparring scenario, the large model can focus on English conversation and needs to handle multiple tasks at the same time, such as maintaining grammar check, colloquial requirements and language style in the conversation, and also needs to check the conversation content (whether there are violations, whether Complete dialogue goals such as ordering food).

  To handle this multitasking situation, two large model instructions are available, one for sparring the conversation and another for checking the conversation content. In this way, the two large models can work together to ensure that the goals of English sparring are achieved while maintaining conversation quality and compliance.

6. Long text processing

  When processing long text summaries, users usually want to refine long texts of tens of thousands of words into summaries of 200 to 300 words. A common strategy is to slice or segment long text and then generate a summary of each segment via the first instruction.

  Subsequently, the summaries of these segments can be merged into an overall summary. If the user's summary needs have been met, the processing can be ended. If the summary goal is not achieved, you can go back to the original text, further segment the text, and repeat the above process. This loop iteration method is similar to the idea of ​​"map reduce" and can effectively handle the problem of long text summary, ensuring that the final generated summary meets the user's expectations.

  Through the techniques discussed above, large models can be successfully applied to existing business systems or solve new problems in different scenarios.

2.3.2 Application methods of large models

For the application of large models, the following methods are very important:

1. Develop a flowchart : At the beginning of a project, it is critical to create a clear flowchart. Flowcharts help simplify and understand the role and tasks of large models in business processes, allowing business parties to have a more accurate understanding of their functions and capabilities.

2. Define evaluation standards : The application of large models often needs to be evaluated based on accuracy. Evaluation standards are also an important means to achieve controllable rates. In the early stages of the project, it is difficult for users to provide complete evaluation criteria, so they need to be gradually formed through an iterative process. Metrics help ensure your launch meets expectations.

3. Use evaluation data : Evaluation data is a key factor in verifying model performance and can also be used to supplement and improve requirements. Users may not be able to accurately describe their expected results, but by collecting real evaluation data, they can better understand user needs and quickly implement the application.

2.3.3 Case: Auto Insurance Q&A System

The following is a demand case that the author has experienced, how to gradually improve the functions of a question and answer system to meet user needs:

1. Initial requirements: The user initially requested a question-and-answer system that could answer questions through a set of instructions and a knowledge base.

2. Adding a crime reporting function: Subsequently, users provided evaluation data, indicating that they hoped that the system would not only answer questions, but also help them complete the action of reporting a crime. This requires the system to have an intent recognition function and be able to guide users to provide reporting information, such as car insurance reports.

3. Meet more needs: In further conversations, users put forward more requirements, such as hoping that the system can infer the specific time and location of the report. The user may mention "just now" or other ambiguous time information in the conversation, and the system needs to provide the exact answer through inference from a large model.

4. Improvement of instructions: In order to meet the new needs of more users, the system needs to further improve instructions so that large models can handle more information and inference tasks.

  Overall, this case highlights how to gradually expand functions according to user needs when building a question and answer system, including intent recognition, information guidance, and inference capabilities, so that the system can meet changing user expectations. These are tasks that large models can handle.

3. Prompt engineering

3.1 Basic knowledge of command engineering

1. what is a directive

  A directive is a request for retrieving information from a model, which describes the user's needs but usually has no specific syntax or fixed format. Instructions can cover a wide range of areas, including knowledge retrieval, task execution, natural language generation, etc. For example, a user may want to query the population of each municipality in China.

There are some problems during the instruction writing process:

1. Limitations of personal experience: Instruction writing often relies on personal experience. In this case, it may be difficult for others to directly use the same instructions because they may be incompatible with other problems or models, resulting in unstable effects.

2. Lack of flexibility: Instructions shared by others are difficult to modify. In order to increase the reusability of instructions, in the instruction project, some contents can be turned into configurable variables, so that others can more easily modify and reuse the instructions.

3. Sensitivity and Comments: Small changes in wording or formatting within instructions may affect the content of the final output, especially in a production environment where there may be strong dependencies between queries. Therefore, during the production process, evaluation and continuous improvement of instructions are usually required.

4. Differences between models: The same instruction may produce different effects between different models. However, with further optimization, the instructions can be made to produce good results on different models.

2. What is command engineering

  Instruction engineering is a workflow and method related to dialogue systems in the NLP field. It aims to create and improve instructions or requests for dialogue with large language models, and continuously improve the performance and adaptability of the model to ensure that it can produce expected results in practical applications. the result of. Instruction engineering is an iterative process that typically includes the following steps:

1. Create instructions: Create a preliminary instruction based on task requirements.

2. Implementation directive: Use this directive to generate the output of the model.

3. Evaluation and analysis: Use the evaluation set to test the effect of instructions and analyze the performance of the model in different situations, including checking for bad cases.

4. Optimization instructions: Modify and optimize instructions based on the results of evaluation and analysis to improve model performance.

5. Continuous evaluation: As the model or task changes, continuously evaluate the effectiveness of the instructions and modify them as needed.

6. Stable evaluation: Use a stable evaluation set to ensure that instructions perform consistently in different scenarios.

3. The importance of command engineering

  Over the past six months or so, our team has completed the development and evaluation of a large number of instructions, and in the process came to some key points:

1. The potential of instructions : On excellent large-scale models, through careful design and development of instructions, high-level effects can be achieved, even reaching 90 points, 95 points or even higher levels. This demonstrates the potential of command engineering to help launch products while also helping individuals solve problems in their lives and work.

2. Flexibility and adaptability : Command engineering needs to be flexible and able to be developed and adjusted according to different demand scenarios. Different tasks may require different types of instructions, so teams need to adapt to changing needs.

  In summary, instruction engineering plays a key role in applying large models to real-world scenarios, helping the model's performance and effectiveness reach optimal levels. Both teams and individuals can benefit from the methods and practices of command engineering to address diverse problems and needs.

3.2 Principles for writing instructions

There are some important knowledge and principles in instruction writing:

1. Be clear and unambiguous:
   Instructions should express what you want the model to do as clearly and unambiguously as possible. Make sure that the core requirements of the task are clearly communicated to the model to avoid ambiguous or undesirable output. Use specific words and requirements to ensure the model understands your intent.

2. Give the model time to think:
   The model needs time to think and process your instructions. Sometimes, explicitly including the thinking process in the instructions can help the model better understand your expectations, indicate how the model should think and handle the task, and ultimately improve the quality of the model output.

  For example, in the instructions for writing a cover letter, you can clearly require that the beginning part indicates why you are interested in the position, the middle part describes why you are suitable for the position, and the ending expresses your expectations for the future and expresses gratitude. This approach helps guide the model to produce content as you expect.

3.3 Instruction writing steps

When writing instructions, they usually include the following key parts:

1. Context (optional): The context part of the instruction should include roles and tasks, which helps the model better understand the task. Some relevant background knowledge should also be provided so that the model can refer to it when performing tasks, including domain-specific information or knowledge bases.

2. The instruction itself (required): In the instruction, clearly specify the steps or logic to perform the task. These steps can be listed in order to tell the model in what order it should complete the task. You can also provide some sample data or examples to help the model better understand the nature of the task.

3. Input content (required): The input content can be in the form of a conversation, article or question.

4. Output format (optional): At the end of the instruction, an output guide should be included to clarify the format of the information or results that the model should output.

  For example, if a large model is needed to extract key information in the sales process from a conversation, you can include the above four parts in the instruction to make the instruction more complete and clear, and use variables to represent the input conversation content and output information point format.

# 结构
你是对话信息提取器，你要按顺序完成任务，从客户对话中找到的客户提出的关注点，按照输出格式输出4个客户认为最重要的关注点词语

你需要按顺序完成任务：
1. 找到在客户说的话中出现的关注点并精简为词语，如果没有找到任何客户提出的关注点，请输出"未涉及"
2. 按照客户的关注度从高到低排序，按输出格式输出4个客户最关注的词语
3. 检查你的输出内容，如果你输出的内容是一句话，请重新输出总结为词，按照输出格式输出 json 
对话：
"""
( input )
"""

输出格式：
{
    
    "客户关注点:["."]}

3.4 Instruction writing skills

  When writing instructions, understanding the capabilities of large language models and using some instruction writing techniques can help you better guide large models to perform tasks. Here are some key points:

1. Zero Shot Capabilities: Large language models now have strong fundamentals, so in many natural language processing tasks you don’t need to retrain the model or provide a lot of example data, just make explicit requests. For example, you can ask the model to generate a passionate sports drink slogan without providing a specific example. The model will generate content based on your description.

2. Few-Shot capability: When requiring the model to perform more specific or complex tasks, you can take advantage of the Few-Shot capability to provide the model with a small number of examples to guide its execution. For example, if you want to generate a catchy slogan, you can provide some examples with similar characteristics, such as word repetition or rhyme, and the model can use these examples to generate more relevant content.
   
3. Thought chain with examples
   A thought chain refers to the thinking and reasoning process of a large language model that is guided step by step so that it can effectively answer a question or perform a task. When writing a thought chain, you should:

   1. Provide clear steps: When writing a thought chain, make sure to provide clear steps for the model to complete the task step by step. Each step should contain clear guidance so that the model thinks and acts how you expect it to.

   2. Example data: In the thought chain, you can include example data or scenarios to help the model better understand the task. These examples can be used to guide the model through inference and computation.

   3. Feedback and iteration: Review the results generated by the model and provide feedback, then continuously improve the chain of thinking based on the feedback. Iteration is the key to continuously improving task performance.

4. Thinking chain without examples
In the following example, we ask the model to think step by step, and indicate that the key point of thinking is the change in the number of fruits.

5. Top-k and Temperature
   In the process of writing instructions, two important hyperparameters need to be adjusted: Top-k and Temperature.

   1. Top-p: The Top-p parameter is used to control the diversity of content generated by the model. When setting a lower Top-k value, the model is more inclined to select words with the highest probability, making the generated content more stable and certain. Setting a higher Top-k value will increase the diversity of generated content and make the model more creative, but may result in some content being less accurate.
      

   2. Temperature: The temperature parameter affects the difference in probability between words through mathematical formulas, thereby affecting the diversity of generated content. Higher temperature values ​​cause the model to generate more diverse content that may be more creative, but also more random. A lower temperature value will make the model more inclined to select words with the highest probability, and the generated content will be more certain and stable.
      
        In some cases, such as information extraction or question-answering tasks, lower temperatures and lower Top-p values ​​may be more appropriate to ensure that the generated answers are more accurate and reliable. In scenarios such as creative writing, higher temperatures and higher Top-p values ​​may be more suitable to generate more diverse and creative content. In practical applications, by repeatedly adjusting these hyperparameters, the most suitable configuration for specific tasks and scenarios can be found to obtain satisfactory generation results.

3.5 Practical skills

1. Use delimiters: Using delimiters can better handle the interaction between input content and instructions, improve the accuracy and reliability of generated results, and reduce additional interference and error information. For example, when the input content involves multiple aspects such as reference copywriting and revised copywriting, using appropriate separators can effectively isolate the interference between the large model instructions and the input content.

original order
Optimization instructions: Input content and instructions are separated by quotation marks

2. Write instructions into items : Try to divide instructions into clear items. This has several advantages:
   • First, clear and unambiguous instructions help eliminate ambiguity, ensuring there is no confusion both literally and structurally.
   • Second, this structure helps large models better understand each individual task. By placing tasks in order, large models can be easier to think about and execute.
   • Contributes to better iterative optimization instructions. Write instructions in separate items. If the generated content is poor, you can modify the instructions accordingly.

3. Knowledge injection: In some scenarios, you can help large models generate more accurate results by injecting knowledge.

   • Text injection: For example, if you ask a large model to provide travel suggestions, you can inject some information about weather conditions.

   

   • Table injection: If you use tabular data, make sure to append table headers or explanation information in front of each data to transform the presentation of tabular data into expressions closer to natural language to help large models better understand the meaning of the data. Improve the accuracy of their understanding and answer generation.

     1. Goal: In many scenarios, it is desirable to use large models to perform simple analysis on tabular data, such as checking for growth or downward trends in the data.

     2. Data source: Data usually comes directly from Excel or PDF files and needs to be parsed and prepared.

     3. Increased difficulty: Directly using the original tiled tabular data is difficult for large models to understand, so measures need to be taken to improve the model's understanding.

     4. Organize data: In order to better inject data, it is recommended to associate headers or explanatory information with each structured data and place it in front of each data. For example, the first row can express revenue in 2022, year-on-year data, revenue growth data, etc.

4. Stable formatting requirements: Try to use stable formatting requirements and require the output of large models to be presented in structured formats such as JSON or Markdown. This helps avoid large models from producing fictitious or divergent content and ensures that the output complies with specific standards and structure; it also facilitates interaction between output results and other business systems.

5. Follow the trend: When writing instructions, try to write them according to the capabilities and cognitive methods of the large model, reduce unconventional or unnecessary logic, and avoid making the model perform incomprehensible operations.

   • Clear instructions : Make sure the context and semantic relationships of instructions are clear and avoid contradictory or confusing expressions. For example, when extracting brand information, express brand content clearly and do not mix in requirements prohibiting export sales, as this may cause confusion.

   

   • Express in natural language and avoid too much logic : Try to use natural language understood by large models to write instructions instead of introducing code or numbering. If data conversion is required, it is best to process it in the subsequent business system to ensure that the instructions are concise and clear.
     For example, in the following scenario, men are required to output 1 and women are required to output 0. This hard conversion operation can be processed in the post-processing business system instead of introducing complex conversion actions during the instruction extraction process.
     
   • Complete logic: Ensure that the logic of the instruction is complete. When the model cannot find specific content in the text, provide clear logic in this case to help the model perform the task correctly.
     In the example below, no platform-related information was found, and the large model generated some fabricated content. This situation no longer occurs when it is specified that the relevant content cannot be found and is not output.
     

  Through these experiences and skills, instructions can be better written to improve the quality and accuracy of the results generated by the large model, while ensuring that the large model can understand and perform each task.

4. Case: Extracting metaphorical sentences, personification sentences and parallelism sentences in compositions

  In an actual instruction production and iteration process, we are faced with a teaching scenario. To assist teachers in correcting essays, they need to extract metaphorical sentences, personification sentences and parallel sentences from the essays. However, the large model is not very clear about these concepts and needs to Keep iterating to improve.

Business requirements:

• Extract metaphorical sentences, personification sentences and parallelism sentences.
• Provide clear analysis of rhetorical devices, including identification of ontology and metaphor.
• Achieve high-quality instructions that can be directly put online and reach high evaluation levels.

Implementation methods and improvement processes:

1. Initial attempts: The initial instructions required the model to directly extract metaphorical sentences, personification sentences and parallel sentences, but the effect was not good. The large model is not very clear about these concepts. For example, it is easy to regard yes-word sentences as metaphorical sentences, so the score is generally between 60 and 70 points.
   

2. More specific definition: Optimize instructions, clearly define metaphorical sentences, personification sentences and parallel sentences, and require the output of rhetorical techniques, etc. The effect improvement is not obvious.
   

   Clear definition, little effect improvement

3. Task splitting: Split the task, for example, focus on extracting metaphorical sentences first, and add the identity information of the Chinese teacher to the instructions. Because the task is more focused, the effect has been improved, but it is still far from the requirements for being online.
   

   Tasks are split and the effect is improved

4. Provide a thinking path to guide the comprehensive analysis of large models: while clarifying the definition of metaphorical sentences, tell the model what is the ontology and what is the metaphor, and guide the large model to think about the path. We require the large model not only to output answers, but also to present a complete analysis process, including whether there is a specific ontology and metaphor in the sentence, whether it is a metaphorical sentence pattern, and ultimately whether it is a metaphorical sentence.
   

   Clarify the concept of ontology and metaphor, guide the thinking and analysis process, and the effect is significantly improved

5. Optimize instruction expression: Adjust the vocabulary and expressions in the instruction text to try to improve the effect of large models, such as changing extraction to recognition and extraction, etc., but the effect is not obvious.
   

   Optimize the instruction content, but the effect is not significantly improved

6. Adding examples: To further improve the performance of large models, we provide more examples for large models. In the example, the large model is informed which sentences are metaphorical sentences, and the ontology and metaphor in each sentence are also clarified. Although examples and clear structure are provided, the improvement is weak
   

   Provide examples, the effect is weak

7. Provide negative examples: To further improve model performance, negative examples are introduced to tell the model which sentences are not metaphorical sentences and why they are not. By introducing negative examples and explaining their reasons, performance is significantly improved.
   

   Provide counterexamples, and the effect will be significantly improved

8. Provide a thinking chain for parallel sentences: For the processing of parallel sentences, the steps for analyzing parallel sentences are clarified, including observing whether there are three or more sentences, and the similarity between words and sentences, etc.
   

9. Judgment when there is no clarity
   

10. Adjust text order: When processing text, pay attention to the sensitivity of the results to the window length of the document and the position where the content appears. If you want to emphasize specific content, you can try adjusting the text order to affect how large models are processed. For example, when raising negative examples, sometimes information that should not be mentioned appears. In this case, placing the negative example in front of the example also slightly improves the effect.
    

Adjust the text order, the effect is slightly improved

  The above is the complete instruction iteration process, which finally achieves high quality of instructions, meets business needs, enables the model to accurately extract metaphorical sentences, personification sentences and parallelism sentences, and provides a detailed analysis process.

5. Q&A after class

5.1 In the to b scenario, what is the difference between using LoRA or profix turning to train the model and using the instruction template for optimization?

1. Cost: Whether using LoRA or fully fine-tuning, it requires a certain amount of data and computing resources.
   In terms of effectiveness, the two are complementary to some extent. Instructional templates are suitable for simple problems, while large models have the advantage of being able to solve a wider range of problems, but the cost may be higher.
2. Effect : The two methods complement each other at the effect level. For some simple problems, they can be solved directly through instruction templates, giving full play to the advantages of large models. For some scenarios, 100% accuracy is not required. You can quickly launch applications through instruction templates combined with productization capabilities, quickly iterate and solve business problems.

5.2 Can instructions only be written and iterated manually? Is it possible to automatically generate instructions and have them evaluated by the large model itself?

  Not yet. Current B-side scenarios are often brand new, and large models have not been seen before, and instructions still need to be written and iterated manually. But as large model capabilities improve, and as more C and B scenarios are understood, the writing of instructions may become more efficient and automated.

5.3 Can we use data from domain A to fine-tune the model in domain B?

  When fine-tuning the model, if there are large differences between field A and field B (such as law and medical care), fine-tuning generally does not have much value. If it is a similar problem in the same field but with different specific requirements, it will be helpful.

  For example, in short text and long text writing, the generalization ability of the model may help to train related instructions and content together, and large models can transfer some abilities to produce helpful effects in related scenarios.

5.4 How much work is required to migrate command projects between different large models? And in the future, as model capabilities increase, will prompt engineering become meaningless?

  The workload of migrating projects between different large models will vary depending on the situation, often requiring re-writing and re-evaluating instructions. Although instructions may need to be rewritten between different large models, some functions from the original product solution can be reused. As the capabilities of future models increase, perhaps there may be better solutions in a few years. However, at present and in the next few years, productivity can still be quickly improved through command engineering. For the implementation of large models to become productive, it is still It is an efficient and valuable method.

5.5 Is there a specified method for building a thinking chain?

  When building a thinking chain, the key is to frame the topic around solving the problem. The human problem-solving process can be given to the large model as structured as possible . For example, questions, contextual information, and related steps can be provided to the large model to help build a thinking chain.

5.6 How to implement the logic of NL2SQL

1. Generate executable SQL: Large models need to generate final SQL query statements, which require relevant header information and questions as input, and the ability to map the semantics of the question to SQL.

2. Preliminary work: Before SQL is generated, it is necessary to select relevant tables from a large number of tables. This can be achieved by converting table-related information (such as comments, business attribution, etc.) into vectors and performing similarity calculations. Table structure information also needs to be taken into account as strong constraints to determine which fields are most important.

3. Construct a knowledge base: Build a knowledge base that includes table information, field information and indicator information for correlation retrieval in the second step.

5.7 For scenarios that are sensitive to data accuracy, is it reliable to use large models?

  In extremely sensitive scenarios, the use of large models may require supplementary peripheral engineering methods, such as scientific research, to ensure data accuracy. Particularly where high precision is required, peripheral auxiliary engineering may be necessary.

  Fine-tuned data security depends on the large model technology used and the services provided by the vendor. If you research and use the model on your own, security depends entirely on where the data is stored. Cloud service providers can provide privatized instances to ensure data only goes in and out, and conduct audits to ensure data security.

5.8 Can instruction fine-tuning only be aligned or can new knowledge be injected?

  The core capability of instruction fine-tuning is alignment, which helps the model better understand and respond to specific instructions. Although fine-tuning can inject some new words or knowledge, this may not be economical if a large amount of new knowledge is required. In this case, a better approach may be to integrate new knowledge into the pre-training of the model and then fine-tune the instructions to use this knowledge more efficiently.

5.9 On which platform can ChatGLM be used to develop applications?

1. Zhipu Qingyan : can be used in C-side applications for conversational interaction, asking questions, etc.

2. Zhipu AI development platform : This platform provides an interface to the GLM model, allowing developers to develop applications locally. The open platform also provides a simple tool for adjusting instructions, as well as some infrastructure such as knowledge base services. Users can upload knowledge and utilize the technical details provided by the platform to implement an embedding database for answering questions. In addition, users can integrate the knowledge base into their organization by sharing links or interfaces for applications such as knowledge Q&A in the cloud.