[Transcript] TVP Technology Salon: Treasure Hunting AI Era

introduction

The big model is a 10x opportunity, but it is not an egalitarian opportunity and there is no low-hanging fruit.
If a company wants to achieve good results on a large model track, what should it do and which tracks should it choose?

Application cases of large models

Application industry	Application direction	Application level
Pharmaceutical companies	Training of medical representatives Training representatives’ expression skills and coping strategies	training
joint-stock bank	AI automatic Q&A Solve customers’ independent business problems and improve customer experience	copilot conversation
consulting firm	AI automatically analyzes data Generate daily and weekly reports	dig

Three questions about large models

• "Bias" or "drift" problem with large models
  • Answers must have sources (original text)
• Big model data security issues
  • Solve it at the technical level, such as federated learning, etc.
• Capability domain boundaries of large models
  • What can a large model do? You can compare a large model to an intelligent agent that has learned human knowledge.

Dimensions to consider for model implementation feasibility

Dimensions	business value	technical complexity	data	cost	Safety	Interpretability
1	Enhance decision-making skills	Do existing models on the market support it?	data availability	Computing hardware cost	Data security and privacy	Business scenario Interpretability of model resultsRequirements
2	Improve internal efficiency	Model fine-tuning complexity	Data quality meets standards	Commercial model cost	Model intellectual property
3	Improve customer experience	prompt project complexity	Data validation standards
4		data processing

Several feasible application directions of AIGC

1. SQL Copilot
   SQL Copilot is an auxiliary tool that can help developers write SQL query statements. It can automatically generate or correct SQL code for users by analyzing the user's query intent and database structure.
   SQL Copilot utilizes natural language processing and database understanding technology to enable developers to interact with the database faster and more accurately. This can improve development efficiency and reduce error rates.
2. BI Copilot
   BI Copilot is an auxiliary tool in the field of business intelligence (BI). It can assist analysts, managers and other personnel to quickly extract data, generate reports, and provide data visualization suggestions.
   BI Copilot can help business personnel use data to make decisions more efficiently through automation and intelligence. It can speed up the data analysis process and reduce tedious manual operations.
3. User Trend
   User trend analysis is to collect statistics and analysis of user behavior data to extract user preferences, behavior patterns and other information to optimize products or services.
   AIGC can help companies quickly analyze large amounts of user data, identify user preferences and behavior patterns, and provide companies with more targeted product optimization and marketing strategies.
4. Customer service session (intelligent customer service)
   Intelligent customer service can provide users with basic customer service support through chatbots or automatic response systems, solve common problems, and transfer users to Call the customer service staff.
   In the field of customer service, AIGC can save human resource costs for enterprises and improve customer service efficiency and satisfaction through automation and intelligence.
5. AI image recognition
   AI image recognition can be used in image recognition, object detection, face recognition and other fields. For example, smart photo albums can automatically identify people, scenes and other information in photos.
   Through deep learning and computer vision technology, AIGC can help people manage, search and share image information more conveniently, and can also play an important role in security, medical and other fields.
6. Internal data sorting (analyzing text content)
   AIGC can help enterprises automatically process and analyze large amounts of text data, extract key information, perform sentiment analysis, etc. to assist decision-making. formulated.
   Internal data organization is a broad application direction, which can be used to process various documents and reports within the enterprise to provide data support for enterprise management and decision-making.

AIGC storage

Three requirements:

1. Unified storage for Data Lake. Throughout the AIGC process, the amount of data stored is very large, and the storage requirements it brings need to be solved by data lakes to avoid the problem of data islands.
2. In the process of processing various businesses, there is a demand for data flow. If these data are stored in some traditional files, they will encounter the problem of data islands, so a unified storage is needed to provide services for them.
3. High throughput and low latency. In the AIGC scenario, GPU computing power is rare and expensive. Customers hope that the entire training can run as fast as possible and the GPU is used as fully as possible. This puts forward the underlying storage requirements. There is a requirement: the faster the data can be read out, the faster it can be provided to the upper level for training, so that the value is the highest.

Evolution of LLM engineering application paradigm

	Hard Coded	Prompt as Service	Orchestration as Service	Agent as Service	Autonomous Agent
Current plan	LingChain CallIndex	PromptPerfect Promptknit	Dify Semantic Kernel	AutoGPT BabyAGl	?
Traditional engineering analogy	Function Coding	Serverless	Wordpress	RPA	?
Technician Challenge	static coupling	Single capability and high integration difficulty	Engineering level and workflow transformation	Performance is uncontrollable	?
business people challenge	Unable to participate in collaboration	Unable to track performance	High learning cost	Unstable results and high cost	?

LLM programming collaboration paradigm changes

	Design First	API First	Prompt First
Profession	Requirements Analyst Designer	engineer	Prompt Engineer
output	User Interface Requirements Document	API 文档 Wearing code	Prompt Flow Agent Service
Task	Requirements analysis Drawing and restoration interface	API design API&open& test	Design expectations and verification set Write Prompt
tool	UML / Documentation / Figma	Swagger / Postman	PlayGround / Jupyter / Prompt IDE