From: Expertise
Enter the NLP group —> join the NLP exchange group
The emergence of large language models marks a revolutionary breakthrough in the field of artificial intelligence. Thanks to unprecedented training scales and model parameters, the capabilities of large language models have been significantly improved to achieve human-like performance in understanding, language synthesis, and commonsense reasoning . This major leap in general AI capabilities will fundamentally change how personalized services are implemented.
First, it will change the way humans interact with personalized systems . Instead of being a passive information filtering medium like traditional recommender systems and search engines, large-scale language models provide the basis for active user engagement. On such a new basis, users' requests can be actively explored and the information required by users can be provided in a natural, interactive and interpretable manner.
Secondly, it will also greatly expand the scope of personalized services, making it develop from merely collecting personalized information to providing composite functions of personalized services . By utilizing a large language model as a common interface, a personalization system can compile user requests into plans, call functions of external tools (such as search engines, calculators, service APIs, etc.) to execute these plans, and integrate the output of these tools, Complete end-to-end personalization tasks. Today, large-scale language models are still developing rapidly and remain largely unexplored in personalization applications.
Therefore, we believe it is time to examine the challenges of personalization and the opportunities to address them with large language models. In particular, we dedicate this forward-looking paper to the development and challenges of existing personalization systems, the emerging capabilities of large language models, and potential ways how to leverage large language models for personalization .
The advent of large language models [1] has shown remarkable progress in understanding human expressions, profoundly impacting the AI community . Equipped with massive amounts of data and large-scale neural networks, these models demonstrate remarkable abilities in understanding human language and generating text similar to ours. Among the capabilities are reasoning [2], few-shot learning [3], and incorporating large amounts of world knowledge in pretrained models [1]. This marks a major breakthrough in the field of artificial intelligence, leading to a revolution in our interaction with machines. As a result, large language models have become indispensable in applications ranging from natural language processing and machine translation to creative content generation and chatbot development. The launch of ChatGPT, in particular, has received a lot of attention from the human community, prompting reflection on the transformative power of large language models and their potential to push the boundaries of what AI can achieve. This disruptive technology promises to change the way we interact with and utilize AI in countless domains, opening new possibilities and opportunities for innovation. As these language models continue to advance and develop, they promise to shape the future of artificial intelligence, allowing us to explore uncharted territory and unlock greater potential in human-machine collaboration.
Personalization, the art of tailoring experiences to individual preferences, is a critical and dynamic link bridging the gap between humans and machines . In today's technology-driven world, personalization plays a key role in enhancing user interaction and engagement with various digital platforms and services. By adapting to individual preferences, personalization systems give machines the ability to meet the unique needs of each user, making interactions more efficient and enjoyable. Furthermore, personalization is more than just content recommendations; it encompasses every aspect of user experience, including user interface, communication style, and more. As artificial intelligence continues to advance, personalization becomes increasingly complex in handling the high volume of interactions and diverse user intents. This requires us to develop more advanced technologies to deal with complex scenarios and provide a more enjoyable and satisfying experience. The quest for improved personalization is driven by the desire to better understand users and meet their changing needs. As technology develops, personalization systems are likely to continue to evolve, ultimately creating a future where human-computer interaction is seamlessly integrated into every aspect of our lives, providing a personalized and tailored experience in our daily lives.
Large-scale language models, with their deep and wide capabilities, have the potential to revolutionize personalization systems, change the way humans interact and expand the scope of personalization . Interaction between humans and machines can no longer only be classified as active and passive, just like traditional search engines and recommender systems. However, these large-scale language models go beyond simple information filtering, and they also provide diverse additional functions. Specifically, the system will actively and comprehensively explore the user's intent, enabling more direct and seamless communication between the user and the system through natural language. Unlike traditional techniques that rely on abstract and hard-to-interpret ID-based information representations, large-scale language models enable a deeper understanding of users' exact needs and interests. This deeper understanding paves the way for higher-quality personalized services that meet user needs and preferences in a more granular and efficient manner. In addition, the integration of various tools has been greatly enhanced through the capability of large language models, greatly expanding the possibilities and application scenarios of personalized systems. By translating user needs into plans, including understanding, generating, and executing them, users can access a wide variety of information and services. Importantly, users are unaware of the complex transition process happening behind the scenes, as they experience a seamless end-to-end model. From this perspective, the potential of large language models for personalization has not been fully explored.
This paper explores the challenges in personalization and explores potential solutions using large language models . In existing related work, LaMP [4] introduces a new benchmark for training and evaluating language models to generate personalized output in information retrieval systems. On the other hand, other related surveys [5], [6], [7] mainly focus on traditional personalization techniques such as recommender systems. From the perspective of learning mechanism, LLM4Rec [5] deeply explores discriminative LLM for recommendation and generative LLM for recommendation. Regarding the "where" and "how" of LLM adaptation to recommendation systems, Li et al. [6] focused on the overall process of the industrial recommendation stage. While Fan et al. [7] conducted a review focusing on pre-training, fine-tuning and hinting methods. While these works discuss pretrained language models like Bert and GPT for ease of analysis, they pay limited attention to the emerging capabilities of large language models. This paper aims to fill this gap by examining the unique and powerful capabilities of large language models in the context of personalization, and to further extend the scope of personalization with tools.
The rest of this review is organized as follows : We review personalized and large-scale language models in Section 2 to outline their developments and challenges. We then carefully discuss the potential role of large language models in personalization in Section 3, ranging from simple exploitation of emerging capabilities to complex integration with other tools. We also discuss possible challenges in adapting large language models for personalization.
Large Language Models for Personalization
In the sections that follow, we delve into the potential of large language models for personalization, from simple usage scenarios, such as exploiting lexical knowledge as features, to more complex integrations with other tool modules, allowing them to act as proxy role. Specifically, we focus on the progression of emerging capabilities, starting with basic world knowledge and understanding user intent, and progressing to advanced reasoning capabilities. We explore how large language models can help build a knowledge base that enriches general knowledge about various items. In addition, we discuss how the comprehension capabilities of large language models empower content explainers and explainers with in-depth analysis of interactions. Additionally, we observe attempts to leverage the reasoning capabilities of large language models to provide recommendation results to system reasoners. These increasingly sophisticated capabilities enable the sophisticated utilization of large language models and other tool modules to better understand user intent and fulfill user instructions. Therefore, we also explore the integration of large language models with other personalization tools, including tool learning, conversational agents, and personalized content creators. An overview of this chapter is shown in Figure 1. Our comprehensive survey aims to provide a deeper understanding of the current landscape and shed light on the opportunities and challenges posed by the integration of large language models into personalization.
Big Model as Knowledge Base
The ability of Large Language Models (LLMs) to retrieve factual knowledge as an explicit knowledge base [38], [39], [40], [41], [42], [43], [40], [ 41], [44], [45], [46] have aroused extensive discussions, which provide opportunities for building more comprehensive knowledge graphs within recommender systems. Dating back to the work of [38], large language models have demonstrated their impressive capabilities in storing factual information, such as entities and common sense, and reliably transferring common sense to downstream tasks. Existing knowledge map methods are difficult to deal with incomplete KGs [47] and use text corpus to construct KGs [48]. Many researchers try to use the ability of LLMs to solve these two tasks, that is, knowledge map completion [49] and The construction of knowledge graph [50]. For knowledge graph completion, which refers to the task of missing facts in a given knowledge graph, recent efforts have been devoted to encoding text or generating facts for knowledge graphs. MTL-KGC [51] encodes text sequences to predict the likelihood of tuples. MEMKGC [52] predicts masked entities for triplets. StAR [53] encodes entities separately using a Siamese text encoder. GenKGC [54] directly generates tail entities using a decoder-only language model. TagReal [55] generates high-quality prompts from an external text corpus. AutoKG [48] directly adopted LLMs, such as ChatGPT and GPT-4, and designed custom hints to predict tail entities. As for another important task, knowledge graph construction, which refers to creating a structured representation of knowledge, LLMs can be applied in the process of building knowledge graphs, including entity discovery [56], [57], coreference resolution [58], [59] and relation extraction [60], [61]. LLMs can also achieve end-to-end construction [62], [50], [42], [63], [55], construct KGs directly from raw text. LLMs allow knowledge extraction to build knowledge graphs. symbolic-kg [64] extracts common sense facts from GPT3 and then fine-tunes a small student model to generate a knowledge graph. These models have demonstrated the ability to store large amounts of knowledge, providing a viable option for increasing the scope and depth of knowledge graphs. Furthermore, these advances have prompted research into the direct transfer of stored knowledge from LLMs to knowledge graphs, eliminating the need for human supervision. This interesting study reveals the possibility of auto-completing knowledge graphs with cutting-edge large-scale language models.
LLMs as content interpreters
Content-based recommenders provide an effective solution to alleviate the sparse feedback problem in recommender systems. By exploiting the attributes and characteristics of items, these systems gain a deeper understanding of their attributes, enabling accurate matching with user preferences. However, content features used in content-based recommendation may also exhibit sparsity. Relying solely on recommended supervisory signals, such as clicks and views, may not fully exploit the potential benefits of these features. To overcome this challenge, language models emerged as powerful fundamental algorithms that act as content interpreters while processing textual features. Their utilization enhances the effectiveness of recommender systems to effectively understand and interpret textual content, thereby improving recommendations.
in conclusion
Overall, the advent of large language models represents a major breakthrough in the field of artificial intelligence. Their enhanced capabilities in understanding, language analysis, and commonsense reasoning open new possibilities for personalization. In this paper, we discuss the timing of adapting large language models to personalization systems from several perspectives. We have observed that this progress has evolved from exploiting the low-level capabilities of large language models to improve performance, to exploiting their potential for end-to-end tasks in complex interactions with external tools. This evolution promises to revolutionize the way personalized services are delivered. We also acknowledge the open challenges of integrating large language models into personalization systems.
Enter the NLP group —> join the NLP exchange group