About the author: A cloud computing network operation and maintenance personnel, sharing network and operation and maintenance technology and useful information every day.
Public account: Internet beans
Motto: Keep your head down and be respectful
Personal homepage: Internet Bean’s homepage
Recommended good books in this issue: "User Portraits: Platform Construction and Business Practice"
Fan benefits: Book gift: A total of 4 copies will be given away.
Participation method: Follow the public account: Internet Dou Cloud Computing School Reply Keywords: The fourth issue of book delivery
deadline: September 24, 2023 12:00 noon
From understanding user portraits to using portrait data to empower business, it is enough to read this book丨"User Portraits: Platform Construction and Business Practice"
In the era of big data, how to effectively mine the value of data and present it through portrait data, how to build platform functions based on portrait data and improve business output are things worth thinking about and putting into practice for all types of companies and business personnel.
Unleash the value of big data through profiling. In the era of big data, there is no lack of data, but a lack of systematic methods to mine the value of data. I hope to use this book to improve readers' understanding of portraits, and guide companies and business personnel to make full use of big data resources from the perspective of portraits and release more data value.
Explain clearly what the portrait platform is. This book clearly explains the construction process of the portrait platform and the ways to empower the business, helping readers gain a comprehensive and in-depth understanding of the portrait platform. By referring to the content in the book, readers will be more targeted in the process of building a profiling platform and using profiling data.
brief introduction
This is a book that explains user portraits from the shallower to the deeper levels from four levels: functional modules, technology implementation, platform construction, and business applications. The author has experienced the entire process of its user portrait platform from 0 to 1 and developed into a portrait middle platform in a leading Internet company, laying a solid technical foundation and accumulating rich business experience. This book examines the user portrait platform from the dual dimensions of technology and business. The entire process was reviewed.
Specifically, this book mainly contains the following contents:
(1) The role of profiling, the core functions and implementation logic of the four mainstream commercial profiling platforms in the industry;
(2) The main functions of the profiling platform, the technical architecture and technology selection of the profiling platform, and the data model of the profiling platform;
(3) The four major functional modules of the profiling platform: tag management, tag service, grouping function, and profiling analysis implementation plan;
(4) Build a user portrait platform from 0 to 1, including environment construction and front-end and back-end engineering framework construction;
(5) How the portrait platform empowers businesses in different life cycle stages of users and various business scenarios;
(6) Optimization and best practices of the portrait platform.
There are more than 200 design drawings and prototype drawings in the book, which can help readers understand the implementation principles and functional forms of the platform more intuitively. 20+ real application cases, technical solutions and cases are all from real projects. This book provides runnable code to help readers quickly build and deploy a user profiling platform.
Table of contents
Scroll up and down to view the catalog↓
Contents directory
Preface
Chapter 1 Understanding the Image Platform 1
1.1 Basic concepts of portrait 1
1.1.1 What is a portrait 1
1.1.2 The importance of portraits 2
1.1.3 Image platform positioning 3
1.2 Introduction to OLAP 3
1.2.1 Comparison between OLAP and OLTP 3
1.2.2 Key features of OLAP scenarios 4
1.2.3 3 types of modeling in OLAP 5
1.2.4 Development history of OLAP related technologies 5
1.3 Introduction to industry portrait platform 6
1.3.1 Shence Data 7
1.3.2 Volcano Engine Growth Analysis 10
1.3.3 GrowingIO 13
1.3.4 Alibaba Cloud Smart User Growth 16
1.4 Positions involved in the profiling platform 18
1.4.1 Data Engineer 18
1.4.2 Algorithm Engineer 18
1.4.3 R&D Engineer 18
1.4.4 Product Manager 19
1.4.5 Operations staff 19
1.5 Summary of this chapter 19
Chapter 2 Portrait Platform Function and Architecture 20
2.1 Main functions of the portrait platform 20
2.1.1 Tag Management 20
2.1.2 Tag Service 24
2.1.3 Grouping function 25
2.1.4 Image analysis 28
2.2 Image platform technical architecture 32
2.2.1 Common technical architecture of portrait platforms 32
2.2.2 Examples of imaging platform technology selection 33
2.2.3 Industry imaging function technology selection 35
2.3 Three data models of the portrait platform 36
2.4 Summary of this chapter 38
Chapter 3 Tag Management 40
3.1 Overall architecture of tag management 40
3.2 Label classification 43
3.2.1 Tag entity and ID type 43
3.2.2 Tag classification method 44
3.3 Implementation of tag management function 48
3.3.1 Tag storage 48
3.3.2 Label production 55
3.3.3 Tag data monitoring 67
3.3.4 Project implementation 69
3.4 Introduction to job division 70
3.5 Summary of this chapter 72
Chapter 4 Label Services 73
4.1 Overall architecture of label service 73
4.2 Tag query service 74
4.2.1 Introduction to tag query service 74
4.2.2 Filling tag data into cache 76
4.2.3 Tag data structure 79
4.2.4 Tag data processing 81
4.2.5 Project implementation 83
4.3 Tag metadata query service 85
4.3.1 Introduction to tag metadata query service 85
4.3.2 Project implementation 87
4.4 Tag real-time prediction service 89
4.4.1 Introduction to tag real-time prediction service 89
4.4.2 Project implementation 90
4.5 ID-Mapping 93
4.6 Introduction to job division 97
4.7 Summary of this chapter 98
Chapter 5 Group Function 99
5.1 Overall architecture of grouping function 99
5.2 Basic data preparation 101
5.2.1 Image table 101
5.2.2 Image BitMap 108
5.3 How to create crowds 111
5.3.1 Rule selection 112
5.3.2 Importing the crowd 119
5.3.3 Combination crowd 121
5.3.4 Behavior details 123
5.3.5 CrowdLookalike 125
5.3.6 Mining the crowd 126
5.3.7 LBS crowd 127
5.3.8 Selection of other groups 128
5.3.9 Project implementation 131
5.4 External output of crowd data 137
5.5 Crowd additional functions 138
5.5.1 Crowd estimation 138
5.5.2 Crowd splitting 140
5.5.3 Automatic crowd update 141
5.5.4 Crowd download 142
5.5.5 ID conversion 143
5.6 Crowd Deposit Service 144
5.6.1 Redis solution 144
5.6.2 BitMap solution 147
5.6.3 Rule-based judgment 149
5.7 Introduction to job division 150
5.8 Summary of this chapter 152
Chapter 6 Image Analysis 153
6.1 Overall structure of portrait analysis 153
6.2 Crowd portrait analysis 155
6.2.1 Crowd distribution analysis 155
6.2.2 Crowd indicator analysis 156
6.2.3 Crowd drill-down analysis 157
6.2.4 Population cross analysis 158
6.2.5 Comparative analysis of crowds 158
6.2.6 Project implementation 159
6.3 Impromptu crowd analysis 165
6.3.1 Distribution analysis and indicator analysis 166
6.3.2 Drill-down analysis and cross-analysis 167
6.3.3 Crowd portrait preview 168
6.4 Detailed analysis of behavior 169
6.4.1 Detailed statistics 171
6.4.2 User analysis 173
6.4.3 Process transformation 176
6.4.4 Value analysis 179
6.4.5 Project implementation 181
6.5 Single-user analysis 183
6.5.1 User portrait query 184
6.5.2 User relationship data analysis 185
6.5.3 Analysis of users gaining and losing followers 190
6.5.4 User content traffic analysis 192
6.6 Other common analyzes 193
6.6.1 Business Analysis Kanban 193
6.6.2 Regional analysis 195
6.6.3 Crowd placement analysis 197
6.7 Introduction to job division 199
6.8 Summary of this chapter 200
Chapter 7 Building a portrait platform from 0 to 1 201
7.1 Basic preparation 201
7.1.1 Technical component collaboration 201
7.1.2 Basic environment preparation 203
7.2 Building a big data environment 206
7.2.1 Hadoop 207
7.2.2 Spark 210
7.2.3 Hive 212
7.2.4 ZooKeeper 215
7.2.5 DolphinScheduler 216
7.2.6 Flink 217
7.3 Storage engine installation 219
7.3.1 ClickHouse 219
7.3.2 Redis 221
7.3.3 MySQL 222
7.4 Engineering framework construction 223
7.4.1 Server-side project construction 223
7.4.2 Front-end project construction 237
7.5 Running open source code 238
7.6 Summary of this chapter 240
Chapter 8 Portrait Platform Application and Business Practice 241
8.1 Common application cases of portrait platforms 241
8.1.1 Tag management application cases 241
8.1.2 Tag service application cases 244
8.1.3 Application cases of grouping function 245
8.1.4 Application cases of portrait analysis 247
8.2 The use of portraits in the user life cycle 248
8.2.1 Division of user life cycle 249
8.2.2 Use of portraits during the introduction period 250
8.2.3 Use of growth portraits 251
8.2.4 Use of mature portraits 252
8.2.5 Use of dormant period portraits 253
8.2.6 Use of portraits during the churn period 254
8.3 Business practice of portrait platform 255
8.3.1 User growth 255
8.3.2 User operation 259
8.3.3 E-commerce sales 263
8.3.4 Content recommendation 266
8.3.5 Risk control 268
8.3.6 Other businesses 271
8.4 Summary of this chapter 273
Chapter 9 Summary of Portrait Platform Optimization 274
9.1 Mission Mode 274
9.1.1 Task definition and execution mode 276
9.1.2 Task priority and concurrency control 277
9.1.3 Splitting parent-child tasks 277
9.1.4 Task anomaly detection and retry 278
9.1.5 Convenient horizontal expansion capabilities 279
9.2 Crowd Creation Optimization Advanced 279
9.2.1 Crowd selection requirements 279
9.2.2 Simple and direct solution 280
9.2.3 Using ClickHouse as a cache 281
9.2.4 SQL optimization 283
9.3 BitMap in the portrait platform
Usage plan 286
9.3.1 Basic principles of BitMap 286
9.3.2 BitMap selected in crowd circle
Usage 287
9.3.3 BitMap in distribution analysis
Usage plan 289
9.3.4 BitMap in the deposit service
Usage plan 291
9.4 Optimization of portrait wide table generation 292
9.4.1 Multi-table left join 293
9.4.2 Grouping and merging 294
9.4.3 Adding data loading layer 296
9.4.4 Using Bucket Join 297
9.5 ID encoding mapping scheme 299
9.6 How to build a platform similar to Shence 301
9.6.1 Shence product introduction 301
9.6.2 Main technical modules 302
9.7 Thoughts on platform technology optimization 305
9.8 Summary of this chapter 307
