Recommendation algorithms and systems have been widely used around the world, providing users with a more personalized and intelligent product recommendation experience. In the field of recommendation systems, the reuse and consistency of feature data in AI modeling seriously affect the modeling efficiency. Alibaba Cloud's machine learning platform PAI launched a feature platform (PAI-FeatureStore) . In all AI modeling scenarios that require features, users can easily share and reuse feature data through the Feature Store, reducing resource and time costs and improving work efficiency.
What is a feature platform
Feature Store is a centralized data management and sharing platform for organizing, storing and managing feature data used in machine learning and data science. Solve the problem of AI model training and reasoning input feature data in multiple subdivision scenarios.
Alibaba Cloud's machine learning platform PAI-FeatureStore is deeply integrated with Alibaba Cloud's multiple cloud products , encapsulating the full link from features to models. Moreover, based on the development of the recommendation algorithm process, it realizes seamless connection with the existing mature recommendation process, further improving the efficiency of algorithm engineers and developers.
Through PAI-FeatureStore, effectively improve work efficiency, reduce resource costs and development time. As a centralized, scalable, and efficient feature data storage and access solution, it solves the problems of feature data reuse, consistency, discoverability, and manageability in AI modeling. PAI-FeatureStore automatically completes the construction of online and offline tables to ensure the consistency of online and offline. At the same time, when only one feature table is stored, features can be shared with multiple people; offline storage supports Alibaba Cloud's cloud-native big data computing services In terms of online storage, MaxCompute supports Alibaba Cloud's real-time data warehouse Hologres, GraphCompute, TableStore and other products. Algorithm engineers can complete feature processing through manual operations on the web page or Python SDK without having to understand the details of the use of each storage product.
Applicable scenarios and advantageous functions of PAI-FeatureStore
The Alibaba Cloud machine learning platform PAI-FeatureStore is suitable for AI modeling scenarios that require features, such as recommendation scenarios, user growth, advertising, or financial risk control scenarios, and provides a unified data feature storage and management platform for data analysts and modelers. It is convenient for data processing, feature extraction and analysis.
Currently, the main functions of PAI-FeatureStore are as follows:
- Offline data is consistent with online data : In PAI-FeatureStore, the data synchronization operation of each product is encapsulated into a line of data synchronization code, which helps users shield the cumbersome data authorization and other operational details of different storage products, ensures data consistency, and improves feature data Accuracy and efficiency of processing and use;
<!---->
- Automatic association of feature tables : In PAI-FeatureStore, it is supported to combine various features required for model training and export them into training tables required for model training. When the features required for training are scattered in multiple different tables, PAI-FeatureStore supports automatic association and export of multiple tables. In addition, it supports sequence table export, export by event_time association, automatic sorting by table size and optimization of export time, etc.;
- Automatic model feature analysis : PAI-FeatureStore supports PAI-EAS to automatically analyze the features that the model needs to use, and automatically load the relevant features. By specifying the project name and model feature name in PAI-FeatureStore, the prediction engine can automatically analyze the required features and load them, simplifying the use process;
- Second-level reading of real-time features : PAI-FeatureStore supports the registration of customers to classify features. In the recommendation scenario where the real-time feature value changes at the second level, the feature link is highly required. When a feature is requested to be read online, PAI-FeatureStore will judge that if the feature to be read is a real-time feature, it will directly store the feature online. read. Thousands of real-time features can be read within 15ms, meeting low latency requirements;
- Multi-version feature management : PAI-FeatureStore supports incremental feature mining to solve the problem of complex feature types and diverse online data sources. It facilitates online model iteration and saves storage resources;
In addition, PAI-FeatureStore is also deeply integrated with PAI full-link recommendation system PAI-REC to realize offline consistency check; all product capabilities of PAI-FeatureStore can be directly used through SDK; PAI-EAS can directly pull features from MaxCompute, reducing Features such as online storage pressure.
How to use PAI-FeatureStore
Please refer to the product documentation for the usage steps: https://help.aliyun.com/zh/pai/user-guide/feature-store/
PAI-FeatureStore is expected to be officially launched in the entire Region in mid-to-late September 2023 .
Currently PAI-FeatureStore is only available for whitelist applications . If you want to use the PAI-FeatureStore function, you can search the group number "34415007523" on DingTalk or scan the QR code below to enter the application Q&A group.
