Scene introduction
Build a TensorFlow Serverless AI inference platform based on Alibaba Cloud Function Computing. .
background knowledge
Function Compute is an event-driven fully managed computing service. Using function computing, you don't need to purchase and manage infrastructure such as servers, just write and upload code. Function computing prepares computing resources for you, runs tasks flexibly and reliably, and provides functions such as log query, performance monitoring, and alarms. Function computing helps you to quickly build applications without the need to manage the server (Serverless) and only focus on the function code. Function calculations can be flexibly scaled, you only need to pay for what you use.
Summary of advantages: CPU-intensive main advantages such as AI reasoning based on function calculations.
1. Easy to get started, only focusing on business logic development, greatly improving engineering development efficiency.
Self-built solutions have too much learning and configuration costs. For example, for different scenarios, ESS needs to do various parameter configuration
system environment maintenance and upgrades.
- Free operation and maintenance, function execution level granularity monitoring and alarm.
3. Elastic expansion in milliseconds to ensure high flexibility and availability, while covering delay-sensitive and cost-sensitive types.
4. In a CPU-intensive computing scenario, by setting a reasonable combined billing model, it has cost advantages in the following scenarios:
Request access has obvious peaks and valleys, and there is no request even at other times. There
is a certain stable load request, but there are sudden changes in the request volume during some time periods.
Scene experience
The scenario will introduce the best practice of using function computing products to deploy a deep learning AI inference platform, based on function computing.
Allows developers in the field of artificial intelligence to focus more on the training of algorithm models and the development of business logic, and delegate the operation and maintenance of computing clusters to function computing to improve work efficiency.
The final result:
By uploading a photo of a cat or dog, it can be recognized whether the animal in the photo is a cat or a dog. DEMO example effect entrance: http://sz.mofangdegisn.cn .
Step 1: Connect to the ECS server
ECS resource address: https://developer.aliyun.com/adc/scenario/35a73acbb9cb477e8a4a5b22021f23a9
Reference article steps link ECS
Step 2: Open function computing service
Before using function calculation, you need to activate the service on the Alibaba Cloud product details page .
Step 3: Install Fun Tool
Fun is a tool used to support serverless application deployment, which can help you conveniently manage resources such as function computing, API gateways, and log services. It uses a resource configuration file (template.yml) to assist you in developing, building, and deploying operations.
1. Download the Fun installation package to the local. Please set the value of the FUN_VERSION variable below to the latest version number. Please refer to Releases for the latest version number. The following operations will install and configure Fun on the ECS server.
FUN_VERSION="v3.6.1"
curl -o fun-linux.zip https://gosspublic.alicdn.com/fun/fun-$FUN_VERSION-linux.zip2. Install the unzip tool unzip, and use unzip to unzip the downloaded Fun installation package to the current directory.
yum -y install unzip
unzip fun-linux.zip3. Move the decompressed executable file to the system path.
mv fun-*-linux /usr/local/bin/fun
4. Configure the Alibaba Cloud account associated with Fun and the timeout period for calling the SDK.
fun config
Please refer to the following information to enter your Alibaba Cloud account ID, AccessKeyID, and AccessKey key information.
- Aliyun Account
ID: The part after the @ symbol in the sub-user name is the main account ID assigned to you. For example , the account ID contained in u-zftb79nc@105997185439 is 105997185439 . - Aliyun Access Key ID and Aliyun Access Key Secret: Please use the sub-account AK ID and AK
Secret provided to you by the platform .
- The timeout in seconds for each SDK client
invoking: It is recommended to set the timeout time a bit longer (for example, 10000) to prevent timeout interruption when uploading the deployment package.
The correct settings are as follows:
Step 4: Install Docker
1. Install Docker's dependent libraries.
yum install -y yum-utils device-mapper-persistent-data lvm2
2. Add the software source information of docker-ce.
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo3. Install docker-ce.
yum makecache fast
yum -y install docker-ce4. Start the Docker service.
systemctl start docker
Step 5: Create NAS file system mount point
1. Log in to the NAS console .
2. Choose File System> File System List.
3. In the upper left corner, select the location of the cloud resource. 
4. Find the file system corresponding to the NAS file system ID in the resource information column on the left side of the experience page, and click More> Add Mount Point.
5. On the Add Mount Point page, configure the following parameters. 
6. Check the mount point address, as shown below.
Step 6: Download the project code
1. Install the Git tool and use Git to clone the project code to the local.
yum -y install git
git clone https://github.com/awesome-fc/cat-dog-classify.git2. Edit the FUN deployment template file template.yml in the project, modify the log service project name and NAS storage mount configuration.
a. Use vim to open the template file.
cd cat-dog-classify
vim template.ymlb. Enter the following command to replace the log service project name globally.
:%s/log-ai-pro/log-ai-test/g
In the command, log-ai-test is the modified project name, and you can replace it with your custom project name.
c. Modify the NAS mount configuration. The default auto method will automatically create the NAS file system. Here we modify and fill in the information of the NAS file system pre-created for the platform.
VpcConfig:
VpcId: 'vpc-uf6u8*****zf1r4'
VSwitchIds: [ 'vsw-uf6******038fy' ]
SecurityGroupId: 'sg-uf6e******w6qz'
NasConfig:
UserId: 10003
GroupId: 10003
MountPoints:
- ServerAddr: '33****u83.cn-shanghai.nas.aliyuncs.com:/'
MountDir: '/mnt/auto'The modified parameters are described as follows:
- For the VpcId, VSwitchId, and SecurityGroupId information in VpcConfig, please check the instance details page of the ECS console.
- Modify the value of ServerAddr to the mount point address in step 3.
After modification, it is shown in the figure below:
Step 7: Upload the dependent files to the NAS
1. Execute the following command to initialize the NAS.
fun nas init
2. View the directory location of the local NAS.
fun nas info
The command output is as follows:
3. Execute the following commands to install related dependencies.
fun install -v
The result of successful command execution is shown below.
The command will be based on the definition in the Funfile file:
- Download the dependencies to the .fun/nas/auto-default/classify/python directory.
- Copy the model files in the model to the .fun/nas/auto-default/classify/model/ directory.
4. Synchronize the local NAS directory to the Alibaba Cloud NAS storage.
fun nas sync
The command execution result is shown below. 
Note that the synchronized files are about 700M, with a lot of content, and the synchronization time is about 40 minutes, please be patient.
5. Check whether the file is uploaded to the remote NAS directory.
fun nas ls nas://classify:/mnt/auto/
Step 9: Deploy to the functional computing platform
Do the following to deploy the function to the function computing platform:
fun deploy -y
The execution result is shown in the figure below. In the figure, 19176540-18576685985*****.test.functioncompute.com is the temporary domain name allocated by the function computing platform. It is valid for 10 days and has a limit of 1000 calls per day. 
If the following error message appears during deployment. 
The cause of the problem: The log item name defined in the template configuration file already exists.
Solution: Please modify the log item name in the template.yml file.
Step 9: Test the prediction service
1. After the deployment is completed, wait for about 30 seconds and open the temporary domain name in the browser, as shown in the figure below. 
If a module error similar to the following is introduced when accessing the page. 
Cause of the problem: Due to network problems, the NAS sync content is incomplete.
Solution: Please execute the command fun nas sync again.
2. Upload a picture of a cat or dog. 
3. Click the Predict button to view the predicted recognition results. 
Function computing has the characteristics of dynamic scaling. According to the amount of concurrent requests, the execution environment is automatically and elastically expanded to execute the environment. When the reserved instance cannot satisfy the request to call the function, the function calculation will automatically expand the instance on demand for the function to execute, and the call at this time will have a cold start process. Using reserved mode can eliminate cold start glitches. For details, see Reserved Instances.