TensorFlow Serving multi-model deployment and call a different version of the model

 

Premise: To achieve multi-model deployment, we must first understand and skilled achieve a single deployment model, you can make use of the official website of the document, use the Docker achieve deployment.

 

1. First prepare two models you need to deploy unified in multiModel / folder (the folder name can be arbitrarily taken), the directory structure is as follows:

multiModel/
├── model1
│   └── 00000123
│   　　 ├── saved_model.pb
│   　　 └── variables
│   　　　　 ├── variables.data-00000-of-00001
│   　　　　 └── variables.index
├── model2
│   　　└── 00000123
│   　　　　 ├── saved_model.pb
│   　　　　 └── variables
│   　　　　　　 ├── variables.data-00000-of-00001
│   　　　　　　 └── variables.index
└── models.config

 

 

2. Create models.config profile multiModel / directory, as follows:

model_config_list: { 
    config: { 
      name: " MODEL1 " , 
      base_path: " / Models / multiModel / MODEL1 " , 
      model_platform: " tensorflow " 
    }, 
    config: { 
      name: " MODEL2 " , 
      base_path: " / Models / multiModel / MODEL2 " , 
      model_platform: " tensorflow " 
    }, 
} 

# Note: the 
   previous base_path path / models / fixed wording, write your name with the model directory path behind
   否则报错：FileSystemStoragePathSource encountered a filesystem access error: Could not find base path /home/node1/model/multiModel/model1 for servable model1

 

 

Configuration file path name defines the model and the model in the container, now run tf-serving containers:

sudo docker run -p 8501: 8501 --mount type = bind, source = / home / node1 / model / multiModel /, target = / models / multiModel -t tensorflow / serving: latest-gpu --model_config_file = / models / multiModel /models.config 

# Note: 
write behind taget 1 is the total path model, some differences with the single mode, single mode is written to a specific model of a path; 
front path profile is fixed behind the wording 2. --model_config_file / models /, followed by the path of the profile, if the write absolute or relative path to find models.config, will complain can not find the file or path;

 

 

4. Finally, saw such an interface, and view GPU already occupied, indicating multiple model has been deployed successfully:

 

 

 

 

 

------------------------------ ----------------- specified model version -----------------------

If there are multiple versions of a model, and hope that when the predicted version of the specified model can be implemented in the following manner.
Modify model.config file, add model_version_policy parameters:

model_config_list:{
    config:{
      name:"model1",
      base_path:"/models/multiModel/model1",
      model_platform:"tensorflow",
      model_version_policy:{
        all:{}
      }
    },
    config:{
      name:"model2",
      base_path:"/models/multiModel/model2",
      model_platform:"tensorflow"
    },
}

When the request predictable, if you want to use for the 00000124 version of the model, as long as the modified request URL:

URL = 'http://公网ip:8501/v1/models/model1/versions/00000124:predict'

After running the container, if the new model file folder as 00,000,124 / of the host / home / node1 / model / multiModel / model1 / file, tfserving will automatically load a new model; Similarly if the removal of the existing model, tfserving also automatically uninstalls model.

 

These documents Reference: https://blog.csdn.net/JerryZhang__/article/details/86516428   thanks ~

Multi-model deployment has been recurring, temporary version of the model does not have this need not reproduce ~