triton client uses https protocol to access services

background

Normally, when calling model services, the http+IP link is used. However, due to the particularity of the author's environment, the access model must go through an https public network URL. Therefore, how to use the triton client to access the https link has become a problem.

reference

• TensorRT&Triton study notes (1): triton and model deployment + client

Research

First of all, what is the principle of nvidia tritonclient accessing url? It can be seen from the content of the infer function that the triton client accesses the server using the classic POST method, and the domain name is "v/models/{model_name}/infer".

Main process

If the domain name you want to access is https://api.aa.com/infer, then you only need the following code. Note that you do not need to urlinclude the protocol name.

url = 'api.aa.com/infer'
triton_client = httpclient.InferenceServerClient(url=url,
                                                     ssl=True,
                                                     insecure=True,
                                                     ssl_context_factory=gevent.ssl._create_unverified_context)

This setting allows you to access https links without verifying SSL. Notice that before executing the program, I made the following configuration.

os.environ['CURL_CA_BUNDLE'] = ''