Seamless integration: the perfect integration of AI model service and nacos based on FastAPI

1 Scenario Introduction

When we use python to build AI model algorithms, we often encounter the following problems:

• How does this model provide other microservice calls (such as microservices built with JAVA)?
• How does this model achieve load balancing of multiple service nodes?
• How does this model achieve service backup and failover?

Through a practical example, this project shows how to realize the perfect integration of AI model service and nacos based on FastAPI, so as to realize the intercommunication, load balancing, and failover between AI model service and other microservices. This project achieves the following four goals:

• Built a kmeans algorithm model based on sklearn
• Publish the model into an api interface based on FastApi
• Based on the nacos sdk, publish the api interface to the nacos registration center for calling by other microservices such as java to realize service discovery, load balancing, and failover
• Based on the nacos sdk, use nacos as the configuration center to synchronize the configuration changes in nacos in real time as parameters when building the model

Input parameters of the service interface: {"points":[[x1,y1],[x2,y2],[x3,y3],...,[xn,yn]]}

Output of service interface: {"results": [center1,center1,center2,...,centerx]}

Output the center number corresponding to each point value

1.1 Introduction to FastAPI

FastAPI is a modern, fast (high performance) python web framework. A web framework for building APIs using Python 3.6+, based on standard python type hints.

The main features of FastAPI are as follows:

• Fast: Very high performance, comparable to NodeJS and Go (thanks to Starlette and Pydantic), one of the fastest Python frameworks.

• Fast Coding: Increases development speed by about 200% to 300%.

• Fewer bugs: About 40% fewer human-induced errors by developers.

• Intuitive: Powerful editor support for shorter debugging times.

• Simple: Easy to use and learn. Spend less time reading documentation.

• Code Conciseness: Minimize code duplication. Each parameter can declare multiple functions to reduce program bugs.

• Robust: Production code automatically generates interactive documentation.

• Standards-based: Based on and fully compatible with open standards for APIs: OpenAPI and JSON Schema.

1.2 Introduction to Nacos

Nacos is a dynamic service discovery, configuration management and service management platform that makes it easier to build cloud-native applications. Nacos is dedicated to helping you discover, configure and manage microservices. Nacos provides a set of easy-to-use feature sets to help you quickly realize dynamic service discovery, service configuration, service metadata, and traffic management.

Nacos helps you build, deliver and manage microservice platforms more agilely and easily. Nacos is a service infrastructure for building a "service"-centric modern application architecture (such as microservice paradigm, cloud native paradigm)

Nacos supports the following core features:

• Service discovery: It supports DNS and RPC service discovery, and also provides multiple service registration methods such as native SDK and OpenAPI, and multiple service discovery methods such as DNS, HTTP and API.
• 2) Service health monitoring: Nacos provides real-time health checks on services, preventing requests from being sent to unhealthy hosts or service instances.
• 3) Dynamic configuration service: Nacos provides a unified configuration management function, which can help us manage the application configuration and service configuration of all environments in a centralized, external and dynamic manner.
• 4) Dynamic DNS service: Nacos supports dynamic DNS service weight routing, which allows us to easily implement middle-tier load balancing, more flexible routing strategies, traffic control, and simple DNS resolution services for the intranet of the data center.
• 5) Service and metadata management: Nacos supports the management of all services and metadata in the data center from the perspective of microservice platform construction, including management service description, life cycle, service static dependency analysis, service health status, service Traffic management, routing and security policies, service SLAs, and the most important metrics statistics.

2 code implementation

2.1 Construct a kmeans machine learning algorithm

from sklearn.cluster import KMeans

class SklKMeans:
    def __init__(self, clusters):
        self.model = KMeans(n_clusters=clusters, random_state=9)

    def fit(self, inputs):
        y_pred = self.model.fit_predict(inputs)
        return y_pred

Based on sklearn implementation, clusters are used as input parameters, the number of cluster center points.

The actual model initialization is that this parameter is synchronized from the nacos configuration center.

2.2 Publish the kmeans machine learning algorithm as an api interface based on FastApi

The resource address is: '/api/skl-kmeans'

The access parameters are: {points:[ [1,2],[2,3],[3,5],[4,6],[8,8],[10,11],[12,12], [12,15],[10,12]]}

The returned result is: { "results": [1, 1, 1, 1, 2, 0, 0, 0, 0]}

The number of default cluster center points: 3

from fastapi import APIRouter
from pydantic import BaseModel
from skl_kmeans import SklKMeans
import settings
import numpy as np

router = APIRouter()

class ClusterRequest(BaseModel):
    points: list

@router.post('/api/skl-kmeans')
async def api_skl_kmeans(req: ClusterRequest):
    try:
        model = SklKMeans(settings.clusters)
        response = model.fit(np.asarray(req.points))
        response = {"results": response.tolist()}
    except Exception as ex:
        response = None
    return response

2.3 Register the FastApi interface to the nacos registration center

The service is registered to nacos, the service name is "skl-kmeans", the service port is 9999, and the heartbeat interval is 30s

import nacos
from net_utils import net_helper

class NacosHelper:
    service_name = None
    service_port = None
    service_group = None

    def __init__(self, server_endpoint, namespace_id, username=None, password=None):
        self.client = nacos.NacosClient(server_endpoint,
                                        namespace=namespace_id,
                                        username=username,
                                        password=password)
        self.endpoint = server_endpoint
        self.service_ip = net_helper.get_host_ip()

    def register(self):
        self.client.add_naming_instance(self.service_name,
                                        self.service_ip,
                                        self.service_port,
                                        group_name=self.service_group)

    def unregister(self):
        self.client.remove_naming_instance(self.service_name,
                                           self.service_ip,
                                           self.service_port)

    def set_service(self, service_name, service_port, service_group):
        self.service_name = service_name
        self.service_port = service_port
        self.service_group = service_group

    async def beat_callback(self):
        self.client.send_heartbeat(self.service_name,
                                        self.service_ip,
                                        self.service_port)

    def load_conf(self, data_id, group):
        return self.client.get_config(data_id=data_id, group=group, no_snapshot=True)

    def add_conf_watcher(self, data_id, group, callback):
        self.client.add_config_watcher(data_id=data_id, group=group, cb=callback)

The necessary parameters for nacos are endpoint, namespace_id, group_name, username, password, data_id

nacos_endpoint = '192.168.1.238:8848'
nacos_namespace_id = '1bc91fa5-37e3-4269-8e41-3f4e4efb7633'
nacos_group_name = 'DEFAULT_GROUP'
nacos_username = 'nacos'
nacos_password = 'nacos'
nacos_data_id = 'skl-kmeans'

service_name = 'skl-kmeans'
service_port = 9999
beat_interval = 30

application.include_router(api_skl_kmeans.router, tags=['skl-kmeans'])

nacos = NacosHelper(nacos_endpoint, nacos_namespace_id, nacos_username, nacos_password)
nacos.set_service(service_name, service_port, nacos_group_name)
nacos.register()

@application.on_event('startup')
def init_scheduler():
    scheduler = AsyncIOScheduler()
    scheduler.add_job(nacos.beat_callback, 'interval', seconds=beat_interval)
    scheduler.start()

2.4 Use nacos as the configuration center, read the configuration at startup, and monitor configuration changes in real time

The data_id of the configuration file in the configuration center is "skl-kmeans". When the project starts, the configuration is forced to be synchronized once; when the configuration file is changed, the program can automatically monitor the configuration file change and automatically update it to the memory.

service_name = 'skl-kmeans'
service_port = 9999
beat_interval = 30

nacos_endpoint = '192.168.1.238:8848'
nacos_namespace_id = '1bc91fa5-37e3-4269-8e41-3f4e4efb7633'
nacos_group_name = 'DEFAULT_GROUP'
nacos_username = 'nacos'
nacos_password = 'nacos'
nacos_data_id = 'skl-kmeans'

def load_config(content):
    yaml_config = yaml.full_load(content)
    clusters = yaml_config['clusters']
    settings.clusters = clusters

def nacos_config_callback(args):
    content = args['raw_content']
    load_config(content)

# 注册配置变更监控回调
nacos.add_conf_watcher(nacos_data_id, nacos_group_name, nacos_config_callback)

# 启动时，强制同步一次配置
data_stream = nacos.load_conf(nacos_data_id,nacos_group_name)
load_config(data_stream)

2.5 Create a configuration file in nacos

The data_id of the configuration file is skl-kmeans

 The content is:

 2.6 Start the service

Check the registration status of skl-kmeans algorithm service in nacos

The registration has been successful. Next, call through postman (or call through java microservices) to construct the request parameter points and realize the clustering of points.

{"points":[[1,2],[2,3],[3,5],[4,6],[8,8],[10,11],[12,12],[12,15],[10,12]]}

{
    "results": [
        1,
        1,
        1,
        1,
        2,
        0,
        0,
        0,
        0
    ]
}

Dynamically adjust the clusters parameters in nacos, change the 3 center points to 4 center points, the adjustment is completed, and the kmeans algorithm has automatically synchronized the configuration changes. Call the interface again through postman, and the returned result has been clustered to 4 points

 This shows that the kmeans algorithm can monitor the configuration changes in nacos in real time and realize dynamic clustering

3 Complete code download

Code address: complete code