1. What is Serverless?
1. Introduction to Serverless Technology
Serverless (serverless architecture) refers to the server-side logic implemented by developers running in a stateless computing container. It is triggered by events and is completely managed by a third party. Its business-level state is used by developers. and storage resources are recorded.
Serverless eliminates the need for developers to deal directly with servers (whether physical machines, virtual machines, containers, etc.). The advantages of no host will greatly reduce the operational overhead of users in terms of server maintenance, and there is no need to worry about upgrading the server. No host also means that the metrics that need to be monitored in the application will be different. This is because most of the underlying services used no longer publish traditional metrics such as CPU, memory, disk size, etc. This makes it no longer necessary to pay special attention to the underlying operational details of the architecture.
Adrian Cockcroft, vice president of AWS cloud architecture strategy, once gave a simple way to define Serverless: " If your PaaS can effectively start an instance within 20 milliseconds and run for half a second, then it can be called Serverless ." And in the description in the A Berkeley View on Serverless Computing paper: Serverless = FaaS + BaaS .
Referring to the technical logic diagram of the serverless architecture above, we can see that the serverless architecture relies heavily on third-party services (also called backend as a service, or "BaaS") or custom code running in temporary containers (function as a service, or "FaaS") applications , so FaaS and BaaS can be regarded as two specific implementations of Serverless . A function is the smallest unit of abstract language runtime in a serverless architecture. In this "use-as-you-go" architecture, it's not about how much CPU or RAM or any other resources are needed to run a function, but rather about the time it takes to run the function, and you only pay for the time those functions run.
The following is a brief analysis of the advantages of Serverless, as well as some current shortcomings:
The advantages of the serverless
- Faster development speed : Compared with IaaS and PaaS, the FaaS platform based on serverless technology abstracts and extracts the logic part of customer business code in the process of running traditional applications ( forming the pattern of user function code + BaaS ), developers only need to According to the cloud platform standard development function set, developers do not need to care about what framework is used at the bottom layer - they only need to develop business logic; they do not know how many instances are running and where they are running, all of which allow developers to focus more on Focus on business code.
- Built-in automated deployment : Serverless architectures usually implement built-in automated deployment through APIs and various tools provided on the platform. Typically, these tools listen for changes to the source code repository and automatically deploy to production when the code is updated. During deployment, the tool packages the code into one or more functions, uploads it to the FaaS platform, and configures it with the appropriate URL and event triggers in response to requests. In this way, each request will trigger the execution of the corresponding function, thus realizing the real-time deployment of the code.
- Reduce resource overhead: When the function is not requested, the function instance is 0. When there is a request, the cloud platform will quickly pull up an instance to carry the traffic. More importantly: Functions can be billed according to the request, that is, you pay once for each call, while traditional services are billed according to the instance specification.
Some shortcomings of the Serverless :
- State management: In order to achieve free scaling, statelessness is a must. For stateful services, using serverless will lose flexibility. Stateful services need to interact with storage, which inevitably increases latency and complexity.
- The computing environment is highly standardized: the underlying server and operating environment on which serverless computing applications run are transparent to users, and users cannot choose or optimize. The environment in which serverless computing programs run is highly standardized, and some dependencies that depend on specific operating environments, specific server versions, or even specific hardware resources are difficult to ensure compatibility.
- Local testing : At present, there is still a lack of mature debugging and development tools in the development process using the Serverless architecture.
- cold start time
2. Serverless key technologies
(1) Event-driven
Serverless 's "computing only when running" feature means that it is a "strict" event-driven computing. Event-driven programming design This design model was conceived in the case of interactive programs (Interactive program). This also means that the system has a huge change in the programming model. When we write GUI programs, such as desktop programs and web front-end applications, we start the corresponding processing logic by listening to the user's operations on buttons, links and other components. This is similar to Serverless, which responds to the user's behavior only when the user is using it.
(2) Container technology
Container technology is usually implemented through Docker in the Serverless architecture. The cloud service provider will package the code for each function into a separate Docker container. Inside a container, you can ensure that code can run in an isolated environment without affecting the operation of other functions or applications.
When a request arrives, the cloud service provider executes the function inside the container and destroys the container when done. This practice helps to ensure that the execution of each function is short-lived and does not take up too many resources, thereby avoiding resource waste.
By using container technology, the serverless architecture can ensure that functions run in an isolated environment and can be rapidly expanded to meet growing demand. At the same time, container technology can also improve the security of applications, because the permissions of functions can be restricted to prevent functions from accessing data that should not be accessed.
3. Comparison with common technical architecture
(1) Traditional Architecture-VS- Serverless
The technical architecture of a traditional Java Web application may be as shown in the following figure:
Such an architecture can make the front-end very light, without any application logic, it is only responsible for rendering the user interface and sending the request to the back-end through HTTP, and all data operations are done by the back-end Java program. Such an architecture is relatively easy to develop, but it is indeed very complicated to maintain. Both front-end development and back-end development require very professional personnel, environment configuration, and special personnel to maintain the database and update and upgrade applications.
In the serverless architecture, we no longer need to store any session state in the server-side code, but directly store them in NoSQL, which will make the application stateless and help elastic expansion. The front-end can directly use BaaS to reduce the coding requirements of the back-end. This architecture essentially reduces the labor cost of application development, reduces the risk of maintaining infrastructure, and makes use of cloud capabilities to facilitate expansion and rapid iteration.
（2）MicroService -VS- Serverless
Microservice (MicroService) is another hot topic in the field of software architecture. If microservices are based on small functional blocks that focus on single responsibilities and functions, and use modularization to combine complex large-scale applications, then we can further think that the Serverless architecture can provide a more "code fragmentation" "Software architecture paradigm, we call it Function as a Services (FaaS). The so-called "function" (Function) provides a smaller program unit than microservices. For example, a microservice could represent the code needed to perform all CRUD operations for a customer, while a "function" in FaaS could represent every operation a customer would perform: create, read, update, and delete. When the "account creation" event is triggered, the corresponding "function" will be executed through the AWS Lambda function. From this level of meaning, we can simply equate the Serverless architecture with the concept of FaaS.
Serverless is inherently complementary to the microservice architecture . A Serverless application has its own gateway, database, and interface, and you can also use your favorite language (limited by service providers) to develop services. In other words, a Serverless might be a perfect microservice instance in this situation.
（3）Serverful Cloud -VS- Serverless Cloud
The three key differences between traditional
- ** Decoupled computing and storage. **Storage and computing are scaled separately, provisioned and priced independently. Typically, storage is provided by a separate cloud service and computation is stateless.
- ** Execute code without managing resource allocation. **Users provide a piece of code instead of requesting resources, and the cloud automatically provides resources to execute that code.
- Pay in proportion to resources used, not allocated . Billing is performed based on a certain dimension related to execution (such as execution time), not according to a certain dimension of the basic cloud platform (such as the size and number of allocated VMs).
Some comparisons between traditional cloud platforms and other technical details of serverless platforms (taking AWS as an example):
4. Serverless industry engineering practice
- AWS Lambda: Introduction to AWS Lambda Cloud Computing Service_How to use AWS Lambda-AWS Cloud Service(amazon.com)
- Alibaba Cloud Function Computing Platform: Function Computing FC_Serverless Computing_Serverless_Containers and Middleware-Alibaba Cloud (aliyun.com)
- Open source FaaS platform
- etc.
2. FaaS
1、IaaS -> CaaS -> PaaS -> FaaS
-
IaaS (Infrastructure as a Service): Infrastructure as a Service. It refers to providing infrastructure (such as computing, storage, network, etc.) as a service, and users can build their own application environment on this basis.
-
CaaS (Containers as a Service): Containers as a Service. It refers to providing container technology to users, and users can run their own applications through containers.
-
PaaS (Platform as a Service): Platform as a Service. It refers to the provision of development platforms and application management tools. Users only need to pay attention to the development and operation of applications, and do not need to pay attention to the underlying infrastructure.
-
Serverless: Serverless architecture. It means that there is no need to explicitly manage and configure server resources in order to run applications, and all underlying management and maintenance are the responsibility of the platform provider. Users only need to upload their own code and wait for the code to be run automatically. Serverless = FaaS + BaaS . BaaS (backend as a service)
-
FaaS (Function as a Service) Function as a Service: As a cloud computing service, it allows developers to deploy individual functions at cloud providers instead of entire applications. The cloud provider automatically executes the function when the request arrives and destroys it when the request is complete. And the most critical "F" in "FaaS", that is, how is the function defined? You can refer to the following definition of CloudFunction by Alex, the initiator of the OpenFaaS project:
2. Key technologies of FaaS
Let's briefly discuss some key technical solutions that have to be faced in FaaS:
(1) Elastic scaling (K8s as the orchestration base)
Auto-scaling is a key part of function computing cost optimization. The main target of auto-scaling is function instances (some FaaS architectures also include MQ trigger instances that are started together). The main process of elastic scaling includes: indicator sensing (triggering elastic scaling) -> scheduling scaling policy calculation -> final scaling to take effect , the above steps are very similar to k8s HPA, but the implementation mechanism has changed for the characteristics of function computing, although In this way, the current capabilities of k8s can also effectively meet the elastic scaling requirements of FaaS. Therefore, Serverless/FaaS products in the open source community are basically implemented based on k8s. /FaaS product development.
The following is the architecture diagram of the open source project OpenFaaS based on k8s:
Regarding elastic scaling , in addition to Kubernetes' HPA, there are events-driven architecture elastic framework KEDA and KPA (Knative implementation) in the community. KEDA, the core problem it solves is the elastic scaling under the event-driven architecture (event-triggered scaling) and the problem of instances from 0 -> 1, 1 -> 0. KEDA works as a Kubernetes Metrics Server, allowing users to use dedicated Kubernetes custom resource definitions to define autoscaling rules.
KEDA can run on the cloud and edge, and can be natively integrated with Kubernetes components (such as Horizontal Pod Autoscaler) and has no external dependencies. , the overall structure of KEDA is as follows:
(2) Function Runtime & Function Loading Mechanism
The FaaS runtime should provide resources and a security-isolated language runtime environment for function execution, and deliver call events, context information, and response information to functions. The function runtime should have a unified interface specification to support multi-language access, and a dedicated monitoring and management component (similar to the Agent started with the function instance) to maintain the life cycle of the function instance.
Taking Lambda of AWS, a mature FaaS platform in the industry, as an example , its function runtime architecture is similar to that shown in the figure below: the dotted box on the right is a function instance , including a RuntimeAgent (that is, API Endpoints) for processing functions Monitor and manage the function process during the calling process, and collect execution data and upload it to the HostAgent (Lambda Service) outside the function instance for analysis and recording; Processes includes the function process that executes the user's business logic and the Runtime environment provided by the FaaS platform.
Lambda supports multiple languages through Runtime and Runtime API . Runtime API and Runtime are called through API specifications, and user functions only need to implement specific interface functions of Runtime to complete operations such as initialization, loading, and calling. Additionally, the Runtime provides a language-specific environment for passing invocation events, context information, and responses between Lambdas and functions. Users can use the runtime provided by the Lambda platform, or build their own runtime.
Each major programming language version has a separate runtime with a unique runtime identifier, such as python3.10 or nodejs18.x. To configure a function to use a new major language version, the runtime identifier needs to be changed. Since AWS Lambda cannot guarantee backward compatibility between major releases, this is a customer discretionary operation. For functions defined as container images, you can choose the runtime and Linux distribution
when creating the container image . To change the runtime, the function's configuration needs to be updated and a new container image created. The runtime is paired with an Amazon Linux distribution. The underlying execution environment provides additional libraries and environment variables that can be accessed from function code. In the k8s cluster, the entire Execution Environment function instance can run as a Pod in a specific cluster to interact with other k8s components or user-defined components. From the above, we can see that Lambda implements function loading by uniformly packaging user functions and mirroring them at runtime. In addition, user function code loading can also be implemented by mounting directories or sharing memory, which will not be expanded here.
(3) Cold start
Cold start refers to the preparation phase of a user function instance from scheduling to startup to being able to provide services. Let's take a look at the steps involved in starting a mirror container containing user functions and function runtimes in a traditional k8s cluster:
- To apply for scheduling function resources, kube-scheduler specifies scheduling nodes based on demand and existing resource calculations.
- The kubelet process of the specified node detects that the pod is dispatched to the node, and when calling the container runtime, it pulls the image from the registry.
- Pull up the container when the low-level container is running, register the k8s endpoint and perform a health check (every 1s)
- The health check passes, and the container is successfully initialized in the cluster.
- (Inside the container: function runtime, function instance, RuntimeAgent, etc. complete initialization and function registration)
The image pull time is uncontrollable, ranging from a few seconds to tens of seconds or even minutes. When the function code is included in the image, due to the heterogeneity of the function image and the existence of a large number of Function mirroring makes it difficult to reuse. Secondly, it usually takes several seconds for kubelet to create a container. After the container is started, the startup time of the function process inside the container (including runtime environment startup, dependency preparation, etc.) also takes time.
To sum up, function cold start delay is a key issue in FaaS. Currently, the industry has the following methods for cold start optimization:
-
For the delay of image pull, you can adopt the method of image code separation. The image itself is layered, so you can separate the user function code from the function runtime, RuntimeAgent and other environment dependent layers, and the node will advance the part of the content that does not change Pull ahead to save time.
-
In view of the time loss caused by starting the container, the function runtime, RuntimeAgent and other environments inside the container, and component startup after the container is started, the delay can be reduced by preheating the function instance, that is, maintaining a cold start resource pool . The FaaS platform can pre-maintain a batch of basic containers that do not contain business functions to form a cold start resource pool. After the cold start process, it is bound to a user function instance, and can be pulled through **shared storage volume** or directly online Load the user function instance by fetching code , etc., and start providing services. The capacity, expansion and contraction of the cold start pool are uniformly monitored and managed by the FaaS platform. (Or use a more lightweight and efficient container runtime: one is a lightweight virtualization technology based on KVM, such as Firecracker; the other is Wasm. AWS's Lambda underlying technology uses Firecracker, and its cold start time is ms level)
-
For health checks, OpenFaaS has tried to reduce latency by using RuntimeAgent (watchdog) to accept and process external traffic in advance before K8s health checks. However, this method has many problems and is not recommended for now.
-
For the time-consuming start-up of user business function process instances inside the container, you can refer to the method of Fork a Process Per Request in the container while maintaining a cache function instance in OpenFaaS.
In addition, general FaaS requires that the function instance instance traffic be reduced to 0 when the flow rate is 0 (K8s native HPA cannot do this temporarily, but KEDA can), but some faas platforms do not enforce restrictions on this , For example, the OpenFaaS open source version runs a function instance by default to carry the initial traffic, and some other platforms adopt the method of maintaining a cold start resource pool to avoid the problem of too long initialization.
(4) How to call/trigger the function
-
Based on Web function architecture : its essence is to call functions through HTTP. The core here is that the Runtime environment of the function instance needs to provide a Web framework and Web Server, and the user function is called after the request enters the Runtime. Each function configures an HTTP trigger by default and assigns it a fixed independent domain name. As long as the domain name is requested, it can be executed as an HTTP trigger function. The function can process the request as long as it implements the corresponding interface, and finally through the API gateway provided by the FaaS platform Centralized processing of function instance interface requests.
-
Event-driven architecture : Both community and public cloud products have the concept of triggers. The so-called trigger is actually the event source. The event source can be received by a unified sidecar container and then invoke the function by HTTP; it can also be directly received by each function Runtime and then invoke the function.
-
Based on message queue MQ: MQ consumption is a common business type in business scenarios. Whether it is offline data calculation or online real-time messages, MQ will be selected as middleware to asynchronously process and decouple business. Therefore, MQ message The queue can be used as the largest traffic entrance of the entire FaaS platform.
-
User-defined triggering method: For example, a component called Connector of OpenFaaS allows developers to customize the triggering method of functions.
(5) Others
The following are some technical points related to FaaS that I personally compiled, which may not be very complete, and are for reference only.
3. Cloud native FaaS framework
(1) OpenFunction
OpenFunction is a modern cloud-native FaaS (Function as a Service) framework, which introduces many excellent open source technology stacks, including Knative, Tekton, Shipwright, Dapr, KEDA, etc. The possibilities are endless:
- Shipwright allows users to freely choose and switch tools for image construction during the function construction process, and abstracts them to provide a unified API;
- Knative provides an excellent synchronous function runtime with powerful auto-scaling capabilities;
- KEDA can automatically scale based on more types of indicators and is more flexible;
- Dapr can abstract the general capabilities of different applications and reduce the workload of developing distributed applications.
(2) OpenFaaS
OpenFaaS is a long-developed FaaS open source project that has gained more attention on GitHub. It has made more explorations in the mode of processing requests when CloudFunction is running. In the third section of this article, we will focus on the introduction of FaaS practice.
(3) Knative：Home - Knative
Knative is an open source, Kubernetes-based platform designed to simplify and accelerate the process of developing, deploying, and managing containerized applications. It provides a set of building blocks and tools to help developers deploy modern, cloud-native applications to Kubernetes clusters, implementing serverless architecture (Serverless) and automated container orchestration.
3. FaaS practice - OpenFaaS
1. Overview
(1) what is
OpenFaaS is an open source FaaS framework that allows users to deploy and run their own functions on it. OpenFaaS uses Docker containers as the underlying technology and can run on any cloud environment or private cloud. The OpenFaaS project aims to turn low-level infrastructure such as Kubernetes clusters or standalone virtual machines into a high-level platform for managing serverless functions.
(2) What did you do
- Function calling : WatchDog is the key implementation of OpenFaaS for calling and monitoring Cloud Function, which acts as a reverse proxy for running functions and microservices. It can be used independently, or as an EntryPoint for OpenFaaS containers, and started in a function container as an " Init Process ". It has a built-in HttpServer implemented in Golang, which provides functions such as concurrent requests, timeouts, and health checks. It allows developers to customize its functionality by loading various environment variables.
- Automatic scaling: OpenFaaS uses the self-designed AutoScaler to automatically scale functions.
- Cold start: OpenFaaS has a focus in architecture design. OpenFaaS does not maintain a cold start pool and then load business code on demand. It hopes to maintain a read-only file system. The container carrier is designed as an immutable unit to obtain predictability life cycle and obtain maximum security. Therefore, OpenFaaS does not reduce the number of function instances to zero by default when there is no traffic. It is only enabled as an optional configuration for the function granularity. The overall cold start data traffic is expected to be carried by the smallest instance to avoid pulling up a The resource scheduling, binary pull, process startup, and health check required by the new instance generate huge cold start overhead.
(3) How to use
OpenFaaS is deployed on native K8s through helm
2. Key technical components
OpenFaaS technology overview diagram:
The above figure is the abstract service process of OpenFaaS, and the following is a brief introduction of each node:
Gateway : HTTP gateway for receiving user requests and internal instructions.
NATS Streaming: Used to execute functions asynchronously.
Prometheus / AlertManager: Used to collect service indicators and scale operations.
faas -netes: Provider for K8S, other Providers such as Docker Swarm can be customized.
Docker Registry: A repository for pulling function images.
（1）WatchDog
WatchDog is the key implementation of OpenFaaS for invoking and monitoring Cloud Function. The latest of-watchdog acts as a reverse proxy for running functions or microservices . It can be used stand-alone or as an EntryPoint for an OpenFaaS container and started in a function container as an " Init Process ". It has a built-in HttpServer implemented in Golang, which provides functions such as concurrent requests, timeouts, and health checks. It allows developers to customize its functionality by loading various environment variables. OpenFaaS has launched two Watchdog modes, ClassicWatchdog and of-Watchdog. The following are introduced respectively:
- Classic WatchDog working mode:
In this mode, watchdog starts a lightweight HTTP server listening on port 8080 , and each incoming request will:
- Read request header and request body
- fork or exec the executable containing the actual function
- Write the request header and request body to the stdin of the function process
- Wait for the exit of the function process (or timeout)
- Read stdout and stderr of the function process
- Send the unread bytes back to the caller in the HTTP response
The above logic is similar to the traditional Common Gateway Interface (CGI) . On the one hand, starting a separate process for each function call seems inefficient, on the other hand, it is really convenient, because any program (including CLI tools) that uses stdio streams for I/O processing can be deployed as an OpenFaaS function .
When it comes to isolation , we need to distinguish between functions and calls :
- Different functions in OpenFaaS are always distributed in different containers
- A function can have one or more containers - depending on scaling options
- Independent calls to the same function may end up in the same container
- Independent calls of the same function will always be made using different processes
- Reverse proxy Watchdog working mode:
If the classic runtime is similar to CGI, then this runtime mode is similar to the later FastCGI . The runtime expects to have a long-running HTTP server behind the watchdog, rather than spawning a new process each time the function is called. This essentially turns the watchdog component into a reverse proxy : when the container starts, the reverse proxy watchdog also creates a lightweight HTTP server listening on port 8080 . However, unlike the classic watchdog, the reverse proxy watchdog only creates the process of the function once and treats it as a (long-running) upstream server. The function call then turns into an HTTP request to that upstream.
However, the reverse proxy mode is not intended to replace the classic mode. The strength of the classic mode is that its functions are very simple to write. This is also the only option for code without an HTTP server. For example, functions written in Cobol, bash, or PowerShell scripts, etc.
When to use reverse proxy runtime mode:
- The function needs to maintain state between calls:
- cache
- Persistent connections (e.g. keep a connection open from a function to a database)
- stateful function
- Starting a process per function can be expensive, introducing latency for each call
- You want to run a (micro)service as a function
According to Alex Ellis , the creator of OpenFaaS , FaaS , and OpenFaaS in particular, can be seen as a simplified way of deploying microservices without relying on server abstractions . That is, FaaS is the canonical example of a serverless architecture.
Thus, functions can be seen as an opinionated way of deploying microservices using a reverse proxy approach. Convenient, fast and easy. But when using stateful functions, be aware of warnings due to the possibility that multiple calls may end up in the same process:
- Concurrent calls that end up in one process can trigger race conditions in your code (for example, a Go function with a global variable that is not protected by a lock).
- Subsequent calls that end up in one process can lead to cross-call data leaks (just like traditional microservices, of course).
- Since the process is reused between calls, any memory leaks in the code will not be deallocated.
Comparison of working modes between wathdog and reverse proxy watchdog:
（2）APIGateway
Openfaas gateway provides external routing of RESTful interface for custom functions, built-in UI, faas-cli and API requests will be processed and distributed by Openfaas gateway.
The Openfaas gateway also integrates Prometheus to provide monitoring capabilities for functions and services, and can also call faas-provider APIs to manage containers.
Faas-provider is an SDK toolset implemented in go language that complies with the Openfaas provider Http REST API standard.
The main responsibilities of OpenFaaS API Gateway:
- CRUD operations on functions
- proxy function call service
- Manage the scaling of function containers
- Container key management
- Log drainage
(3)FaaS-Provider interface / phase-nets
Reference reading: The power of interfaces in OpenFaaS (alexellis.io)
（3）Auto-Scaling
Reference reading: Autoscaling - OpenFaaS , Scale to Zero and Back Again with OpenFaaS
(4) NATS-based asynchronous function call
The following two figures briefly illustrate the calling process of synchronous/asynchronous calls to upload a PDF file function, and illustrate how to implement asynchronous function calls through NATS.
3. Others
(1) OpenFaaS developer perspective
Further reading: