Abstract: For start-up and growth-oriented enterprises, offline computing is indispensable. Through offline computing, we can generate complex business reports, and through offline computing, we can also accurately calculate user portraits. Offline computing has become an indispensable presence in today's enterprises. So what benefits can the use of elastic computing bring to the field of offline computing? This article will introduce how to use elastic computing to save the cost of offline computing for enterprises.
background
For the current entrepreneurial and growing enterprises, offline computing is indispensable. Through offline computing, we can generate complex business reports, and through offline computing, we can also accurately calculate user portraits. Offline computing has become an indispensable presence in today's enterprises. So what benefits can the use of elastic computing bring to the field of offline computing? The editor tells you that there are definitely benefits, and listen to the editor.
Common Offline Computing Architectures
From the above figure, we use the most common offline computing open source product Hadoop as an example. As shown in the above figure, you will put storage services and computing on one ECS. However, with the expansion of the business, we have an increasing demand for computing, and we need to use data to perform various operations. In order to enhance the computing power of the offline cluster, we will think of directly adding computing nodes to obtain it. But with the addition of computing nodes, we will encounter another problem, that is, the problem of insufficient resource utilization. Although computing can increase computing power through capacity expansion, data does not increase suddenly like computing, so for storage, the capacity expansion operation will temporarily lead to a decrease in resource utilization. In addition, the daily computing time for offline computing is only for a certain period of time. After the expansion, the ECS will be idle a lot after the offline computing is completed every day. This period of time will be a great waste of cost! Then many customers will wonder if there is a way to expand the computing storage separately, expand the computing nodes during offline computing, and release the expanded computing nodes after the calculation is completed, so as to save costs. This architecture is definitely there, but in order to let you better understand the next architecture, Xiaobian will first introduce you to bidding instances and elastic scaling.
A brief introduction to bidding examples
From the above figure, you can see that a spot instance is a postpaid instance whose price fluctuates according to the supply and demand relationship, and has a lower discount compared to the price of a pay-as-you-go instance. However, at the same time, the auction instance may be released by Alibaba Cloud at any time. Please pay attention to this point. In short, Spot Instances are cheap, but may be released at any time.
A brief introduction to elastic scaling
Elastic scaling has the following three advantages:
Improve fault tolerance
Auto Scaling will regularly check the health status of the ECS. If the ECS is found to be unhealthy, the ECS will create a new ECS to replace it and release the unhealthy ECS.
Enhanced usability
Elastic scaling can ensure that the application always has a suitable capacity to meet the current request traffic through timing and automatic scaling.
Optimize costs
Elastic computing automatically and dynamically increases or decreases instances on demand, adding instances when needed and releasing instances when not needed, thereby saving IT costs.
Offline computing architecture after separation of storage and computing
After a brief introduction to bidding examples and elastic scaling above, I can now tell you loudly that there is an elastic computing-based architecture that can not only meet users' computing power requirements for massive data, but also reduce your computing power. cost.
- Separate Hadoop storage and computing nodes
- Use elastic scaling to create and release computing nodes on time and on demand
- The payment type of the computing node adopts the bidding method
I believe that everyone must have doubts here, won't the auction instance be released by Alibaba Cloud, and is it appropriate to use the auction instance here? The answer is absolutely appropriate. First, the price variable of the Spot Instance is much cheaper to pay. Second, even if the Spot Instance is released by Alibaba Cloud, it will only affect the speed of our offline computing. In the above architecture diagram, the editor also left an ECS without joining the scaling group. For this ECS, we can use prepaid methods to provide our computing resources with "guaranteed" computing power. Of course, we can also according to our own needs. to set the number of "guaranteed" ECSs.
Tips for Building Offline Computing Architecture Using Spot Instance + Elastic Scaling
Spot Instance bid strategy
View the historical price of Spot Instance Multi-AZ on the sale page of Alibaba Cloud ECS, so as to choose a reasonable bid.
AutoScaling scaling group configuration
- Do not trigger scheduled capacity expansion tasks on the hour. There are many customers who expand capacity on the hour. You can choose to expand 5-10 minutes after the hour, so the price will be relatively lower.
- Trigger alarm tasks by monitoring CPU/MEM metrics of compute nodes
- Increase the probability of successfully purchasing Spot Instances by selecting Multi-AZ in the scaling group
- Configure the maximum number of ECSs in a scaling group to prevent the number of ECSs created by auto scaling from exceeding expectations
- Create a scaling task for manual scaling in case of emergency
Using Spot Instance + AutoScaling Cost
To read more good articles, please scan the following QR code: