- What is vSan:
vsan is a software-based distributed storage solution, which is built in the hypervisior and supports all the features of VMware vsphere that require shared storage.
- Why use vSan:
In an enterprise, the most important thing is the data, and the device that carries the data is the storage device. The storage device of the enterprise is generally composed of multiple hard disks and RAID cards. Through the RAID card, multiple disks can be formed into a logical array, making the data scattered. Save in multiple disks to achieve efficient read and write, achieve redundancy, and avoid data loss caused by a single disk failure.
Among them, there are two common configuration modes of RAID: RAID1 and RAID5.
The security performance of RAID1 is the highest among all RAID configurations, which can ensure that data will not be lost. The read and write performance is no different from that of a single disk, but its disk utilization is very low, only 50%.
The read performance of RAID5 is the best, and the write performance is lower than that of a single disk, which is suitable for the environment with more reads and less writes. Security performance RAID5 is inferior to RAID1, but higher than other configurations. The utilization rate of the disk is higher than that of RAID1, and the cost is also lower than that of RAID1.
Therefore, RAID1 is suitable for storing important data, and RAID5 is a solution that balances storage performance, data security, and storage cost. These two configuration methods are still used by many companies.
But now there are many problems exposed in this configuration method:
(1) Unable to smoothly upgrade capacity and performance according to business needs
(2) The configuration is complicated, the learning cost is high, and it is prone to misoperation.
(3) Inability to differentiate services
(4) The reading and writing speed is not high enough
In order to solve the above problems, the best existing solution is vSan, which is the software-defined data center.
- vSan：
- (1) Connections inside the server
The architecture of the X86 server determines that each server must have a RAID card to use the hard disk. Multiple hard disks are aggregated using the RAID card and then connected to the main board.
In each vSAN server, at least one SSD + one HDD needs to be configured, HDD
is used for storage capacity, and SSD is only used for read and write cache.
For RAID card selection, it is recommended to use a RAID card that supports Pass Through mode . It is very convenient to replace hard disks and add new hard disks in this mode (and if you configure RAID for vSAN, you often need to restart the host to enter the BIOS to configure).
70% of SSD capacity is used for read cache and 30% for write cache. By default, all written data will be first placed in SSD to reduce write delay, and these data will be gradually written to HDD through some mechanisms. When reading data from vSAN, vSAN has a set of algorithms to determine which data is hot data, and then pre-cache it to SSD to speed up the reading speed.
Therefore, it is generally not recommended to use the read-write cache that comes with the RAID card, because vSAN already has a very optimized read-write cache acceleration.
However, special attention should be paid here that
vSAN requires SSD, HDD, and IO Controller to be in the vSAN compatibility list, otherwise instability and other situations will occur.
(2) Connection between servers
There are two configurations of vsan, and the two require different connection configurations.
All-flash architecture vsan:
In the all-flash architecture, flash memory devices are divided into two categories: one is a persistent and fast flash memory device for writing cache, and the other is a large and relatively cheap flash memory device as a capacity device. In this architecture, 100% of the capacity of the Cache layer is used for writing the cache because reading data directly from the flash memory device in the capacity layer will have better performance. However, in order to prolong the service life of the capacity layer flash memory device as much as possible, most of the writes that need to be written will be placed in the cache layer, and only when the data needs to be written to the capacity layer will the data be written into the capacity layer flash memory device.
This architecture has better performance and can provide more reliable and powerful performance for various workloads, but it also has higher requirements on the network. It requires a dedicated 10-gigabit network and does not support Gigabit networks.
Hybrid architecture vSan:
The cache algorithm will maximize the read and write performance of the entire vSAN cluster. 70% of the available cache will be allocated to store disk blocks with frequent read operations to minimize read operations on low-speed ordinary disks. The other 30% of the available cache will be allocated for write operations. In order to maximize disk performance, multiple write operations will be combined as much as possible before being written to disk.
This architecture is more cost-effective, cheaper than all-flash memory, and its performance is relatively good. The requirements for the network will also be lower, and a dedicated Gigabit network is sufficient.
(3) Fault domain:
Host:
vSphere provides the HA function to ensure that the business can run on other hosts after a single host fails. During maintenance, all storage resources that a host can provide will go offline).
The same data of the same virtual machine must be saved on different hosts.
Combining the above architecture with the network architecture of vSAN, what if there is a problem with the heartbeat detection of a host? (split brain)
At this time, an arbitration mechanism is needed to ensure that only one piece of data is active and up-to-date at the same time, otherwise conflicts will occur.
Therefore, in the architecture in the above figure, an arbitration file is created for each piece of data and stored in the third host.
This is the simplest architecture of vSAN. This architecture allows a host to fail. Of course, any hardware failure on a host is also allowed, as long as the failure occurs within a host.
The diagram below is a simple representation of vSAN fault domains. There is a word in vSAN called FTT (Fault
to Tolerance), which means the maximum number of hosts allowed to fail at the same time. FTT determines the data protection level of the virtual machine, and also determines the minimum number required for a cluster. The number of hosts in a cluster is >=2N+1, and N=FTT value.
- Why FTT=2, need more than 5 hosts
Someone once questioned: "Please explain the three copies of vSAN (FTT=2, that is, a maximum of 2 hosts are allowed to fail), why do we need more than 5 hosts? We need more than official requirements (in the official document, the minimum number of hosts allowed =2n+1, n represents the maximum number of hosts that can fail), and you need to know the specific design principles."
In fact, this is mainly done to prevent split-brain. Assuming a split-brain split into two disconnected sub-clusters, if a sub-cluster needs to take over data services, the number of hosts in this sub-cluster must be greater than 50% of the number of hosts in the original cluster. This also shows that the minimum number of hosts must be an odd number.
In layman's terms, four hosts cannot handle two hosts failing, because split brains may occur. For example, there are four hosts A, B, C, and D. If there is a network partition, A and B cannot contact C and D. At this time, it seems to the outside world that only two hosts have failed, and reading and writing must be allowed to continue according to the requirements of FTT=2. There are two hosts on both sides, so both sides feel that they can read and write, which will cause data inconsistency on both sides (commonly known as split brain). The solution to split-brain is to require that the number of hosts in the sub-cluster that can take over the reading and writing is greater than 50% of the total number of the original cluster. The total number of cluster hosts is an odd number, so it is impossible for two subclusters to meet the conditions.
- Why FTT=3, need more than 7 hosts
The user asked again: "If a maximum of 3 hosts are allowed to fail, then 3+1 hosts are required as replica nodes, plus 1 host as a witness (that is, quorum) node, and a total of 5 hosts is enough. Why? Need 7 hosts?"
Then I fell into a long thought, which made me think why it can't be 2n-1, or 2n-3. I think it can be summed up as follows: Assuming that n means that at most n hosts are allowed to fail, if n+1 hosts all store copy data, this can only ensure that data will not be lost. For enterprise-level storage, no data loss is the minimum. In addition, it is necessary to ensure continuous access to data, that is, HA.
Disk:
It is mentioned above that each host needs a RAID card to connect multiple hard disks, and also needs at least one SSD and one HDD. SSDs only do read-write caching. From an economic point of view, it is impossible to configure an SSD for each HDD, and multiple HDDs need to share the resources of an SSD.
In vSAN, there is the concept of disk group, which is a logical group used to allow multiple HDDs to share one SSD.
vSAN stipulates that each disk group needs at least one SSD+one HDD, and at most one+7 HDDs. No more than 5 disk groups per host
Sharing one SSD with multiple HDDs is cost-effective, but it also has certain risks. For example, in case of SSD failure, the data in the entire disk group will be inaccessible. Therefore, it is generally recommended to use multiple disk groups to disperse data and reduce such failures. the impact.
For example, disk group 1 can be composed of one 400G SSD + eight 800G HHD
Disk group 2 can be composed of one 200G SSD + four 800G HHD
If the SSD in disk group 1 fails, vSan will lose 3.2T capacity
If the SSD in disk group 2 fails, Vsan will lose 1.6T capacity
- (4) differentiated services
For traditional storage, service differentiation is at the storage volume level. The bottom layer of a storage volume uses RAID 10, and the storage resources obtained by upper-layer services are the protection level and performance of RAID 10; the bottom layer of a storage volume uses RAID 5, and the storage resources obtained by upper-layer services are the protection level and performance of RAID 5.
vSan allows you to set different protection levels for different virtual machines to protect virtual machines.
If the service running on a virtual machine is not so important, we can set the storage policy for it as follows:
FTT=1，不预留缓存，限制IOPS为100
And another virtual machine is running a very important service, and it needs to ensure that its service is uninterrupted and data is not lost. We can set the storage policy as follows:
FTT=2，预留10%的SSD缓存，不限制IOPS
In this way, we divide resources more accurately, reduce resources for unimportant services, and increase resources for more important services, so that they can obtain better services.
- vCentr6.7 installation vSan cluster requirements
Reference article
https://new.qq.com/omn/20180717/20180717G0AUBL.html
https://blog.csdn.net/yjk13703623757/article/details/80786090
https://blog.csdn.net/weixin_33810006/article/ details/92143861