Involved! The difference between DAS, NAS, SAN and FC SAN storage
Storage classification
The difference between NAS, SAN and DAS
Direct Attached Storage
1. Directly connect to the server through IDE, SCSI and optical fiber (FC) interfaces, with the server as the center
2. Each server needs an independent storage device (disk), and the connection distance is short, and the number of connections is limited
3. Data is scattered, sharing and management are difficult
4. High unit cost-waste of storage space
Network Attached Storage (NAS)
1. File-level shared access based on NFS and CIFS, supports HTTP
2. The storage device is functionally independent of the main server in the network and does not occupy server resources
3. Easier to expand, extensive support for operating systems and applications, simple and convenient installation
4. Data backup and recovery occupy network bandwidth
Storage Area Networks
1. Isolate storage and server, simplify storage management, and manage various resources in a unified and centralized manner
2. SAN can shield the hardware of the system and can use storage devices from different manufacturers at the same time
3. This method can reduce file redundancy.
4. Cross-platform performance is not as good as NAS, and the price is high. Building a SAN is much more complicated than installing NAS on the back end of the server.
FC SAN: Using Fibre Channel and other special storage protocols, connecting devices such as Fibre Channel switches to connect the network server and various storage devices in a high-speed dedicated network independent of Ethernet. The data is accessed in blocks using the FCP protocol. Does not occupy the network bandwidth of the server's computing processing
IP SAN: Use Ethernet instead of Fibre Channel (Fibre Channel) network and use iSCSI protocol instead of FCP protocol, SAN for block data storage access.
FC SAN and IP SAN must have storage management software (such as volume management, data snapshot, mirroring, backup, recovery, archiving, etc.).
The difference between SAN and NAS
NAS and SAN differ in the following aspects:
1. SAN is data-centric, while NAS is network-centric
2. SAN has the advantage of high-bandwidth block data transmission, while NAS is more suitable for data access at the file system level
3. Users can deploy SAN to run key applications, such as databases, backups, etc., for centralized access and management of data, and NAS supports file sharing between several clients or between servers and clients, so users can use NAS As a place where small files need to be exchanged frequently in daily office, such as file server, web page storage, etc.
The connection between SAN and NAS
NAS and SAN provide complementarity in the following aspects:
1. NAS products can be placed in a specific SAN network to provide excellent file transfer
Performance
2. SAN can be expanded to include IP and other non-storage-related network protocols
to sum up
FC SAN and IP SAN architecture
In the early SAN storage system, the data transmission between the server and the switch was carried out through optical fiber. Because the server transmits SCSI commands to the storage device, it cannot use the IP protocol of the ordinary LAN network, so it needs to use FC transmission, so this kind of SAN It's called FC-SAN.
In the later period, a SAN encapsulated with IP protocol appeared, which can completely take the common LAN network, so it is called IP-SAN. The most typical one is the popular iSCSI.
Comparison of FC SAN and IP SAN
In the IP SAN, the Ethernet switch replaces the expensive and only FC SAN dedicated fiber optic switch, and its port rate can be unlimited with the development of Ethernet technology.
In IP SAN, the client's Initiator or iSCSI card replaces the more expensive host HBA card
In IP SAN, cost-effective storage devices with iSCSI interfaces replace fiber-optic disk arrays
FC overview
FiberChannel is abbreviated as FC (FibreChannel), which is a high-speed network technology standard (T11), mainly used in storage networks.
FibreChannel (FC) technical standard is a network technology suitable for gigabit data transmission and communication formulated by the ANSI standardization organization in 1994. Fibre Channel is used for the connection of shared storage devices in servers, and the internal connection between storage controllers and drives.
From the perspective of the layered protocol stack, FC only includes specifications from the physical layer to the transport layer. Its upper layer defines interfaces that encapsulate other protocols as application layer protocols, such as SCSI or IP protocols. After encapsulating SCSI, the entire protocol is FCP (FCProtocol).
The FC physical layer has a very high transmission bandwidth, from 1Gb/s, 2Gb/s, 4Gb/s to 8Gb/s, 16Gb/s, using NMb encoding and synchronous serial transmission.
FC protocol stack
Comparison between FC and OSI reference models
FCSANFC-0
As a high-speed network transmission technology, the physical layer of the FC protocol system has a relatively high speed, from 1Gb/s, 2Gb/s, 4Gb/s to the current 8Gb/s. As a representative of high-speed networks, its bottom layer also uses synchronous serial transmission, and in order to ensure the characteristics of electrical and DC balance, clock recovery, and error correction during transmission, the transmission coding method adopts the NMb coding method. In order to achieve long-distance transmission, the transmission medium must at least support optical fiber. Copper wires are also available, but the distance is limited. The electrical sublayer of the physical layer in the FC protocol set is FC-0.
FC SANFC-1
Modern communication is generally framed at the link layer, that is, a certain number of bit streams sent from the upper layer are packaged and transmitted at the beginning and end. The FC protocol is also framed at the link layer. Since framing is required, frame control characters must be defined. The FC protocol defines a series of frame control strategies and corresponding characters. These control characters are not those defined in the ASCII code character set, but a separate set of character sets specifically used for the FC protocol, called "ordered set". Each of the control characters is actually composed of 4 8-bit bytes, called a "word" (word), and the beginning of each control word is always 0011111010 (left-handed) encoded by 810b Or 1100000101 (right-handed).
The FC protocol gave this character a name, called K28.5. The value of this word before 810b encoding is hexadecimal BC, that is, 10111100, its low 5 digits are 11100 (decimal 28), and the high 3 digits are 101 (decimal 5). The FC protocol represents this word as "K28.5", that is to say, the high three digits in decimal is 5, and the low 5 digits in decimal is 28, so that it can be combined into the corresponding binary code. Then add a description symbol K (control character) or D (data character). The K28.5 character does not conflict with the ASCII character encoding. Its binary stream contains five consecutive ones, which is very easy to be recognized by the circuit. Of course, there are several characters that meet these conditions. Each control word starts with the K28.5 character, followed by 3 other characters (which can be data characters). The word composed of these 4 characters represents a meaning, such as SOF (StartOfFrame), EOF (EndOfFrame), etc. .
The FC protocol defines a 24B frame header. The Ethernet frame header is only 14B, which is more than enough to use. Why does FC need to define 24B? On this issue, the designer of the protocol is unique, because the 24B frame header not only includes the addressing function, but also includes the transmission guarantee function. The logic of the network layer and the transport layer use this 24B information to transmit. We know that the total cost of an Ethernet-based TCP/IP network is: 14B (Ethernet frame header) + 20B (IP header) + 20B (TCP header) = 54B, or replace the TCP header with an 8B UDP header, total It's 42B. This is destined for FC's overhead to be smaller than that of Ethernet plus TCP/IP, and the realized functions are similar.
FC frame format
FC topology FC-2
Fibre Channel has three topologies:
1. Point-to-Point-interconnection between two devices
2. Arbitrated Loop-Supports up to 126 (using one byte addressing capacity) device interconnection to form an arbitrated loop
3. Switch Fabric (Switch Fabric)-up to 2 to the 24th power of equipment interconnection
Point-to-Point
• Dedicated connection between'N' port Fibre Channel devices
• All link bandwidth is allocated for communication between two nodes
• A solution suitable for small-scale storage devices without sharing function
Arbitration Ring (FC-AL)
The TX port of each node is connected to the RX port of the neighboring node until a closed loop is formed
•Maximum bandwidth: 800MB/sec (shared among all nodes on the loop)
• Up to 126 nodes on the loop
• Not a token transmission scheme-does not limit the time the device retains control
•Operation sequence:
Loop control arbitration
Open the channel to the target device
Transfer data
shut down
• The number of nodes on the loop directly affects performance. When a node fails, this node will be bypassed
Switch (Fabric)
• 8Gb/16Gb bandwidth per port
• Adding new equipment can increase the total bandwidth
• Possible addresses up to the 24th power of 2
•Support zoning partition function
Protocol C—Network Layer: Addressing FC-2
Any network needs an addressing mechanism, FC is no exception, but the addressing and programming method of FC is different from that of Ethernet. For example, there is no need to have a MAC address on the port of the Ethernet switch, while the port on the FC switch Each has its own WWPN address. This is because in the FC network, the FC switch plays a very important role, and it has to deal with the uppermost layer of the FC protocol-the application layer. In other words, FC terminal equipment is only responsible for generating data, and other functions (packaging, flow control, security, etc.) are guaranteed by the FC switch. The following are the addressing types of FC:
• ——WWNN: No matter how many FC ports there are on this device, the FC device itself has a unique WWNN address to represent itself.
• —— WWPN: Each port of the FC device has a globally unique WWPN address, the length of the address is 64 bits, but it is unwise to use 8 bytes for routing, so another layer of address needs to be mapped .
•—— Fabric ID: A mapping relationship needs to be established between WWPN and Fabric ID. Just like the mapping between MAC and IP, the device will allocate a Fabric ID for each interface connected to the FC network, and use this ID to embed it in the link For routing in the frame, the ID is 24 bits long and the format is as follows:
Domain ID: The first 8 bits are the Domain ID, which is used to distinguish each FC switch in the network. The one with the smallest WWNN wins and becomes the master switch, and then this switch assigns Domain IDs to all other switches.
Area ID: The 8 bits are the Area ID, which is used to distinguish different port groups on the same switch. For example, ports 1, 2, 3, and 4 belong to Area 1.
Port ID: The last 8 bits are Port ID, used to distinguish different ports in the same area.
• Through this addressing system, each switch, each port group, and each port can be distinguished in an FC network
• The FC switch uses the Name Server (name service) to convert the WWPN number and FCID.
• The command to view the HBA card of the device on the Linux system is lsscsi -t -H, and you can see the WWPN and FCID information in it.
FC protocol-network layer: addressing (1)
• Now that two sets of system are defined: WWPN and FabricID, there must be a mapping mechanism, just like the ARP protocol, the address mapping steps in the FC protocol are as follows:
1. Registration: When an interface is connected to the FC network, if it is a Fabric architecture, then this interface will initiate an action to register to the Fabric network and send a login frame to the destination address FFFFFE (registration server), called FLOGIN.
2. Mapping: After the switch receives the frame with the destination address of FFFFFE, it will dynamically assign a 24-bit FabricID to this interface, record the WWPN corresponding to this interface, and do a good job of mapping.
3. Sending: All frames sent out from this interface will not carry the WWPN address, but will carry the assigned FabricID as the source address.
4. Arbitrated ring: When connected to the arbitrated ring network, all nodes will select a temporary node (the one with the smallest WWPN number wins), and then this node will send a series of initialization frames to assign a loop ID to each node.
FC protocol-network layer: addressing (2)
• Because FC was designed for dedicated, high-speed, and efficient network use from the beginning, in order to avoid human error, all operations in the FC network do not require manual intervention, and the equipment will automatically allocate and manage various addresses (WWPN), Automatically run and set the routing protocol (SPF shortest path first).
• When the device connected to the FC switch interface logs in to the FC network, it will send a registration frame to a specified ID (this ID is just a name service program running on the switch). After the device is registered, the name service program will The information of other nodes present on this interface tells the device connected to this interface.
• Addressing security issues:
—— Soft ZONE: Let the name server only tell a specific node of a device. For example, there are three nodes A, B, and C on the network, and the name service can only advertise node B to node A while hiding node C so that A cannot see C. But if A knows the ID of C, he can also directly visit d, which is the soft zone.
—— Hard ZONE: You can also divide A and B into one ZONE. This method isolates the underlying hardware, so that even if you know the ID, you can't communicate.
—— LUN Masking: There is a command in the SCSI command set called Report LUN, which is used by the initiator to issue this command. After receiving the command, the target must report its LUN information to the initiator. According to this principle, we can let the disk controller provide the corresponding LUN to the initiator according to the WWPN address of the initiator. For example: for host A, the controller reports LUN1 and LUN2, and host B reports LUN3. If a host forcibly accesses a LUN that does not belong to it, the disk array controller will reject the request. It can also be configured to selectively allocate a certain LUN to the designated front-end port of the disk array.
FC-LS link service-session management
• log in
FLOGI — Fabric Login
PLOGI — Node Port Login
PRLI — Process Login
• Log out of LOGO/PRLO
FLOGI — Fabric Login
• Determine if the switch exists
• Negotiate operating parameters, such as maximum frame length, BB_Credit
• Establish a dialogue with F_port
PLOGI — Port Login
• Establish a dialogue with N_port
• Negotiate service parameters, such as EE_Credits
• Create a dialogue between two N_ports
• Before PLOGL succeeds, there is no upper-level operation
PRLI — Process Login
• Optional
• Service parameters at the communication process level
General service
• FC-PH defines multiple addresses for special functions: the upper 16 addresses of the 24-bit address space
• Commonly recognized addresses
Name Server (Name Serevr)
• The well-known address of the name server is 0xFFFFFC
• N _port registers the information in the database of the name server
• N_port query the database to obtain information about other ports
• N_port can be deregistered from the name database
FC port type
• N port: Node Port node port; Fibre Channel communication terminal; host port, storage port, or fiber switch port with AG mode enabled
• NL port: Node Loop Port
• F port: Fabric Port fiber port; a switch connection port
• FL port: Fabric Loop Port optical fiber loop port; AL equipment provides a port to enter the optical fiber network service
• E port: Expansion Port expansion port; used to connect multiple switches through ISL (Internal Exchange Link)
• G/U port: Generic Port; can switch between F port and E port according to the connection mode
FC protocol-transport layer: packet structure (FC-4)
• The FC transport layer is also similar to TCP. It also segments the upper-layer data and also distinguishes the upper-layer programs. TCP uses ports to distinguish, and FC uses Exchange ID to distinguish.
• Each data packet sent by Exchange is divided into Information Units by the FC transport layer, which is equivalent to the segments divided into TCP. Then the FC transmission layer submits these Units to the lower layer of the FC for transmission. The lower layer treats each segment as a Sequence and gives a Sequence ID, and then divides this Sequence into FC suitable frames again, and assigns a Sequence Count to each frame, so that the order of the frames can be guaranteed. After the receiver receives the frame, it will be combined into a Sequence, and then submitted to the upper layer protocol for processing in order according to the Sequence ID.
• Another important role of the transport layer is to adapt the upper layer protocol, as shown in the following figure: For example, IP can be transmitted through FC, and SCSI commands can also be transmitted through FC. FC will provide interfaces that adapt to upper-layer protocols, namely IP over FC and SCSI over FC. FC only provides a channel and a transmission method for IP and SCSI, just like IP over Ethernet and IP over ATM.
• At the transport layer, FC defines several service types, which are similar to the TCP and UDP specified in the TCP/IP protocol. The specific types are as follows:
• ——Class 1: This is a connection-oriented service, similar to a circuit switching mode, which will reserve a virtual circuit for both parties of communication for reliable transmission.
• —— Class 2: It provides an end-to-end confirmed transmission service, similar to TCP.
• —— Class 3: This service type does not provide confirmation, similar to UDP.
• —— Class 4: This type reserves certain bandwidth resources on the link for upper-layer applications, but does not reserve the entire link. The working principle is similar to the RSVP service.
• In order to further improve the speed and efficiency of the FC network, most of the logic functions of the FC protocol are directly implemented in an independent adapter card instead of running in the operating system, because if the protocol logic is placed on the system to run, it will Occupy host CPU and memory resources, the following is a comparison between TCP/IP and FC protocols:
Operating in the operating system: IP and TCP/UDP modules are protocols that run on the operating system, and only the Ethernet logic runs in the Ethernet card chip, and the data received by the CPU from the Ethernet card carries the IP header and TCP The /UDP header requires the TCP/IP protocol code running in the CPU to further process these headers in order to generate the data required by the final application.
Running in the adapter card: most of the logic functions from the FC protocol physical layer to the transport layer run in the FC switch and FC adapter card chip, only a small part of the logic about the upper API runs in the operating system FC card driver , In this way, the speed and efficiency of the FC protocol are higher than that of the TCP/IP protocol.