First, the basics of computer and network
1.1, basic computer knowledge
Application software to the hardware operation must call the operating system interface, controlled by the operating system hardware
The client software wants to send a message to a server-based network software process:
1, the client software to generate data, the client software stored in the memory, and then sends the call interface to the operating system memory or copy data in their memory.
2, after the operating system receives data, rules (protocol) specified by the client software, call to send data card.
3, the data transmission network.
4, call server software system interface to the operating system sends the data from the operating system into memory copy own memory instructions.
5, the server 4 receives the instruction of the operating system, using the same client rule (protocol), the data received from the network card, a copy of the server software.
1.2 What is a network
+ = The physical network media links Internet Protocol
Internet Protocol is divided into seven layer OSI or TCP / IP protocol five
Two, TCP / IP layers Comments
2.1 Physical Layer
Function: high and low voltage transmission based on the electrical characteristics (electrical signal), a high voltage corresponding to the number 1, a low voltage corresponding to the digital 0.
2.2, the data link layer
Function: a simple electrical signals 0 and 1 does not make sense, therefore defines the grouping of electrical signals, namely Ethernet protocol.
2.1.1 Ethernet protocol (ethernet)
1, a predetermined set of electrical signals constitute a data packet, called 'frames'. Each frame consists of two parts 'header' and 'data'.
A header (18 bytes): sender / source address (6 bytes) + recipient / destination address (6 bytes) of data type + (6 bytes)
Data (46--1500 bytes)
2, mac address
That is the source and destination addresses
Each card are fired on a world unique mac address, a length of 48 binary factory, generally indicated by 12 hexadecimal
mac address (12) = 6 + 6 line vendor ID No.
2.1.2 Broadcast
Obtaining a host mac address arp by another host within the same network protocol, which then communicates with the broadcast.
2.3, the network layer
Function: the introduction of a new address to distinguish between different broadcast domain / subnet, i.e. the network address.
2.3.1, IP protocol
1, IP protocol defines the network address for the IP address, there are two versions of ipv4 and ipv6 to ipv4-based. Ipv4 IP address is represented by a 32-bit binary, usually written as four decimal numbers (decimal point). Range: 0.0.0.0 - 255.255.255.255
2, the core IP protocol is TCP / IP protocol, all TCP, UDP IP data is transmitted in a data format. Since the IP protocol does not provide data not communicate after a handling mechanism, it is not a reliable protocol.
. 3, the maximum length of the IP packet of 65535 bytes, 2 bytes as described in the data length of the IP datagram using. 16 described up to 2 to the power.
2.3.2, subnet mask
1 shows a sub-network characteristic parameter, the form is equivalent to an IP address, also 32-bit binary, are all part of a network, the host portion are all 0.
2, effect: combined with the IP address, the IP address identifies a network which bits represent bits, which bits represent the host bits.
2.3.3, IP address classification
1, A Class A addresses: the network address 1 byte + 3 byte host address. Ranging from 1.0.0.0 to 126.0.0.0, there are available a class A network 126, each network can accommodate more than 100 million hosts
2, Class B addresses: the network address 2 bytes + 2 bytes of the host address. Ranging from 128.0.0.0 to 191.255.255.255, Class B network has the available 16382, each network can accommodate a plurality of hosts 60,000
3, Class C addresses: the network address of 3 bytes + 2 bytes of the host address. Ranging from 192.0.0.0 to 223.255.255.255, available Class C network has more than a 209, 254 can accommodate each network host
4, Class D address: used for multicast, the address is a special reserved. Multicast group address is used to address one computer, it identifies a set of computer share the same protocol. Ranging from 224.0.0.0 to 239.255.255.255
5, E Class Address: reserved for future use, the range of 240.0.0.0 to 255.255.255.254
All-zero address (0.0.0.0) corresponding to the current host
All "1" address (255.255.255.255) is the current subnet broadcast address
Loopback address (127.0.0.1), also known as the host address, called loopback interface.
1, any data to the loopback address are input as IP
2, any data to the host IP address are sent to the loopback interface.
3, passed to a broadcast address or multicast IP packets will pass a copy of the loopback interface, and then sent to the Ethernet. This is because the definition of broadcast transmission and multicast transmission includes the host itself
2.3.4, ARP protocol
Functions: sending broadcast packets ways to get mac address of the target host
Way of working:
1, by IP address and subnet mask to distinguish which own subnet
2, analysis host and destination address is in the same network, if not, get the other side of the gateway mac.
3, the packet will be transmitted in a broadcast manner in which the end of the transmission within a subnet, the host receives all the disassembled packet, the destination IP is found on its own in response, return to their mac.
2.3.5, ICMP protocols
function: When the IP packet transmission errors, such as host unreachable, routes and other unreachable, ICMP error message packet protocol will transmit back to the host, the host to handle the chance of error.
There are tools through ICMP ping and traceroute.
1、ping
Whether using the ICMP protocol packets to detect up to another host. Principle is requesting ICMP type code 0, the host receives a request code with a ICMP echo type 8. ping program delivery interval and number of packets is calculated.
Where the user can generally determine the network
2、traceroute
View from the current host to an address after a total of how many routes.
2.4, the transport layer
Port: Applications and associated card number, in the range of 0 - 65535,0 --1023 is occupied by the system
Transport layer function: establishing a communication port to port by the TCP and UDP protocols
2.41, TCP protocol
. 1, the length of the TCP packet is not limited, and can be theoretically infinite, network efficiency is guaranteed units, usually not more than the length of the IP packet, to ensure that a single TCP packet is not divided.
2, reliable transmission: they do not get recognized on the re-send the data until it receives the acknowledgment.
3, when connected "three-way handshake" disconnect "fourth wave"
4, UDP protocol network overhead is relatively large
2.4.2, UDP protocol
1, unreliable delivery
2, UDP header of 8 bytes comes
3, a small network overhead.
Three, UDP packet size issues
1, the data link layer, and last bytes of the data frame 18 is removed, the maximum length (MTU) data portion of 1500 bytes.
2, the network layer, the IP packet header occupies 20 bytes, where the MTU is 1480 bytes.
3, the transport layer, UDP packet header occupies 8 bytes, where the MTU is 1472 bytes.
4, in the application layer, UDP data generated not more than 1472 bytes. When exceeding, the IP packet will be fragmented transmission, a data loss when a certain, UDP data will be discarded.
Four, socket module
the intermediate socket is an application abstraction layer and transport layers, is a set of interfaces. The complex TCP / IP protocol suite is hidden behind the socket interface. Invoking the interface, you can send messages to other machines.
4.1, socket socket common method
socket.socket(family=AF_INET, type=SOCK_DTREAM, proto=0, fileno=None)
4.1.1, socket type type
TCP-type type socket.SOCK_STREAM
type type UDP is socket.SOCK_DGRAM
4.1.2, the server socket functions
1, s.bind () bind (host, port number) to the socket
2, s.listen () to start listening TCP
3, s.accept () passive acceptance of client TCP connections (blocking) waiting for connection
4.1.3 client socket functions
1, s.connect () initialize TCP server connections active
2, s.connect_ex () returns an error code when the connect () function is an extended version of error, rather than throwing an exception
4.1.4, public use socket functions
1, s.recv () received data
2, s.send () to send data (data to be sent when the amount is larger than the remaining space hexyl side buffer, data is lost, is not finished)
3, s.sendall () complete TCP data transmission (when the cycle is essentially calling send (), the amount of data to be transmitted in the remaining space in the cache has an end, data is not lost, cycle call send () until finished)
4、s.recvfrom()
5, s.getpeername () is connected to the distal end address of the current socket
6, s.close () closes the socket
7、s.setblocking()
8、s.getaddrinfo()
9、s.getfqdn()
10、s.gethostbyname()
4.2, stick pack phenomenon
4.2.1 reasons:
Each time data is sent, the data to be copied from one program to the client socket buffer. In order to improve the transmission efficiency, the sender to send to each other tend to collect enough data after the disposable. T a, computer and networking basics
1.1, basic computer knowledge
Application software to the hardware operation must call the operating system interface, controlled by the operating system hardware
The client software wants to send a message to a server-based network software process:
1, the client software to generate data, the client software stored in the memory, and then sends the call interface to the operating system memory or copy data in their memory.
2, after the operating system receives data, rules (protocol) specified by the client software, call to send data card.
3, the data transmission network.
4, call server software system interface to the operating system sends the data from the operating system into memory copy own memory instructions.
5, the server 4 receives the instruction of the operating system, using the same client rule (protocol), the data received from the network card, a copy of the server software.
1.2 What is a network
+ = The physical network media links Internet Protocol
Internet Protocol is divided into seven layer OSI or TCP / IP protocol five
Two, TCP / IP layers Comments
2.1 Physical Layer
Function: high and low voltage transmission based on the electrical characteristics (electrical signal), a high voltage corresponding to the number 1, a low voltage corresponding to the digital 0.
2.2, the data link layer
Function: a simple electrical signals 0 and 1 does not make sense, therefore defines the grouping of electrical signals, namely Ethernet protocol.
2.1.1 Ethernet protocol (ethernet)
1, a predetermined set of electrical signals constitute a data packet, called 'frames'. Each frame consists of two parts 'header' and 'data'.
A header (18 bytes): sender / source address (6 bytes) + recipient / destination address (6 bytes) of data type + (6 bytes)
Data (46--1500 bytes)
2, mac address
That is the source and destination addresses
Each card are fired on a world unique mac address, a length of 48 binary factory, generally indicated by 12 hexadecimal
mac address (12) = 6 + 6 line vendor ID No.
2.1.2 Broadcast
Obtaining a host mac address arp by another host within the same network protocol, which then communicates with the broadcast.
2.3, the network layer
Function: the introduction of a new address to distinguish between different broadcast domain / subnet, i.e. the network address.
2.3.1, IP protocol
1, IP protocol defines the network address for the IP address, there are two versions of ipv4 and ipv6 to ipv4-based. Ipv4 IP address is represented by a 32-bit binary, usually written as four decimal numbers (decimal point). Range: 0.0.0.0 - 255.255.255.255
2, the core IP protocol is TCP / IP protocol, all TCP, UDP IP data is transmitted in a data format. Since the IP protocol does not provide data not communicate after a handling mechanism, it is not a reliable protocol.
. 3, the maximum length of the IP packet of 65535 bytes, 2 bytes as described in the data length of the IP datagram using. 16 described up to 2 to the power.
2.3.2, subnet mask
1 shows a sub-network characteristic parameter, the form is equivalent to an IP address, also 32-bit binary, are all part of a network, the host portion are all 0.
2, effect: combined with the IP address, the IP address identifies a network which bits represent bits, which bits represent the host bits.
2.3.3, IP address classification
1, A Class A addresses: the network address 1 byte + 3 byte host address. Ranging from 1.0.0.0 to 126.0.0.0, there are available a class A network 126, each network can accommodate more than 100 million hosts
2, Class B addresses: the network address 2 bytes + 2 bytes of the host address. Ranging from 128.0.0.0 to 191.255.255.255, Class B network has the available 16382, each network can accommodate a plurality of hosts 60,000
3, Class C addresses: the network address of 3 bytes + 2 bytes of the host address. Ranging from 192.0.0.0 to 223.255.255.255, available Class C network has more than a 209, 254 can accommodate each network host
4, Class D address: used for multicast, the address is a special reserved. Multicast group address is used to address one computer, it identifies a set of computer share the same protocol. Ranging from 224.0.0.0 to 239.255.255.255
5, E Class Address: reserved for future use, the range of 240.0.0.0 to 255.255.255.254
All-zero address (0.0.0.0) corresponding to the current host
All "1" address (255.255.255.255) is the current subnet broadcast address
Loopback address (127.0.0.1), also known as the host address, called loopback interface.
1, any data to the loopback address are input as IP
2, any data to the host IP address are sent to the loopback interface.
3, passed to a broadcast address or multicast IP packets will pass a copy of the loopback interface, and then sent to the Ethernet. This is because the definition of broadcast transmission and multicast transmission includes the host itself
2.3.4, ARP protocol
Functions: sending broadcast packets ways to get mac address of the target host
Way of working:
1, by IP address and subnet mask to distinguish which own subnet
2, analysis host and destination address is in the same network, if not, get the other side of the gateway mac.
3, the packet will be transmitted in a broadcast manner in which the end of the transmission within a subnet, the host receives all the disassembled packet, the destination IP is found on its own in response, return to their mac.
2.3.5, ICMP protocol
Function: When the IP packet transmission errors, such as host unreachable, routes and other unreachable, ICMP error message packet protocol will transmit back to the host, the host to handle the chance of error.
There are tools through ICMP ping and traceroute.
1、ping
Whether using the ICMP protocol packets to detect up to another host. Principle is requesting ICMP type code 0, the host receives a request code with a ICMP echo type 8. ping program delivery interval and number of packets is calculated.
Where the user can generally determine the network
2、traceroute
View from the current host to an address after a total of how many routes.
2.4, the transport layer
Port: Applications and associated card number, in the range of 0 - 65535,0 --1023 is occupied by the system
Transport layer function: establishing a communication port to port by the TCP and UDP protocols
2.41, TCP protocol
. 1, the length of the TCP packet is not limited, and can be theoretically infinite, network efficiency is guaranteed units, usually not more than the length of the IP packet, to ensure that a single TCP packet is not divided.
2, reliable transmission: they do not get recognized on the re-send the data until it receives the acknowledgment.
3, when connected "three-way handshake" disconnect "fourth wave"
4, UDP protocol network overhead is relatively large
2.4.2, UDP protocol
1, unreliable delivery
2, UDP header of 8 bytes comes
3, a small network overhead.
Three, UDP packet size issues
1, the data link layer, and last bytes of the data frame 18 is removed, the maximum length (MTU) data portion of 1500 bytes.
2, the network layer, the IP packet header occupies 20 bytes, where the MTU is 1480 bytes.
3, the transport layer, UDP packet header occupies 8 bytes, where the MTU is 1472 bytes.
4, in the application layer, UDP data generated not more than 1472 bytes. When exceeding, the IP packet will be fragmented transmission, a data loss when a certain, UDP data will be discarded.
Four, socket module
the intermediate socket is an application abstraction layer and transport layers, is a set of interfaces. The complex TCP / IP protocol suite is hidden behind the socket interface. Invoking the interface, you can send messages to other machines.
4.1, socket socket common method
socket.socket(family=AF_INET, type=SOCK_DTREAM, proto=0, fileno=None)
4.1.1, socket type type
TCP-type type socket.SOCK_STREAM
UDP-type type socket.SOCK_DGRAM
4.1.2, the server socket functions
1, s.bind () bind (host, port number) to the socket
2, s.listen () to start listening TCP
3, s.accept () passive acceptance of client TCP connections (blocking) waiting for connection
4.1.3 client socket functions
1, s.connect () initialize TCP server connections active
2, s.connect_ex () returns an error code when the connect () function is an extended version of error, rather than throwing an exception
4.1.4, public use socket functions
1, s.recv () received data
2, s.send () to send data (data to be sent when the amount is larger than the remaining space hexyl side buffer, data is lost, is not finished)
3, s.sendall () complete TCP data transmission (when the cycle is essentially calling send (), the amount of data to be transmitted in the remaining space in the cache has an end, data is not lost, cycle call send () until finished)
4、s.recvfrom()
5, s.getpeername () is connected to the distal end address of the current socket
6, s.close () closes the socket
7、s.setblocking()
8、s.getaddrinfo()
9、s.getfqdn()
10、s.gethostbyname()
Fifth, stick pack phenomenon
5.1 reasons:
Each time data is sent, the data to be copied from one program to the client socket buffer. In order to improve the transmission efficiency, the sender to send to each other tend to collect enough data after the disposable. such
The other will receive the packet data stick. The problem is that the receiver does not know the boundaries between the message, do not know how many bytes the extraction time, causing confusion reception data.
5.2, sticky problem only exists in the TCP packet in
1, TCP protocol is a stream-oriented protocol, application sees the data as a whole, or a stream. Unable to determine the boundaries between data. So stick package prone to problems. We must provide mechanisms for scientific unpacking.
2, message oriented protocol is UDP protocol, each segment is a UDP message, in the application must extract the data units of the message, not a byte to extract any data.
3, TCP is a stream-based, so the message transmission and reception can not be empty, which requires handling mechanisms are added in the empty message client and server, a program to prevent jamming. The UDP datagram is based, even
Input air content, the message is not blank, UDP protocol automatically package the message header.
5.3, stick package Problem Solution: struct module by means.
Sixth, to achieve a large file upload through the socket