1 Operating system overview
1.1 Computer Principles
Most modern computers are based on the von Neumann structure. The core idea of this structure is to store the program and data in the computer, execute the first instruction of the program according to the first address of the stored program in the memory, and then process the data. calculate.
A computer should include five basic components: arithmetic unit, controller, storage, input and output devices.
The computer uses binary to represent instructions and data, puts the written program into the memory, and then starts the computer to work, without the intervention of the operator, it can automatically take out the instructions one by one and execute the instructions.
A computer is made up of software and hardware. The hardware is mainly composed of CPU, storage devices, and input and output devices. Software includes operating system, system software and application software.
1.2 What is an operating system
Operating system, English is operating system (referred to as OS). Wikipedia defines it as:
An operating system is a computer program that manages computer hardware and software resources, and is also the core and cornerstone of a computer system.
Simply put, an operating system is a program, slightly different from the ones we usually use, that helps us coordinate, manage, and control computer hardware and software.
A computer without an operating system is called "bare metal." The operating system runs on the bare metal, and any other software can only run with the support of the operating system.
Operating systems exist to make programming easier. If there is no operating system, a programmer who writes an application program needs to first write a program that can control computer hardware, and then write an application program based on this program. To write a program that controls computer hardware, one needs to understand the computer hardware. The working principle, and then call the corresponding instruction set to control it.
1.3 Classification of operating systems
Operating systems have been developed to date, and there are many types, which can be divided according to different fields of application:
Desktop Operating System, Server Operating System, Embedded Operating System, Mobile Device Operating System
1.3.1 Desktop Operating System
Usually the so-called desktop refers to the graphical operation page. Before the desktop operating system was born, the most famous operating system was DOS, but the operation interface of DOS was very unfriendly, it was just code. For this reason, Microsoft launched their first graphical interface operating system-windows 1.0, although There are only 256 colors, but it was attractive enough at the time, until today the three-legged situation of mac os, windows, and linux.
1.3.2 Embedded Operating System
Embedded system is a kind of "special computer system completely embedded in the controlled device and designed for specific applications", such as μClinux (embedded Linux), WinCE (Microsoft embedded, mobile computing platform), RTOS (embedded real-time operation systems, military aviation) or other operating systems. It is widely used in industry, military, aviation and other fields.
1.3.3 Server Operating System
Server operating systems generally refer to operating systems installed on large computers, such as Web servers, application servers, and database servers, which are the infrastructure platforms of enterprise IT systems.
Server operating systems are mainly divided into four genres: Unix, Linux, Windows Server and Netware.
1.3.4 Mobile Device Operating System
Mobile devices are also known as cell phones. The operating system is mainly used on smartphones.
The operating systems currently used on mobile phones mainly include Android (Google), iOS (Apple), Symbian (Nokia), BlackBerry OS (Blackberry), windows mobile (Microsoft), Harmony (Huawei Hongmeng), etc.
2 Introduction to Linux Basics
2.1 History of Linux Development
2.1.1 The origin and development of Unix
Unix is one of the earliest widely used computer operating systems;
In 1969, the first version of the Unix system was implemented by Ken Thompson at AT&T Bell Labs. In the late 1970s: AT&T established the Unix System Laboratory;
Late 1970s: CSRG (Computer Systems Research Group, University of California, Berkeley) used Unix to conduct research on the operating system and subsequently extended the famous BSD Unix (Berkeley Software Distributions) version;
2.1.2 Development of open source code
Source code refers to an uncompiled text file written in accordance with a certain programming language specification. Open source code means that the program developer discloses the code file written by himself to others for learning, use or modification.
At first, Unix was an open source project of AT&T laboratory, so the source code of Unix can be easily obtained, but with the wide application of Unix, AT&T began to realize the value of Unix, and established a special Unix laboratory in 1979 (USL ), and AT&T simultaneously announced ownership and commercialization of Unix.
This also led to the release of BSD Unix by the CSRG; in 1983, in order to oppose the trend of privatization of software ownership, Richard Stallman launched the GNU project to promote the development of open source code, and developed a lot of software for this purpose, and created the Free Software Foundation Council (FSF) and developed the GNU General Public License, put forward the concept of "anti-copyright".
2.1.3 The Birth of Linux
Detailed development history of inux: https://www.linuxprobe.com/linux-historical-chronology.html
In a nutshell, the Linux kernel project was created by Finland's Linux Torvalds and is in charge of subsequent upgrades and maintenance, and the logo is taken from the Finnish mascot. Most importantly, open source!
2.2 Linux kernel (kernel)
The kernel is the heart of the operating system, the core program that runs programs and manages hardware devices such as disks and printers. In linux terms, the kernel refers to the core of the operating system. Linux kernel version:
-
stable version
With industrial strength, it can be widely used and deployed.
-
beta version
Changes quickly due to experimenting with various solutions
Kernel source URL: http:www.kernet.ort
All modifications to the linux source code from all over the world will eventually be collected on this website, and the open source community led by Linus will screen and modify them, and finally decide whether to enter the Linux mainline kernel source code.
Therefore, the so-called linux operating system consists of the following parts:
Linux operating system = linux kernel + GNU software and system software + necessary applications
2.3 Linux distributions
The Linux distribution is a collection of products consisting of the Linux kernel and various commonly used software. Today, there are hundreds of Linux distributions in the world.
From a large perspective, Linux distributions can be roughly divided into personal desktop editions and enterprise server editions.
2.3.1 Personal desktop version
For individual users. Because the image interface is installed, the interface is beautiful and easy to operate, which is more suitable for novices, but it consumes more resources than the server version. Common desktop versions include Ubuntu, Debian, Fedora, CentOS, openSUSE, etc.
2.3.2 Enterprise Server Edition
For enterprise users. The system has good stability, low resource consumption, and meets the needs of enterprises, but it is basically a command-line interface, which requires a certain foundation to get started. Common server versions include RedHat Enterprise Edition, CentOS, Debian, Ubuntu, openSUSE, etc.
3 Vmware virtual machine
3.1 VMware virtual network
3.1.1 Virtual network card, virtual switch
A network card, also known as a network adapter, is a piece of computer hardware designed to allow a computer to communicate over a computer network. It allows users to connect to each other via cable or wirelessly. Every computer network card on the network must have a unique MAC address.
When using VMware Workstation to install a virtual machine, a virtual wired network card will be automatically installed. In addition, multiple virtual network cards can be manually added to the virtual machine.
Switches are used for the forwarding of electrical (optical) signals. An exclusive electrical signal path can be provided for any two network nodes connected to the switch. A simple understanding can connect some computers together to form a local area network.
20 virtual switches were automatically installed when VMware Workstation 12 was installed.
Commonly used switches are VMnet0, VMnet1, and VMnet8, corresponding to the three network modes of VMware.
3.1.2 Virtual Bridge
A bridge is a bridge that establishes a connection between a local area network and another local area network. It is a means of expanding the network and communication, forwarding data signals in various transmission media, and extending the distance of the network.
In VMware, the virtual network bridge will only be applied if the virtual network card used by the created virtual machine is connected to the VMnet0 virtual switch.
3.1.3 Virtual DHCP server
When the virtual network card is connected to the VMnet1 or VMnet8 virtual switch, it will automatically install and run the virtual DHCP server and connect to this virtual switch.
DHCP server: (Dynamic Host Configuration Protocol, Dynamic Host Configuration Protocol), there are two main purposes:
1. Automatically assign IP addresses to the internal network;
2. As a means of centrally managing all computers.
3.1.4 Virtual NAT server
Only when the virtual network card is connected to the VMnet8 (NAT) virtual switch will the virtual NAT server be automatically installed and run, and connected to this virtual switch.
NAT server: (Network Address Translation, Network Address Translation) server that provides NAT function: it converts the IP address used in the internal network into the IP address used in the external network, and converts the non-routable IP address into a routable IP address, concealing the internal network from the external network.
3.1.5 Host virtual network card
When we install VMware Workstation 12, the host virtual network card will be installed on our physical PC machine.
3.2 VMware network mode
3.2.1 Bridged
The bridge mode is to use the virtual bridge to communicate between the host network card and the virtual network card of the virtual machine. Under the function of bridging, it is similar to virtualizing the physical host as a switch. All the virtual machines set up by bridging are connected to an interface of this switch, and the physical host is also plugged into this switch, so all network cards and network cards under the bridge are connected. It is in exchange mode and can be accessed without interfering with each other. In bridge mode, the IP address of the virtual machine needs to be on the same network segment as the host. If networking is required, the gateway and DNS need to be consistent with the host network card.
As for whether to bridge to the host's wireless network card or wired network card, it depends on the host's Internet access mode.
The biggest feature of the bridge mode is that the network characteristics of the virtual machine are almost the same as those of the host except for the IP.
3.2.2 NAT (Network Address Translation)
In the NAT network, the VMnet8 virtual switch is used, and the VMware Network Adapter VMnet8 virtual network card on the host is connected to the VMnet8 switch to communicate with the virtual machine, but the VMware Network Adapter VMnet8 virtual network card is only used to communicate with the VMnet8 virtual network card. It is used for the network segment communication of the switch. It does not provide the routing function for the VMnet8 network segment. The virtual machines under the virtual NAT network use the virtual NAT server to connect to the Internet.
At this time, the virtual machine and the host machine can achieve mutual access. If the host machine is connected to the Internet at this time, the virtual machine can also be connected to the Internet. The VMware Network Adapter VMnet8 virtual network card here only provides an interface for the communication between the Host and the virtual machine under the NAT virtual network. Therefore, even if the virtual network card is uninstalled, the virtual machine can still access the Internet, but the host can no longer access the VMnet8 network segment.
In NAT mode, the host needs to enable VMware NAT Service and VMware DHCP Service.
The biggest advantage of using the NAT mode is that it is very simple for the virtual system to access the Internet. You do not need to perform any other configurations, and you only need the host machine to be able to access the Internet. If you want to use VMware to install a new virtual system, you can directly access the Internet without any manual configuration in the virtual system, it is recommended that you use the NAT mode.
3.2.3 Host-only (host)
The Host-Only mode is actually the NAT mode that removes the virtual NAT device, and then uses the VMware Network Adapter VMnet1 virtual network card to connect to the VMnet1 virtual switch to communicate with the virtual machine. The Host-Only mode separates the virtual machine from the external network, making the virtual machine a An independent system that only communicates with the host computer.
3.3 SSH (Secure Shell Protocol)
SSH is the abbreviation of Secure Shell, which is a network security protocol designed to provide security for remote login sessions and other network services. By using SSH, the transmitted data can be encrypted to effectively prevent information leakage during remote management.
From the perspective of the client, there are two authentication methods: password-based and key-based
3.3.1 Username and password verification methods
illustrate:
(1) When the client initiates an ssh request, the server will send its own public key to the user;
(2) The user will encrypt the password according to the public key sent by the server;
(3) The encrypted information is sent back to the server, and the server decrypts it with its own private key. If the password is correct, the user logs in successfully.
3.3.2 Key-based login method
illustrate:
(1) First generate a pair of keys (public key, private key) on the client;
(2) The client copies the generated public key to the server machine that needs to be accessed;
(3) When the client sends a connection request again, including ip and user name;
(4) After the server receives the client's request, it will search in authorized_keys. If there is a corresponding IP and user, it will randomly generate a string, for example: qwer;
(5) The server will use the public key copied from the client to encrypt the random string qwer, and then send it to the client;
(6) After receiving the message from the server, the client will use the private key to decrypt, and then send the decrypted string to the server;
(7) After the server receives the string sent by the client, it compares it with the previous string, and if it is consistent, it allows password-free login.
3.3.3 Use of OpenSSH
OpenSSH is a free and open source implementation of the SSH protocol.
OpenSSH consists of client and server software. The server is a daemon that runs in the background and responds to connection requests from clients. The server is generally the sshd process, which provides the processing of remote connections. The client includes the ssh program and other applications like scp (remote copy), slogin (remote login), sftp (secure file transfer), etc. By default, the CentOS system comes with the OpenSSH service installed.
Example: Configure password-free login for machines from node1 to node2.
-
on the node1 machine
ssh-keygen -t rsa [dsa] Press Enter four times to generate the key file and private key file id_rsa, id_rsa.pub
-
Copy the public key to the node2 machine
ssh-copy-id node2 For the first connection, you need to enter the user password, and after the verification is successful, you can log in without password.
3.4 File upload and download
In Linux-related development, it is often necessary to upload and download files and installation packages between Linux and Windows. sftp and lrzsz are two widely used methods.
3.4.1 SFTP
sftp is the abbreviation of Secure File Transfer Protocol, a secure file transfer protocol. It can provide a secure network encryption method for transferring files. sftp is a part of SSH, and it does not have a separate daemon process. It uses the sshd daemon process (port number is 22 by default) to complete the corresponding connection and reply operations.
After SecureCRT connects to CentOS remotely, press alt+p to open the sftp session window.
Commonly used sftp commands are:
pwd and lpwd
pwd is the default current directory of the machine where the sftp service is located (ie CentOS)
lpwd is to look at the Windows local default directory.
ls和lls
ls View the contents of the default current directory of the sftp server
lls is to see the contents of the default current directory of Windows
put d:/sparksql_textdata.csv
Upload files on Windows to the default current directory of the sftp server
get install.log.syslog
Download the files in the current directory of the sftp server to the current directory of windows
Help
help command. List the usage of sftp.
Section 3.4.2
lrzsz is a program that can replace sftp upload and download in linux.
In the Centos system, you can directly run yum -y install lrzsz and the program will be installed automatically. You can also download the installation package and install it offline. For details, refer to the attached information.
Upload files: Enter rz in the command line to open the file upload session window.
Download file: sz download file path
3.5 System backup (snapshot)
3.5.1 Backup system
a. Right-click the target centos system -> snapshot -> snapshot manager
b. Take a snapshot
3.5.2 System recovery (restore to snapshot state)
Right-click the target centos system -> snapshot -> restore to the specified snapshot