第23章 HTTP服务和APACHE

更多内容请点击:

Linux学习从入门到打死也不放弃,完全笔记整理(持续更新,求收藏,求点赞~~~~) 

http://blog.51cto.com/13683480/2095439


第23章  HTTP服务和APACHE

本章内容:

               socket套接字基础

               http协议介绍

               httpd介绍

               httpd2.2相关配置

               httpd2.4特性

               编译安装httpd

                    

INTERNET介绍以及历史:略

               

--------------------------------------------------------------------------------

Sockte:套接字:

               跨internet的主机间通讯:

               在建立通信连接的每一端,进程间的传输要有两个标志:

                             IP地址和端口号,合称为套接字地址 Socket  address

                             客户机套接字地址定义了一个唯一的客户进程

                             服务器套接字地址定义了一个唯一的服务器进程

               如:192.168.65.132 80

               

               socket:套接字,进程间通信(IPC)的一种实现,允许不同主机(或同一主机)上

                             不同进程之间进行通信和数据交换,socketAPI出现于1983年,4.2BSD实现

               socketAPI:

                             封装了内核中所提供的socket通信相关的系统调用

               socket domain:根据其所使用的地址

                            AF_INET:       Address Family IPv4

                            AF_INET6:      IPv6

                            AF_UNIX: 同一主机上不同进程之间通信时使用

               Socket type:根据使用的传输层协议

                            SOCK_STREAM:     流,tcp套接字,可靠的传递,面向连接

                            SOCK_DGRAM:            数据报,udp套接字,不可靠的传递,无连接

                             SOCK_RAW:        裸套接字,无须tcp或udp,APP直接通过IP包通信

                             

               套接字相关的系统调用:

                            socket():  创建一个套接字

                            bind():            绑定IP和端口

                             listen():    监听

                            accept():  接收请求

                            connect():       请求连接建立

                            write():    发送

                            read():            接收

                            close():    关闭

                             

socket通信示例:

1      准备脚本:

服务器端tcpserver.py

vim tcpserver.py

#!/usr/bin/python

import socket

HOST='0.0.0.0'

PORT=9090

BUFFER=4096

sock=socket.socket(socket.AF_INET,socket.SOCK_STREAM)

sock.bind((HOST,PORT))

sock.listen(3)

print('tcpserver listen at: %s:%s\n\r'  %(HOST,PORT))

while True:

         client_sock,client_addr=sock.accept()

         print('%s:%s connect' %client_addr)

         while True:

                 recv=client_sock.recv(BUFFER)

                if not  recv:

                         client_sock.close()

                         break

                print('[Client %s:%s said]:%s'  %(client_addr[0],client_addr[1],recv))

                 client_sock.send('tcpServer  has received your message')

sock.close()

                             

客户端tcpclient.py

vim tcpclinet.py

#!/usr/bin/python

import socket

HOST='192.168.65.132'

PORT=9090

BUFFER=4096

sock=socket.socket(socket.AF_INET,socket.SOCK_STREAM)

sock.connect((HOST,PORT))

sock.send('hello,tcpServer!')

recv=sock.recv(BUFFER)

print('[tcpServer said]: %s' %  recv)

sock.close()          

                             

2      通信过程:

               服务器端(192.168.65.132) 开启服务

                            python  tcpserver.py

               客户端执行:python tcpclient.py

               

               

-----------------------------------------------------------------------------

HTTP协议:

               相关术语:

               http: Hyper text transfer protocol,80/tcp 超文本传输协议,

               html:  hyper text markup language  超文本标记语言,编程语言

                            示例:

                                           <html>

                                           <head>

                                                         <title> html语言</title>

                                           </head>

                                           <body>

                                           <h1>标题1</h1>

                                           <p><a href=http://www.magedu.com>马哥教育</a>欢迎你</p>

                                           <h2>标题2</h2>

                                           </body>

                                           </html>

               CSS:      Cascading style sheet 层叠样式表

               js: javascript

               MIME:   multipurpose internet mail extensions

                             多用途互联网邮件扩展,/etc/mime.types

               

http协议介绍:

               http/0.9:

                             1991年,原型版本,功能简陋,只有一个命令GET  /index.html,服务器只能回应

                             HTML格式字符串,不能回应别的格式

                             

               http/1.0:

                     1996年5月,支持cahce,MIME,method

                     1     每个tcp连接只能发送一个请求,发送数据完毕,连接就关闭,如果还要请求

                             其他资源,就必须再新建一个连接

                     2     引入了POST命令和HEAD命令

                     3     头信息是ASCII码,后面数据可以为任意格式。服务器回应时会告诉客户端,数据

                            是什么格式,即Content-Type字段的作用,这些数据类型总称为MIME,多用途互联网

                             邮件扩展,每个值包括一级类型和二级类型,预定义的类型,也可以自定义类型

                     4     常见content-type值:text/xml,image/jpeg,audio/,mp3

               

               http/1.1:

                     1997年1月

                     1     引入了持久连接(persistent  connection),即tcp连接默认不关闭,可以被多个

                             请求复用,不用生命connection:keep-alive。对于同一个域名,大多浏览器允许

                            同时建立6个持久连接

                     2     引入了管道机制(pipelining),即在同一个tcp连接里,客户端可以同时发送多个

                             请求,进一步改进了http协议的效率

                     3     新增方法:     PUT,PATCH,OPTIONS,DELETE

                     4     同一个tcp连接里面,所有的数据通信时按次序进行的。服务器只能顺序处理回应,

                             前面的回应慢,会有许多请求排队,造成"对头堵塞"(head-of-line blocking)

                     5      为避免上述问题,有两种方法:一是减少请求数,二是同时多开持久连接。

                             网页优化技巧:比如合并脚本和样式表,将图片嵌入css代码,域名分片(domain sharding)  

                     6      http协议不带有状态,每次请求都必须附上所有信息。请求的很多字段都是重复的

                             浪费带宽,影响速度。

               

               sady:2009年,谷歌研发,解决HTTP/1.1效率不高问题

               http/2.0:  2015年

                     1     头信息和数据体都是二进制,称为头信息帧和数据帧

                     2     复用tcp连接,在一个连接里,客户端和浏览器都可以同时发送多个请求或回应,

                            且不用按顺序--对应,避免了“对头堵塞”,此双向的实时通信称为多工(multiplexing)

                     3     引入头信息压缩机制(header  compression),头信息使用gzip或compress

                             压缩后再发送;客户端和服务器同时维护一个头信息表,所有字段都会存入

                             这个表,生成一个索引号,不发送同样字段,只发送索引号,提高速度

                     4      http/2.0允许服务器为经请求,主动向客户端发送资源,即服务器推送

                            (server push)

               

http工作机制:

               工作机制:

                            http请求:http request

                            http响应:http response

                            一次http事务:请求 <--> 响应

               web资源: web resource

                             一个网页有多个资源构成,打开一个页面,会有多个资源展示出来,但是每个

                             资源都要单独请求。因此,一个web页面,通常并不是单个资源,而是一组资源的

                            集合

               静态文件:无需服务端做出额外处理

                            文件后缀:.jpg .html .txt .js .css .mp3 .avi

               动态文件:服务端执行程序,返回执行的结果

                            文件后缀:.asp .php .jsp

                             

               http连接方式和性能:

                             串行连接:建立连接(tcp握手)-http事务-断开连接(tcp挥手)

                                            低效率,多个请求会顺序执行                 

                             并行连接:通过多条tcp连接发起并发的http请求

                            持久连接:keep-alive,长连接,重用tcp连接,以消除连接和关闭的时延

                                            以事务个数和时间来决定是否关闭连接

                                            一次tcp连接过程可以处理多次请求,省去断开之后重新建立连接的过程

                             管道化连接:通过共享tcp连接发起并发的http请求

                                            建立持久连接之后,并行处理发起httpd事务

                             复用的连接:交替传送请求和相应报文(实验阶段)

URL 介绍:

               URI: Uniform Resource Identifier 统一资源标识,分为URL和URN

               

               URN: Uniform Resource Naming ,统一资源命名

                     示例:P2P下载使用的磁力链接是URN的一种实现

                             magnet:?xt=urn:btih:660557A6890EF888666

               

               URL:Uniform Resoruce  Locator

                      统一资源定位符,用于描述某服务器某特定资源位置

               

               两者区别:URN如同一个人的名称,而URL代表一个人的住址。换言之,URN定义某事物的身份,

                            而URL提供查找该事物的方法,URN仅用于命名,而不指定地址

                             

               URL 组成:

                             <scheme>://<user>:<password>@<host>:<port>/path;<params>?<query>#<frag>

                            例如:

                                           http://user5:[email protected]/download/

                             

                            scheme:  方案,访问服务器以获取资源时要使用哪种协议

                            user:     用户,某些方案访问资源时需要的用户名

                             password:密码,用户对应的密码,中间用":"分隔

                            host:       主机,资源宿主服务器的主机名或IP地址

                            port:     端口,资源宿主服务器正在监听的端口号,很多方案有默认端口号

                            path:     路径,服务器资源的本地名,由一个"/"将其与前面

                            params:  参数,指定输入的参数,参数为名/值对,多个参数用";"分隔

                            query:  查询,传递参数给程序,如数据库,用"?"与前面分隔,多个查询使用"&"分隔

                            frag:      片段,一小片或一部分资源的名字,此组件在客户端使用,用#分隔

                             

                            示例:

        

网站访问量:

               IP(独立IP):即internet  protocol,指独立IP数。一天内来自相同客户机IP地址只

                                    计算一次,记录远程客户机IP地址的计算机访问网站的次数,是衡量网站

                                    流量的重要指标

               PV(访问量):即Page  View,页面浏览器或点击量,用户每次刷新即被计算一次,PV反映

                                    的是浏览某网站的页面数,PV与来访者的数量成正比,PV并不是页面的来访者

                                    数量,而是网站被访问的页面数量

               UV(独立访客):即Unique  Visitor,访问网站的一台电脑为一个访客,一天内相同的客户端

                                    只被计算一次,可以理解成访问某网站的电脑的数量,网站判断来访电脑的身份

                                    是通过来访电脑的cookies实现的。如果更换了IP后但不清除cookies,再访问

                                    相同网站,该网站的统计中UV数是不变的。

               网站统计:http://www.alexa.cn/rank

               

               QPS:       request per second      每秒请求数   

               PV,QPS,并发连接数换算公式

                             QPS=PV*(页面衍生连接次数/统计时间(86400))

                            并发连接数=QPS*http 平均响应时间

               峰值时间:每天80%的访问集中在20%的时间里,这20%时间为峰值时间

               峰值时间每秒请求数(QPS)=( 总PV数 *页连接次数)*80% ) / ( 每天秒数* 20% )

               

http:一次完整的请求处理过程

               1     建立连接(TCP握手,ssl握手)

                             接收或拒绝连接请求

               

               2     接收请求:

                             接收客户端请求报文中对某资源的一次请求的过程

                                    

                     web访问相应模型(web I/O)

                            单进程I/O模型:

                                           启动一个进程处理用户请求,而且一次只处理一个,

                                           多个请求被串行相应

                            多进程I/O模型:

                                           并行启动多个进程,每个进程相应一个连接请求

                            复用I/O结构:

                                           启动一个进程,同时响应N个连接请求

                                           多线程模型:一个进程生成N个线程,每个线程相应一个连接请求

                                           事件驱动:event,一个进程处理N个请求

                            复用的多进程I/0模型:

                                           启动M个进程,每个进程相应N个连接请求,同时接受M*N个请求

                                           

               3     处理请求:

                             服务器对请求报文进行解析,并获取请求的资源及请求方法等相关信息,

                             根据方法,资源,首部和可选的主体部分对请求进行处理

                                    元数据:请求报文首部

                                    <method> <URL> <VERSION>

                                   HEADERS  格式 name:value

                                   <request  body>

                                    示例:

                                    Host:www.magedu.com          请求的主机名称

                                    Server:Apache/2.4.7

                             

                             HTTP常用请求方式,method:

                                    GET,POST,HEAD,PUT,DELETE,TRACE,OPTIONS

                                    

               4     访问资源:

                             服务器获取请求报文中请求的资源web服务器,即存放了web资源的服务器,

                             负责向请求者提供对方请求的静态资源,或动态运行后生成的资源

                             

                             资源放置于本地文件系统特定的路径:DocRoot

                                   SERVER  DOCROOT --> /var/www/html     

                                    /var/www/html/images/logo.jpg 访问路径即为:

                                    http://SERVER/images/logo.jpg

                             

                             WEB服务器资源路径的映射方式:

                                   (1)      docroot

                                   (2)      alias

                                   (3)      虚拟主机docroot

                                   (4)      用户家目录docroot

                                    

               5     构建相应报文:

                            一旦web服务器识别出了资源,就执行请求方法中描述的动作,并放回相应

                            报文。

                             相应报文中,包含有响应状态码,响应首部,如果生成了响应主体的话,

                            还包括响应主体

                             

                            响应主体:

                                           如果事务处理产生了响应主体,就将内容放在响应报文中回送过去,响应

                                           报文中通常包括:

                                           (1)描述了响应主体MIME类型的Content-type首部

                                           (2)描述了响应主体长度的content-length

                                           (3)实际报文的主体内容

                                           

                            URL重定向:

                                           web服务器构建的相应并非客户端请求的资源,而是资源另外一个访问路径

                                           永久重定向:301

                                           临时重定向:302

                                                  

                            MIME类型:

                                           web服务器要负责确定相应主体的MIME类型,多种配置服务器的方法

                                           可以将MIME类型与资源管理起来

                                           魔法分类:apache  web服务器可以扫描每个资源的内容,并将其与一个已知

                                                         模式表(被称为魔法文件)进行匹配,以决定每个文件的MIME类型,

                                                         这样做可能比较慢,但很方便,尤其是文件没有标准扩展名时。

                                           显示分类:可以对web服务器进行配置,但其不考虑文件的扩展名或内容,强制

                                                         特定文件或目录内容拥有某个MIME类型

                                           类型协商:有些web服务器经过配置,可以以多种文档格式来存储资源。在这种

                                                         情况下,可以配置web服务器,使其可以通过与用户的协商来

                                                         决定使用哪种格式(及相关的MIME类型)“最好”

                                                         

               6     发送相应报文

                             web服务器通过连接发送数据时也会面临与接收数据一样的问题。

                             服务器可能有很多条到各个客户端的连接

                             有些是空闲的,有些在向服务器发送数据,还有一些在向客户端回送相应数据。

                             服务器要记录连接的状态,还要特别注意对持久连接的处理。

                             对非持久连接而言,服务器应该在发送了整条报文之后,关闭自己这一端的连接。

                             对持久连接来说,连接可能扔保持打开状态,在这种情况下,服务器要正确地

                            计算content-length首部,不然客户端就无法知道响应什么时候结束了

                             

               7     记录日志:

                             最后,当事务结束时,web服务器会在日志文件中添加一个条目,来描述

                            已执行的事务

                             

http服务器应用:

               http服务器程序:

                            httpd  apache

                             nginx

                             lighttpd

               应用程序服务器:

                            IIS  .asp

                            tomcat  .jsp

                            jetty       开源的servlet容器,基于jave的web容器

                            Resin  CAUCHO公司,支持sevlets和jsp的引擎

                             webshpere(IBM),weblogic(BEA,已被oracle收购),jboss,oc4j(oracle)

               市场占有率统计:

                             www.netcraft.com

                             

------------------------------------------------------------------------------

httpd介绍:

               20世纪90年代初,美国国家超级计算机应用中心NSCA开发

               1995年开源社区发布apache (a  patchy server)

               软件基金会:ASF(apache software  foundation)

               特性:

                            高度模块化:core + modules

                            DSO: Dynamic Shared Object 动态装/卸载

                            MPM: multi-processing module 多路处理模块

               功能特性

                            虚拟主机:

                                    基于IP,PORT,FQDN

                            CGI: Common Gateway  Interface,通用网关接口

                            反向代理

                            负载均衡

                            路径别名

                             丰富的用户认证机制

                                    basic

                                    digest

                            支持第三方模块

               

MPM工作模式:

               prefork:

                            多进程I/O模型,每个进程相应一个请求,默认模型

                             一个主进程:生成和回收n个子进程,创建套接字,不相应请求

                             多个子进程:工作进程,每个子进程处理一个请求,系统开始时,预先生成

                                                  多个空闲进程,等待请求,最大不超过1024个

                                                  ulimit -a

               worker:

                            复用的多进程I/O模型,多进程多线程,IIS使用此模型

                            一个主进程:生成m个子进程,每个子进程负责生成n个线程,每个线程相应一个

                                                  请求,并发相应请求m*n

               event:

                            事件驱动模型(worker模型的变种)

                            一个主进程:生成m个子进程,每个进程直接相应n个请求,并发相应已请求m*n

                             有专门的线程来管理这些keep-alive类型的线程,当有真实请求时,将请求

                             传递给服务线程,执行完毕后,又允许释放,这样增强了高并发场景下的请求处理

                            能力

                             centos6默认使用httpd-2.2 event 为测试版

                             centos7默认使用httpd-2.4 event 为稳定版

                             

httpd安装:

               安装:

                     centos6 httpd-2.2  

                     centos7  httpd-2.4

               安装方式:

                     rpm:centos发行版,稳定,建议使用

                     编译:定制或特殊需求

                    

centos6   httpd程序环境:

               配置文件:

                             /etc/httpd/conf/httpd.conf

                             /etc/httpd/conf.d/*.conf

               检查配置文件语法:

                            httpd -t  

                            service httpd  configtest

                             

               服务脚本:/etc/rc.d/init.d/httpd

                            脚本配置文件:/etc/sysconfig/httpd 主要用来配置MPM

               

               站点网页文档根目录:

                             /var/www/html

               模块文件路径:

                             /etc/httpd/modules

                             /usr/lib64/httpd/modules

                             前者是后者的软链接

                             

               主程序文件:

                             /usr/sbin/httpd

                             /usr/sbin/httpd.worker

                             /usr/sbin/httpd.event

                             

               主进程文件:

                             /etc/httpd/run/httpd.pid

                             开启服务此文件自动生成,记录主进程id号,关闭服务自动销毁

                             

               日志目录文件:

                             /var/log/httpd/

                            /etc/httpd/log/  是前者的软链接

                            access_log:     访问日志

                             error_log:错误日志

                             

               帮助文档包:

                             httpd-manual

                             

--------------------------------------------------------------------------------                    

httpd 2.2 常见配置:

               httpd 2.2 配置文件的组成

                            grep "Section"  /etc/httpd/conf/httpd.conf

                            ### Section 1:  Global Environment

                            ### Section 2:  'Main' server configuration

                            ### Section 3:  Virtual Hosts

                             

               配置格式:directive value   指令+值

                            directive:  不区分字符大小写

                             value:为路径时,是否区分大小写,取决于文件系统

                             

               1     修改监听的IP和port

                            Listen  [IP:]PORT

                            省略IP表示为本机所有IP

                            Listen  指令至少一个,可重复出现多次

                                          listen  80

                                          listen  9627

                                          listen  192.168.65.132:8080

                      注意:修改端口之后,必须重启服务才能生效

                    

               2     持久连接:

                            Persistent  Connection:连接建立,每个资源获取完成后不会断开连接,而是

                             继续等待其他请求完成,默认为关闭。

                            断开条件:

                                           数量限制:100     表示最多接受100个请求之后断开

                                           时间限制:15        表示15秒之后断开

                             副作用:对并发访问量较大的服务器,持久连接功能会使有些请求得不到相应

                             折中方案:使用较短的持久连接时间

                     配置选项:

                            Keepalive  ON|OFF 默认OFF,如果为off 下面两项失效

                            keepalivetimeout  15

                            maxkeepaliverequests  100

                    

                     测试:    telnet WEB_SERVER_IP  PORT

                                   GET /URL  HTTP/1.1

                                    HOST:WEB_SERVER_IP

                                    

               3     MPM(Multi-processing module) 多路处理模块

                            prefork,worker,event(实验阶段)

                             httpd-2.2不支持同时编译多个模块,所以只能编译时选定一个;rpm安装的包提供

                             三个二进制程序文件,分别用于实现对不同MPM机制的支持

                            确认方法:

                                          ps aux  |grep httpd

                            默认为/usr/sbin/httpd,即prefork模式

                             

                              查看静态编译的模块列表:

                                          httpd  -t  httpd.worker -t

                             查看静态编译及动态装载的模块

                                          httpd  -M

                            动态模块加载:

                                           不需要重启服务

                                           只需要reload 就生效                                      

                            动态模块路径:

                                           /etc/httpd/modules ->  ../../usr/lib64/httpd/modules

                             

                     更换MPM模块,即更换一个httpd程序

                            vim  /etc/sysconfig/httpd

                             HTTPD=/usr/sbin/httpd.worker

                            重启服务生效

                             

                                    prefork的默认配置:

                                   <IfModule  prefork.c>

                                    StartServers      8            开机启动多少进程

                                    MinSpareServers    5 最少保持多少空闲进程

                                    MaxSpareServers    20      最多保持多少空闲进程,超过就杀掉

                                    ServerLimit      256 最多进程数,最大20000

                                    MaxClients      256 最大并发

                                    MaxRequestsPerChild  4000  子进程最多能处理的请求数量。在处

                                           理MaxRequestsPerChild 个请求之后,子进程将会被父进程终止,这

                                           时候子进程占用的内存就会释放(为0时永远不释放)

                                    </IfModule>           

                                           

                                    worker的默认配置:

                                   <IfModule  worker.c>

                                    StartServers    4         开机启动的进程数

                                    MaxClients    300              最大用户连接数

                                    MinSpareThreads    25     最小保持线程数

                                    MaxSpareThreads 75            最大保持线程

                                    ThreadsPerChild    25       每个进程开启线程数

                                    MaxRequestsPerChild 0 无限制

                                    </IfModule>

                             

               4     DSO:Dynamic Shared Object

                     加载动态模块配置:

                             /etc/httpd/conf/httpd.conf

                             配置指定实现模块加载格式:

                                   LoadModule  <mod_name> <mod_path>

                             模块文件路径可使用相对路径:

                                    相对于Serverroot (默认为/etc/httpd)

                     示例:

                                   LoadModule  auth_basic_module modules/mod_auth_basic.so

                                    

               5     定义'main' server的文档页面路径

                                   DocumentRoot  "/path"

                             

                     文档路径映射:

                                   documentroot  指向的路径为URL路径的起始位置"/"

                                    

                     注意:selinux 和iptables的状态

                    

               6     定义站点主页面

                                    DirectoryIndex index.html index.html.var

                                    

               7     站点访问控制:

                      可基于两种类型的路径对哪些些资源进行何种访问控制

                     文件系统路径

                                    基于目录

                                   <directory  "/path">

                                    ...

                                    </directory>

                                    

                                    基于文件

                                   <file  "/path/file">

                                    ...

                                    </file>

                                    

                                    使用正则表达式

                                   <filematch  "PATTERN">

                                    ...

                                    </filematch>

                             

                     URL路径

                                   <Location  "">

                                    ...

                                    </Location>

                                    

                                    <LocationMatch "">

                                    ...

                                    </LocationMatch>

                                    

               8     访问控制机制:

                                    基于来源地址

                                    基于账号

                    

                      <directory>中“基于源地址”  实现访问控制

                            (1)options:后跟1个或多个以空白字符分隔的选项列表

                                    所有可用特性:Indexes Includes FollowSymLinks  SymLinksifOwnerMatch ExecCGI MultiViews

                                    在选项前+,-表示增加或者删除指定选项

                                    

                                    常见选项:

                                           index:指明的url路径下不存在与定义的主页面资源相符的资源文件时

                                                     返回索引列表给用户

                                                    

                                           followsymlinks:允许访问符号链接文件所指向的源文件

                                           ALL:      全部允许

                                           none:    全部禁用

                                           

                            (2)AllowOverride

                                    与访问控制相关的哪些指令可以放在指定目录下的.htaccess(由accessfilename指定)

                                    文件中,覆盖之前的配置指令。

                                    只对<directory>语句有效

                                           allowoverride all     所有指令都有效

                                           allowoverride now       .htaccess  文件无效 大多情情况选这个即可

                                           allowoverride authconfig indexes       除了authconfig和indexes  的其他指令都无法生效

               

                            (3)order 和allow,deny

                                    放在directory,.htaccess中

                                    order:定义生效次序,写在后面的表示默认法则

                                          order  allow,deny

                                          allow  from ..

                                           默认deny,在allow from  后面添加白名单

                                           

                                          order  deny,allow

                                          deny  from ..

                                           默认allow,在deny from  后面添加黑名单

                                           

                                    也可以:

                                          order  allow,deny

                                          allow  from 192.168.65

                                          deny  from 192.168.65.128

                                           用来控制范围

                             

                                    客户端地址:

                                           IP

                                           网络:     172.16

                                                         172.16.0.0

                                                         172.16.0.0/16

                                                         172.16.0.0/255.255.0.0

                             

               9     日志设定      

                     (1)   错误日志:

                                           ErrorLog   logs/error_log

                            错日志记录级别:

                                           loglevel warn (默认值)

                            可选loglevel:

                                           debug,info,notice,warn,error,crit,alert,emerg

                                           级别越低(左)记录越详细

                    

                     (2)访问日志      

                            定义日志格式:

                                           LogFormat format strings

                            默认:LogFormat "%l %h %u %t \"%r\" %>s %b \"%{Referer}i\"  

                                    \"%{User-Agent}i\"" combined

                             

                            使用日志格式:

                                           Customlog logs/access_log combined

                            参考帮助:

                                           http://httpd.apache.org/docs/2.2/mod/mod_log_config.html#formats

                            常用格式参数:

                                   %h   客户端IP地址

                                   %l  远程用户,启用mod_ident才有效,通常为空"-"

                                   %u   验证(basic,digest)远程用户,非登录访问时,为"-"

                                   %t    服务器收到请求时的时间

                                   %r   First line of  request,即表示请求报文的首行,记录了此次请求

                                           的方法 URL 协议版本

                                   %>s  相应状态码

                                   %b   相应报文的大小,单位是字节;不包括响应报文http首部

                                   %{Referer}i      请求报文中首部referer的值;即从哪个页面中的超链接跳转

                                                                至当前页面的

                                    %{User-Agent}i      请求报文中首部"User-Agent"的值;即发出请求的应用程序                

                             

                             修改日志的时间格式:

                                   %{%F %T}t  

               

               

               10    显示服务器版本信息:

                     对方浏览器打开页面之后使用F12 可以看见服务器版本号

                                    ServerTokens:Major|Minor|Min[imal]|Prod[uctOnly]|OS|Full

                                   ServerTokens  Prod[uctOnly] :Server: Apache

                                   ServerTokens  Major: Server: Apache/2

                                   ServerTokens  Minor: Server: Apache/2.0

                                   ServerTokens  Min[imal]: Server: Apache/2.0.41

                                   ServerTokens  OS: Server: Apache/2.0.41 (Unix)

                                   ServerTokens  Full (or not specified): Server: Apache/2.0.41 (Unix) PHP/4.2.2  

                                    MyMod/1.2

                                   This setting  applies to the entire server and cannot be enabled or disabled on a  

                                    virtualhost-by-virtualhost   basis.

                                   After version  2.0.44, this directive also controls the information presented by the  

                                    ServerSignature directive.

                     建议使用:ServerTokens Prod    

               11    显示默认字符集:

                            AddDefaultCharset  UTF-8

               12    定义路径别名:

                            格式:    Alias /URL/ "/PATH/"  

                                           注意前后"/"必须一致,前面有,后面就必须有

                            Documentroot  "/data/www"

                                           http://www.wow.com/download/f1

                                          ==>  /data/www/download/f1

                                           

                            Alias /download/  "/app/dir1"

                                           http://www.wow.com/download/f1

                                          ==>  /app/dir1/f1

               13    基于用户的访问控制

                            认证质询:WWW-Authenticate,响应码为401,拒绝客户端请求,并说明要求客户端

                                           提供账号和密码

                            认证:     Authorization,客户端用户填入账号和密码后再次发送请求报文,认证通过

                                           时,则服务器发送响应的资源

                            认证方式两种:

                                           basic:明文

                                           digest:消息摘要认证,兼容性差

                             安全域:需要用户认证后方能访问的路径,应该通过名称对其进行标识,以便于告知用户

                                           认证的原因

                             虚拟账号:仅用于访问某服务时用到的认证标识,区分于linux用户账号

                             账号密码存储:文本文件,SQL数据库,ldap目录存储,nis等

                             

                     basic认证配置示例:

                            (1)定义安全域:

                                   <Directory  "/path">

                                           options none

                                           AllowOverride none

                                           AuthType basic

                                           AuthName "string"

                                           AuthUserFile "path/file"

                                           Require user username1 usename2...

                                    <directory>

                                    允许账号文件中的所有用户登录访问:

                                           Require valid-user

                                           

                            (2)提供账号和密码存储,(文本文件方式)

                                    使用专用的命令完成此类文件的创建和用户管理

                                                  htpasswd [options] /PATH/FILE username

                                    选项:

                                                  -c    自动创建文件,仅应该在文件不存在时使用,like  >

                                                  -p   明文密码

                                                  -d   crypt格式加密,默认

                                                  -m md5格式加密

                                                  -s   sha格式加密

                                                  -D 删除指定用户

                    

                     基于组账号进行认证:

                            (1)定义安全域

                                   <Directory  "/path">

                                           options none

                                           allowoverload none

                                            authtype basic

                                           authname "string"

                                           authuserfile "/path/file"

                                           authgroupfile "/path/groupfile"

                                           require group groupname1 groupname2 ...

                                    </Directory>

                             

                            (2)创建用户账号和组账号文件

                                    组文件:每一行定义一个组

                                   groupname:  username1 username2 ...

                                    

                            注意:centos7定义require group 之后可以单独定义 require user

                                     centos6定义require  group 之后再定义require user 会使group失效

                    

                     远程客户端和用户验证的控制

                            satisfy  All | Any

                                          All  客户机IP和用户验证都需要通过才可以

                                          Any  客户机IP和用户验证,有一个满足即可

                     示例:

                                   require  valid-user

                                   order  allow,deny

                                   allow from  192.168.1

                                   satisfy  any

                                    

                     注意:如果不加satisfy any 这一行,默认效果为satisfy  all

                              即:全都需要满足才可以

               

               

               14    实现用户家目录的http共享

                                    基于模块mod_userdir.so实现

                                    SElinux:http_enable_homedirs

                            相关设置:

                                   vim  /etc/httpd/conf/httpd.conf

                                   <ifmodule  mod_userdir.c>

                                           #userdir disabled

                                           userdir public_html   指定共享目录的名称

                                    </ifmodule>

                            准备目录:

                                          su -  hello;mkdir ~/public_html

                                           setfacl -m u:apache:x ~hello

                            访问:

                     不懂,待续。。。。 

                    

               

               15    ServerSignature ON | OFF |  Email

                             当客户请求的网页并不存在时,服务器将产生错误文档,缺省情况下由于打开了

                             ServerSignature选项,错误文档的最后一行将包含服务器的名字、apache的版本

                            等信息

                            例如:

                                          Not  Found

                                          The  requested URL /dd was not found on this server.

                                           Apache/2.2.15 (CentOS) Server at 192.168.65.150 Port  80

                                                                

                             如果不对外显示这些信息,就可以将这个参数设置为OFF

                            Centos  6有效,7无此参数,且不显示此信息

                    

               16    status页面

                            需开启模块:

                                   LoadModule  status_module modules/mod_status.so

                                   <location  /server-status>

                                                  SetHandler server-status

                                                  Order allow,deny

                                                  allow from 192.168.65                                                                

                                    </location>

                                    Extendedstatus ON

                     查看页面:   

                                    192.168.65.150/server-status

                     认证方式:

                                    可以同时使用基于账号密码认证

               

               17    虚拟主机

                            站点标识:    socket  (IP:PORT)

                    

                     三种实现方式:

                            基于ip:为每个虚拟主机准备至少一个IP地址,基于ip报文头部识别

                            基于port:为每个虚拟主机使用至少一个独立的port,基于tcp报文头部识别

                            基于FQDN:为每个虚拟主机使用至少一个FQDN,基于http数据头部来识别

                    

                     注意:一般虚拟机不要与main主机混用;因此,要使用虚拟主机,一般禁用main主机

                     禁用方法:注释中心主机的 Documentroot 指令

                     虚拟主机的配置方法:

                                    <VirtualHost IP:PORT>

                                           servername FQDN

                                           Documentroot "/path"

                                    </VirtualHost>

                                    建议:将上述配置存放在独立的配置文件中

                    

                     其他可用指令:

                                    ServerAlias:虚拟主机的别名,可多次使用

                                    errorlog

                                    customlog

                                   <directory  "/path">

                                           ...

                                    </directory>

                                    alias

                    

                     配置示例:

                     基于ip和port实现虚拟主机

                                    <VirtualHost 192.168.65.150:80>

                                                  DocumentRoot "/app/web1/"

                                                  servername web1.wow.com

                                                  errorlog "/var/log/httpd/web1.error_log"

                                                  customlog "/var/log/httpd/web1.access_log" combined

                                    </VirtualHost>

                                    <VirtualHost 192.168.65.151:80>

                                                  DocumentRoot "/app/web2/"

                                                  servername web2.wow.com

                                                  errorlog "/var/log/httpd/web2.error_log"

                                                  customlog "/var/log/httpd/web2.access_log" combined

                                    </VirtualHost>

                                    <VirtualHost 192.168.65.151:8080>                                                            

                                                  servername web3.wow.com

                                                  DocumentRoot "/app/web3/"

                                                  errorlog "/var/log/httpd/web3.error_log"

                                                  customlog "/var/log/httpd/web3.access_log" combined

                                    </VirtualHost>                                   

                     基于FQDN实现虚拟主机:

                                    NameVirtualHost 192.168.65.150:80                                                            

                                    <VirtualHost 192.168.65.150:80>

                                                  servername web1.wow.com

                                                  DocumentRoot "/app/web1/"

                                                  errorlog "/var/log/httpd/web1.error_log"

                                                  customlog "/var/log/httpd/web1.access_log" combined

                                    </VirtualHost>

                                    <VirtualHost 192.168.65.150:80>

                                                  servername web2.wow.com

                                                  DocumentRoot "/app/web2/"

                                                  errorlog "/var/log/httpd/web2.error_log"

                                                  customlog "/var/log/httpd/web2.access_log" combined

                                    </VirtualHost>

                                    <VirtualHost 192.168.65.150:80>

                                                  servername web3.wow.com

                                                  DocumentRoot "/app/web3/"

                                                  errorlog "/var/log/httpd/web3.error_log"

                                                  customlog "/var/log/httpd/web3.access_log" combined

                                    </VirtualHost>

        

                    

                     注意:    centos6基于FQDN必须加

                                           NameVirtualHost *:80   

                                    

                                    centos7添加虚拟主机需要对每个目录添加授权

                                           <directory "/app/">

                                                  require all granted

                                           </directory>

        

mod_deflate模块:

               使用mod_deflate模块压缩页面又换传输速度

               使用场景:

                     (1)节约带宽,额外消耗cpu;同时,可能有些较老浏览器不支持

                     (2)压缩适于压缩的资源,例如文本文件

                    

               实现步骤:

                     1     确认mod_deflate处于开启状态

                            LoadModule  deflate_module modules/mod_deflate.so SetOutputFilter  DEFLATE

                    

                     2     添加如下信息到配置文件的任意位置

                             

                            Setoutputfilter  DEFLATE

                            # mod_deflate  configuration

                            # Restrict  compression to these MIME types

                             AddOutputFilterByType DEFLATE text/plain

                             AddOutputFilterByType DEFLATE text/html

                             AddOutputFilterByType DEFLATE application/xhtml+xml

                             AddOutputFilterByType DEFLATE text/xml

                             AddOutputFilterByType DEFLATE application/xml

                             AddOutputFilterByType DEFLATE  application/x-javascript

                             AddOutputFilterByType DEFLATE text/javascript

                             AddOutputFilterByType DEFLATE text/css

                            #Level of  compression (Highest 9 - Lowest 1)

                             DeflateCompressionLevel 9

                            # Netscape 4.x  只压缩text/html

                            BrowserMatch  ^Mozilla/4 gzip-only-text/html

                            #Netscape  4.06-08三个版本 不压缩

                            BrowserMatch  ^Mozilla/4\.0[678] no-gzip

                            #Internet  Explorer标识本身为“Mozilla / 4”,但实际上是能够处理请求的压缩。

                             #如果用户代理首部匹配字符串“MSIE”(“B”为单词边界”),就关闭之前定

                            #义的限制

                            BrowserMatch  \bMSI[E] !no-gzip !gzip-only-text/html

                    

                     3     重新装载配置文件

                            systemctl reload  httpd

        

               测试:

                            curl -I  wb1.wow.com

                            curl -I wb1.wow.com  --compressed

        

        

https 通信过程:

               https:http over ssl

               https通信过程

                     1     建立tcp连接

                             客户端发起请求,建立到服务器端口443的tcp连接,三次握手

                     2     建立ssl会话

                            (1)客户端发送可供选择的加密方式,并向服务器请求证书

                            (2)服务器端发送证书以及选定的加密方式给客户端

                            (3)客户端取得证书并进行证书验证

                                    如果信任给其发证书的CA

                                   (a)  验证证书来源的合法性,用CA的公钥解密证书上数字签名

                                   (b)  验证证书的内容的合法性:完整性检验

                                           即使用证书中给定的单向加密算法加密证书内容生成指纹,并与解密

                                           数字签名之后得到的指纹对比是否一致

                                   (c)    检查证书的有效期限

                                   (d)  检查证书是否被吊销

                                   (e)  证书中拥有者的名字,与访问的目标主机是否一致

                            (4)客户端生成临时会话密钥(对称密钥),并使用服务器端的公钥加密此数据发送

                                    给服务器,完成秘钥交换

                            (5)服务器用此密钥加密用户请求的资源,响应给客户端

                     3     事务处理完毕之后,先断开ssl连接,在断开tcp连接

               

               注意事项:

                            SSL是基于IP地址实现,单IP的主机仅可以使用一个https虚拟主机

                             

https 实现:

               1     为服务器申请数字证书

                            建立私有CA

                             在服务器创建证书签署请求

                            CA签证

        

               2     配置httpd支持使用ssl,及使用的证书

                     安装模块:   

                                   yum install  mod_ssl -y

                     修改配置文件:   

                                   vim  /etc/httpd/conf.d/ssl.conf

                                    DocumentRoot

                                    ServerName

                                    SSLCertificateFile           服务器证书文件路径

                                    SSLCertificateKeyFile      服务器私钥文件

                    

               3     测试基于https访问相应的主机

                            openssl  s_client   [-connect host:port] [-cert filename]

                                   [-CApath  directory] [-CAfilefilename]

                    

                     使用curl命令测试:

                     对比:

                                   curl  https://wb1.wow.com 

                                   curl --cacert  cacert.pem   https://wb1.wow.com

                             

               4     http重定向https

                                   将http请求转发至https的URL

                     重定向:

                                   Redirect  [status] URL -path URL

                      status:

                            1     permanent:returns a permanent redirect  status(301) indicating that

                                   the resource  has moved permanently

                            2     temp:returns a temporary redirect  status(302).this is the default

                    

                     示例:

                                   Redirect temp  / https://www.testwow.com

                     注意:   

                                    如果定义了虚拟机提供https服务,再定义重定向"/"到 https

                                    会出现循环重定向,无法正常打开页面

                                    如下:

                                           <virtualhost *:80>

                                                         servername web1.wow.com

                                                         documentroot "/data/www/wb1"

                                                         errorlog "/var/log/httpd/wb1_error_lor"

                                                         customlog "/var/log/httpd/wb1_log" combined

                                           </virtualhost>

                                           <virtualhost *:80>

                                                         servername wb2.wow.com

                                                         documentroot "/data/www/wb2"

                                                         errorlog "/var/log/httpd/wb2_error_log"

                                                         customlog "/var/log/httpd/wb2_log" combined

                                           </virtualhost>                                    

                                           redirect temp / https://wb1.wow.com/

                                    

                                    此段代码,wb1实现了https,如果注释掉redirect行

                                                  #redirect temp / https://wb1.wow.com/

                                    那么测试wb1.wow.com wb2.wow.com https://wb1.wow.com  均正常

                                    但是去掉redirect行的注释之后,所有页面均无法正常访问

        

HSTS:

               HSTS: http strict transport  security

               服务器端配置支持HSTS后,会在给浏览器返回的HTTP首部中携带HSTS字段,浏览器获取到该信息

               后,会将所有的HTTP访问请求在内部做307跳转到HTTPS。而无需任何网络过程

               

               HSTS preload list

                     是chrome浏览器中的HSTS预载入列表,在该列表中的网站,使用chrome浏览器访问

                     时,会自动转换成https。

                      firefox,safari,edge浏览器也会采用这个列表

                    

                                    

               实现HSTS:

                            vim  /etc/httpd/conf/httpd.conf

                            添加如下行即可

                            Header always set  Strict-Transport-Security "max-age=31536000"

                            RewriteEngine  on

                            RewriteRule ^(/.*)$  https://%{HTTP_HOST}$1 [redirect=302]                      

                                    

httpd自带的工具程序:                                  

               htpasswd:basic认证基于文件实现时,用到的账号密码文件生成工具

               apachectl:       httpd自带的服务控制脚本,支持start和stop

               rotalelogs:日志滚动工具

               

httpd的压力测试工具:

               ab,webbench,http_load,seige

               jmeter     开源

               loadrunner    商业,有认证

               tcpcopy 网页,复制生产环境中的真实请求,并将之保存

               

               ab命令的用法:

                            来自 httpd-tools 包

                            ab [options]  URL

                                   -n   请求总数

                                   -c   模拟的并行数

                                   -k   以持久连接模式测试

               示例

                            ab -c 100 -n 10000  http://www.testwow.com/

               注意:

                            URL  后面需加上 "/"

               ulimit -n #     调整能打开的文件数

                             

        

        

        

httpd 2.4  :----------------------------------------------------------------------

        

httpd2.4 新特性:

                            MPM支持运行为DSO机制;以模块形式按需加载

                            event MPM  生产环境可用

                            异步读写机制

                             支持每模块及每目录的单独日志级别定义

                             每请求相关的专用配置

                             增强版的表达式分析器

                             毫秒级持久连接时长定义

                            基于FQDN的虚拟主机不需要NameVirtualHost 指令

                            新指令,allowoverridelist

                             支持用户自定义变量

                            更低的内存消耗

               

                             修改了一些配置机制:

                                           不再支持使用order deny,allow  来做基于IP的访问机制

                            新模块:

                                          1     mod_proxy_fcgi

                                           FastCGI Protocol backend for mod_proxy

                                          2     mod_remoteip

                                           Replaces the apparent client remote IP address and hostname for the  request

                                          with  the IP address list presented by a proxies or a load balancer via the request  

                                           headers.

                                          3     mod_ratelimit

                                           Provides Bandwidth Rate Limiting for Clients

                                           

               安装:

                            centos7:yum安装,源码编译安装

                             centos6:源码编译

               

               Rpm安装程序环境:

                     配置文件:

                             /etc/httpd/conf/httpd.conf

                             /etc/httpd/conf.d/*.conf

                     模块相关的配置文件:          

                             etc/httpd/conf.modules.d/*.conf

                            systemd unit file:

                             /usr/lib/systemd/system/httpd.service

                     主程序文件:

                             /usr/sbin/httpd

                             httpd-2.4支持MPM的动态切换             

                     日志文件:

                             /var/log/httpd

                             access_log:访问日志

                             error_log:错误日志

                     站点文档:

                             /var/www/html

                     模块文件路径:

                             /usr/lib64/httpd/modules

                     服务控制:

                            systemctl  enable|disable  httpd.service

                            systemctl  {start|stop|restart|status}  httpd.service

httpd-2.4配置:

               1     切换使用的MPM

                     centos7 rpm安装

                            vim  /etc/httpd/conf.modules.d/00-mpm.conf

                            启用要启用的MPM相关的LoadModule指令即可

                     centos6编译安装:

                            vim  /etc/httpd24/httpd.conf

                            include  /etc/httpd24/extra/httpd-mpm.conf

                            LoadModule  mpm_event_module modules/mod_mpm_event.so

                             

               2     主目录

                            Documentroot  /path

                             

               3     基于IP的访问控制

                             无明确授权的目录,拒绝访问

                             允许所有主机访问:    require all granted

                             拒绝所有主机访问:     require all denied

                            控制特定的IP访问:

                                           Require ip IPADDR:   授权指定来源的IP访问

                                           Require not ip IPADDR:拒绝特定的IP访问

                             控制特定的主机访问:

                                           Require host  HOSTNAME        授权特定主机访问

                                           Require not host HOSTNAME     拒绝

                             HOSTNAME:

                                           FQDN:    特定主机

                                           domain.tld      指定域名下的所有主机

                    

                      不能有失败,至少有一个成功匹配才成功,即失败优先:

                     匹配全部条件方可访问

                                    <RequireAll>

                                           require all granted

                                           require not ip 172.16.1.1       拒绝特定主机

                                    </RequireAll>

                      多个语句有一个成功,则成功,即成功优先:

                     匹配任意条件即可访问:

                                    <RequireAny>

                                           require all denied

                                           require ip 172.16.1.1      允许特定IP

                                    </RequireAny>

                                           

               4     虚拟主机:   

                            基于FQDN的虚拟主机不再需要NameVirutalHost指令

                             任意目录下的页面只有显式授权才能被访问

                                           

               5     ssl:

                            安装ssl_mod,和httpd2.2相同

               

               6     持久连接功能默认启用,5  100

                     KeepAlive  on

                             KeepAliveTimeout   #ms

                            MaxKeepAliveRequests  100

                             毫秒级持久连接时长定义

                    

        

                                           

http协议进阶:-------------------------------------------------------------------

http报文语法格式:

               reuqest报文:

                            <method>  <request-URL> <version>

                             <headers>

                             

                             <entity-body>

               reponse报文:

                            <version>  <status> <reason-phrase>

                             <headers>

                             

                             <entity-body>

               

               method:请求方法,标明客户端希望服务器对资源执行的动作

                             GET,HEAD,POST等

                             

               version:

                             HTTP/<major>.<minor>

               

               status:     状态码,如200,301,302,404,502;

                             标记请求处理过程中发生的情况

               

               reason-phrase:

                             状态码所标记的状态的简要描述

               

               headers:

                             每个请求或响应报文可包含任意个首部;每个首部都有首部名称,后面跟

                             一个冒号,而后跟一个可选空格,接着是一个值

                                           首部字段名: 值

               entity-body:

                             请求时附件的数据或响应时附加的数据

                             

method:方法

               GET: 从服务器获取一个资源

               HEAD:   只从服务器获取文档的响应首部

               POST:   向服务器输入数据,通常会再由网关程序继续处理

               PUT:      将请求的主体部分存储在服务器中,如上传文件

               DELETE:请求删除服务器上指定的文档

               TRACE:    追踪请求到达服务器中间经过的代理服务器

               OPTIONS:请求服务器返回指定资源吃食使用的请求方法

               

协议查看或分析的工具:

               tcpdump,wireshark,tshark

               

http协议状态码分类:

               1xx  100-101         信息提示

               2xx  200-206         成功

               3xx  300-305         重定向

               4xx 400-415           错误类信息,客户端错误

               5xx 500-505           错误类信息,服务器端错误

               

http协议常用的状态码:

               200: 成功,请求数据通过相应报文的entity-body部分发送;OK

               301:      请求的URL指向的资源已经被删除;但在响应报文中通过首部Location指明了

                             资源现在所处的新位置,Moved permanenty

               302:      响应报文location指明资源临时新位置  moved temporarily

               304:      客户端发出了条件式请求,但服务器上的资源未曾发生改变,则通过此响应

                             转态码通知客户端,not modified

               401:      需要输入账号和密码认证方能访问资源;unauthorized

               403:      请求被禁止;forbidden

               404:      服务器无法找到客户端请求的资源,not found

               500:      服务器内部错误,internal server  error

               502:      代理服务器从后端服务器收到了一条伪响应,如无法连接到网关,bad  gateway

               503:      服务不可用,临时服务器维护或过载,服务器无法处理请求

               504         网关超时

               

headers:

               request 报文头部示例:

                            Accept   

                             text/css,*/*;q=0.1

                            Accept-Encoding 

                            gzip, deflate,  br

                            Accept-Language

                             zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2

                            Cache-Control     

                             max-age=0

                            Connection   

                             keep-alive

                            Cookie   

                             MySQL_S=nr39nu9jdjkr0u2i1qmi0l…174105336;  s_sq=%5B%5BB%5D%5D

                            Host

                             labs.mysql.com

                             If-Modified-Since   

                            Tue, 05 Jun 2018  17:50:40 GMT

                            Referer  

                             https://dev.mysql.com/doc/refm…rce-configuration-options.html

                            User-Agent   

                            Mozilla/5.0 (Windows  NT 6.3; W…) Gecko/20100101 Firefox/61.0

               

               response报文头部示例:

                            Cache-Control     

                            no-store, no-cache,  must-revalidate

                            Cache-Control     

                            no-cache,  private

                            Connection   

                             Keep-Alive

                             Content-Encoding

                             gzip

                            Content-Length   

                             32057

                            Content-Type      

                            text/html;  charset=UTF-8

                            Date      

                            Thu, 28 Jun 2018  09:30:09 GMT

                            Expires   

                            Thu, 19 Nov 1981  08:52:00 GMT

                            Keep-Alive    

                            timeout=5,  max=10

                            Pragma 

                             no-cache

                            Server    

                             Apache

                             Strict-Transport-Security      

                             max-age=15768000

                            Vary

                             Accept-Encoding

                             X-Content-Type-Options     

                             nosniff

                            X-Frame-Options

                             SAMEORIGIN

                             X-XSS-Protection   

                            1;  mode=block

http首部字段:

               HTTP首部字段包含的信息最为丰富,首部字段同时存在于请求和响应报文内,

               并涵盖http报文相关的内容信息。使用首部字段是为了给客户端和服务器端

               提供报文主体大小,所使用的语言,认证信息等内容

               

               首部字段结构:     HTTP首部字段是由首部字段名和字段值构成的,中间用

                                           ":"分隔

               

               字段值对应单个HTTP首都字段可以有多个值

               

               报文首部中出现了两个或以上具有相同首部字段名的首部字段时,在规范内

               尚未明确,根据浏览器内部处理逻辑的不同,优先处理的顺序可能不同,结果

               可能并不一致

               

首部的分类:

               通用首部:请求报文和响应报文两方都会使用的首部

                            date:           报文的创建时间

                             connection:连接状态,如keep-alive,close

                            via:         显示报文经过的中间节点(代理,网关)

                             chche-control:     控制缓存,如缓存时长

                             MIME-Version:    发送端使用的MIME版本

                            Warning:     错误通知

                             

               请求首部:从客户端向服务器端发送报文时使用的首部。补充了请求的附加内容、

                              客户端信息,请求内容相关优先级等信息

                             Accept:通知服务器自己可接受的媒体类型

                             accept-charset:客户端可接受字符集

                             accept-Encoding:客户端可接受的编码格式,如gzip

                             accept-Language:客户端可接受的语言

                             

                             Client-IP:      请求的客户端IP

                            Host:            请求的服务器名称和端口号

                            Referer:       跳转至当前URI的前一个URL

                             User-Agent:客户端代理,浏览器版本

                             

               响应首部:从服务器端向客户端返回响应报文时使用的首部。补充了响应的附加

                              内容,也会要求客户端附加额外的内容信息

                     1     信息性:

                                    Age:从最初创建开始,响应持续时长

                                    server:服务器程序软件名称和版本

                     2     协商首部:

                                    accept-ranges:服务器可接受的请求范文类型

                                    vary:     服务器查看的其他首部列表

                     3     安全响应首部:

                                    set-cookie:向客户端设置cookie

                                    set-cookie2:与上面相似

                                    www-authenticate:来自服务器对客户端的质询列表

                             

               实体首部:针对请求报文和响应报文的实体部分使用的首部。补充了资源内容更新

                              时间等与实体相关的信息

                            Allow:  列出对此资源实体可使用的请求方法

                             Location:告诉客户端真正的实体位于何处

                            Content  -Encoding:对主体执行的编码

                            Content  -Language:理解主体时最适合的语言

                            Content -Length:  主体的长度

                            Content -Location:  实体真正所处位置

                            Content  -Type:主体的对象类型,如text

                            缓存相关:

                             ETag:实体的扩展标签

                             Expires:实体的过期时间

                            Last  -Modified:最后一次修改的时间

               扩展首部:

        

               条件式请求首部:

                             Expect:允许客户端列出某请求所要求的服务器行为

                             if-modified-since:自从指定的时间之后,请求的资源是否发生过修改

                             if-unmodified-since:与上面相反

                             if-none-match:本地花村中存储的文档etag标签是否与服务器文档的etag

                                           不匹配

                             if-match:与上面相反

               

               安全请求首部:

                            authorization:  向服务器发送认证信息,如账号和密码

                             cookie:客户端向服务器发送cookie

                             cookie2:用于说明请求支持的cookie 版本

                             

               代理请求首部:

                             proxy-authorization:向代理服务器认证

                             

cookie:

               HTTP 是一种无状态协议。

               协议自身不对请求和响应之间的通信状态进行保存。

               也就是说在 HTTP  这个级别,协议对于发送过的请求或响应都不做持久化处理。

               这是为了更快地处理大量事务,确保协议的可伸缩性,而特意把 HTTP 协议设

               计成如此简单的。

               可是随着 Web 的不断发展,很多业务都需要对通信状态进行保存。

               于是引入了 Cookie 技术。

               使用 Cookie 的状态管理Cookie  技术通过在请求和响应报文中写入 Cookie

               信息来控制客户端的状态。

               Cookie 会根据从服务器端发送的响应报文内的一个叫做 Set -Cookie  的首部字段

               信息,通知客户端保存Cookie。

               当下次客户端再往该服务器发送请求时,客户端会自动在请求报文中加入 Cookie  

               值后发送出去。

               服务器端发现客户端发送过来的 Cookie 后,

               会去检查究竟是从哪一个客户端发来的连接请求,然后对比服务器上的记录,

               最后得到之前的状态信息

Set -Cookie首部字段:

               Set -cookie首部字段示例:

                     Set -Cookie: status=enable;  expires=Fri, 24 Nov 2017 20:30:02 GMT; path=/;

               NAME=VALUE 

                     赋予  Cookie 的名称和其值,此为必需项

               expires=DATE   

                     Cookie  的有效期,若不明确指定则默认为浏览器关闭前为止

               path=PATH 

                     将服务器上的文件目录作为Cookie的适用对象,若不指定则默认为文档所在的文件目录

               domain=域名   

                     作为  Cookie 适用对象的域名,若不指定则默认为创建Cookie的服务器的域名

               Secure 

                     仅在  HTTPS 安全通信时才会发送 Cookie

               HttpOnly 

                     加以限制使 Cookie 不能被 JavaScript 脚本访问

curl工具:----------------------------------------------------------------------

               curl是基于URL语法在命令行方式下工作的文件传输工具

               它支持FTP,FTPS,HTTP,HTTPS,GOPHER,TELNET,DICT,FILE,LDAP等协议。

               支持HTTPS认证,并且支持HTTP的POST,PUT等方法。FTP上传,kerberos认证,HTTP上传,

               代理服务器,cookies,用户名/密码认证,下载文件断点续传,上载文件断点续传,

               HTTP代理服务器管道(proxy tunnneling),还支持IPv6,socks5代理服务器

               通过http代理服务器上传文件到FTP服务器等,功能十分强大

               

               使用格式:

                            curl [options]  [URL...]

               选项:

                             -A,--user-agent<sting>             设置用户代理发送给服务器

                            -e,--referer  <URL>                    来源网址

                            --cacert  <file>                    CA证书(ssl)

                            -k,--insecure                      允许忽略证书进行SSL连接

                            --compressed                            要求返回是压缩的格式

                            -H,--header  <line>                    自定义首部信息传递给服务器

                            -i                                                显示页面内容,包括报文首部信息

                            -I,--head                                    只显示响应报文首部信息

                            -D,--dump-header <file>        将URL的header信息存放在指定文件中

                            --basic                              使用HTTP基本认证

                            -u,--user  <user:[password]> 输入服务器的用户和密码

                            -L                                              如果有3xx响应码,重新发送请求到新位置

                            -O                                              使用URL中默认的文件名保存文件到本地

                            -o <file>                              将网络文件保存为指定的文件中

                            --limit-rate  <rate>                     设置传输速度

                            -0,--http1.0                        数值0,使用HTTP1.0

                            -v,--verbose                      更详细

                            -C                                              选项可对文件使用断点续传功能

                            -c,--cookie-jar  <file-name> 将url中cookie存放在指定文件中

                            -x,--proxy  <proxyhost[:port]> 指定代理服务器地址

                            -X,--request  <command>          向服务器发送指定请求方法

                            -U,--proxy-user  <user:password> 代理服务器用户和密码

                            -T                                              选项将指定的本地文件上传到FTP服务器上

                            --data,-d                                    指定使用POST方式传递数据

                            -b name=data                           从服务器响应set-cookie得到值,返回给服务器

               

               示例:

                     使用basic认证:

                            curl --basic --user  user5:centos 192.168.65.132/download/

                    

                     伪造agent和referer

                            curl -A  "sogoliulanqi" -e "www.baidu.com" 192.168.65.132

                             access_log记录:

                            192.168.65.155 - -  [01/Jul/2018:11:45:39 +0800] "GET / HTTP/1.1"

                                   200 175  "www.baidu.com" "sogoliulanqi"

                     下载并限速:

                            curl  192.168.65.132/mariadb -o mariadb-server --limit-rate  1024000

                    

elikes工具:

               elinks [options] [url]...

                     -dump          非交互式模式,将url的内容输出至标准输出

                     -source         打印源码

Sentfile机制:

               不用 sendfile 的传统网络传输过程:

                      read(file, tmp_buf ,  len)

                      write(socket, tmp_buf ,  len)

               硬盘 >> kernel buffer >> user buffer  >> kernel socket buffer >> 协议栈

               

               一般网络应用通过读硬盘数据,写数据到 socket 来完成网络传输,底层执行过程:

                  1     系统调用 read()  产生一个上下文切换:从 user mode 切换到 kernel  mode,然后 DMA 执行拷贝,

                             把文件数据从硬盘读到一个 kernel buffer 里。

                  2     数据从 kernel buffer  拷贝到 user buffer,然后系统调用 read()  返回,这时又产生一个上下文切换:

                            从kernel mode 切换到 user  mode

                  3     系统调用 write()  产生一个上下文切换:从 user mode 切换到 kernel  mode,然后把步骤2读到

                            user buffer  的数据拷贝到 kernel buffer(数据第2次拷贝到 kernel buffer),不过这次是个不同的

                            kernel  buffer,这个 buffer和 socket  相关联。

                4       系统调用 write() 返回,产生一个上下文切换:从 kernel mode  切换到 user mode(第4次切换),然后

                            DMA从 kernel buffer 拷贝数据到协议栈(第4次拷贝)

               

               上面4个步骤有4次上下文切换,有4次拷贝,如果能减少切换次数和拷贝次数将会有效提升性能   

               在kernel 2.0+ 版本中,系统调用  sendfile() 就是用来简化上面步骤提升性能的。

               

               sendfile() 不但能减少切换次数而且还能减少拷贝次数

               

               用 sendfile() 来进行网络传输的过程:

                sendfile(socket, file, len);

                硬盘 >> kernel buffer (快速拷贝到kernel socket buffer) >> 协议栈

                1       系统调用 sendfile() 通过 DMA 把硬盘数据拷贝到 kernel buffer,然后数据被

                            kernel  直接拷贝到另外一个与   socket 相关的 kernel buffer。这里没有 user mode 和

                            kernel mode  之间的切换,在 kernel 中直接完成了从一个 buffer  到另一个 buffer 的

                            拷贝。

                2       DMA 把数据从 kernel buffer 直接拷贝给协议栈,没有切换,也不需要数据从  user

                            mode 拷贝到 kernel mode,因为数据就在  kernel 里

               配置选项:     

                            EnableSendfile  on

                             httpd2.4默认启用

反向代理功能:

               启用反向代理:

                     ProxyPass "/"  "http://www.example.com/"

                     ProxyPa***everse "/"  "http://www.example.com/"

               特定url反向代理:

                     proxypass "/images"  "http://www.example.com/"

                     proxypa***everse "/images"  "http://www.example.com/"

               示例:

                            proxypass  "/homepage" "http://www.testwow.com"

                            proxypa***everse  "/homepage" "http://www.testwow.com"

                     测试结果:

                            curl  wb1.wow.com

                            curl  wb1.wow.com/homepage

               注意:

                             后端服务器访问日志中客户端为代理服务器

                             且代理服务器也会记录日志,客户端为发起请求的用户

--------------------------------------------------------------------------------

编译安装httpd-2.4

ARP项目:

               APR(Apache portable Run-time libraries,Apache可移植运行库) 主要为上

               层的应用程序提供一个可以跨越多操作系统平台使用的底层支持接口库。在早

               期的Apache版本中,应用程序本身必须能够处理各种具体操作系统平台的细节,

               并针对不同的平台调用不同的处理函数

            

               随着Apache的进一步开发,Apache组织决定将这些通用的函数独立出来并发

               展成为一个新的项目。这样,APR的开发就从Apache中独立出来,Apache仅

               仅是使用 APR而已。目前APR主要还是由Apache使用,由于APR的较好的移植

               性,因此一些需要进行移植的C程序也开始使用APR,开源项目比如用于服务器

               压力测试的Flood loader tester,该项目不仅仅适用于Apache,

               http://httpd.apache.org/test/flood

在centos6上编译安装httpd-2.4

安装前准备:

               1     安装http-2.4

                     依赖于apr-1.4+,apr-util-1.4+,[apr-iconv]

                     apr: apache portable  runtime,     解决跨平台实现

                     centos6.9:默认安装为apr1.3.9,apr-util-1.3.9

               

               2     安装前准备开发包

                            yum groupinstall  "Development Tools" "Server Platform Development"

                     相关包:

                            yum install  pcre-devel openssl-devel expat-devel

               

               3     下载源代码并解压缩

                             httpd-2.4.33.tar.bz2

                             apr-1.6.3.tar.gz

                             apr-util-1.6.1.tar.gz

               

编译安装方法一:

               1     安装apr-1.4+

                            cd  apr-1.6.3

                            ./configure  --prefix=/app/apr

                            make && make  install

               

               2     安装apr-util-1.4+

                            cd  ../apr-util-1.6.1

                            ./configure  --prefix=/app/apr-util --with-apr=/app/apr

                            make -j 2 &&  make install

                             

               3     安装httpd2.4

                            cd  ../httpd-2.4.33

                            ./configure  --prefix=/app/httpd24 --enable-so \

                            --enable-ssl  --enable-cgi --enable-rewrite \

                            --with-zlib  --with-pcre --with-apr=/app/apr/ \

                             --with-apr-util=/app/apr-util/ \

                            --enablemodules=most  --enable-mpms-shared=all \

                             --with-mpm=prefork

                            make -j 4 &&  make install

编译安装方法二:

               1     cp -av apr-util-1.6.0  httpd-2.4.27/srclib/apr -util

                     cp -av apr-1.6.2  httpd-2.4.27/srclib/apr

               2     cd  httpd-2.4.33

                     ./configure  --prefix=/appl/httpd24 \

                     --enable-so --enable-ssl  --enable-cgi \

                     --enable-rewrite  --with-zlib --with-pcre \

                     --with-includedapr  --enable-modules=most \

                     --enable-mpms-shared=all  --withmpm=prefork

               

               3     make && make  install

编译安装http-2.4程序环境:

               自带的服务控制脚本:

                             /app/httpd24/bin/apachectl  start|stop|restart|status

               应用程序目录:/app/httpd24/bin/

               网页文件目录:/app/httpd24/htdocs

               

               自定义启动脚本:(参考httpd-2.2的服务脚本)

                     cp /etc/rc.d/init.d/httpd  /etc/rc.d/init.d/httpd24

                     vim  /etc/rc.d/init.d/httpd24

                             apachectl=/app/httpd24/bin/apachectl

                             httpd=${HTTPD-/app/httpd24/bin/httpd}

                             pidfile=${PIDFILE-/app/httpd24/logs/httpd.pid}

                             lockfile=${LOCKFILE-/var/lock/subsys/httpd24}                             

                     service httpd24  start|stop

                     chkconfig --add httpd24  

               配置文件:

                     默认没有httpd24.d/ 可以自己建立并在主配置文件中使用include  即可

                                       

2018年7月3日14:53:32


猜你喜欢

转载自blog.51cto.com/13683480/2138849