python- network security programming fifth day (threading multi-threaded module & Queue module)

Foreword

Yesterday 21:00 sleep more than 2 points up and did not sleep. . . Then learn it emmmm, picked up the python courses idle days. Learn now at 5.58 summarizes continue to start learning new content

A lot of thread?

Thread (English: thread) is an operating system capable of operation scheduling smallest units. It is included in the process of being, it is the process of the actual operation of the unit. A thread refers to a process in a single sequential flow of control, a process multiple threads concurrently, each thread in parallel to perform different tasks. In the Unix System V and SunOS it is also referred to as lightweight processes (lightweight processes), but more lightweight process refers to kernel threads (kernel thread), while the user thread (user thread) called threads.

Process (Process) is a computer program run on a set of data on the activities of the system is the basic unit of resource allocation and scheduling, is the underlying operating system architecture. In the computer architecture for the early design process, the basic process is the execution of the program entity; in contemporary computer architecture design for the thread, the thread is the process vessel. Program is instruction, organization of data and its description of the process is a solid program.

Multithreading similar to the simultaneous execution of multiple different programs, multi-threaded run has the following advantages:

The program uses threads can occupy long tasks into the background to deal with.
The user interface can be more attractive, so for example the user clicks a button to trigger the processing of certain events, you can pop up a progress bar to show the progress of the process
Running speed may accelerate
On some tasks, such as waiting for user input to achieve, document literacy and send and receive network data, the thread is more useful. In this case we can free up some valuable resources such as memory usage and so on.
Thread can be preempted (interrupted)
In other threads are running, the thread can be held in abeyance (also known as sleep) - This is the thread concessions.

python multithreading - the introduction of threading module

threading module for providing thread-related operation, the thread is the smallest unit of work application. The current version of python multithreading library does not implement priority, thread group, the thread can not be stop, pause, resume interrupted.

threading module provides classes:
　　the Thread, Lock, R lock, for condition Condition, [Bounded] Semaphore, the Event, the Timer, local

import threading

Threading module

Python provides support for threading through two standard libraries thread and threading. thread provides a low level, the original thread and a simple lock.

Other methods threading module provides:

threading.currentThread (): Returns the current thread variable.
threading.enumerate (): Returns a list of running threads. Refers to the thread starts running, before the end, does not include a thread before starting and after termination.
threading.activeCount (): Returns the number of running threads, and len (threading.enumerate ()) have the same result.

In addition to using the method, the threading module also provides a Thread class processing thread, Thread class provides the following methods:

run (): to indicate the method of active threads.
start (): start thread activity.
join ([time]): Wait until the thread is suspended. This blocks the calling thread until the thread's join () method is called suspension - normal exit or throw an unhandled exception - or the optional timeout occurs.
isAlive (): Returns the thread is active.
getName (): Returns the thread name.
setName (): Set the thread name.

python multithreading - () creates a thread through threading.Thread

() Creates a thread through threading.Thread. Where the target is received to execute the function name, args receive parameters passed into the function
eg:

import threading
import time

def xiaohua(n):
    print("xiaohua:%s"%(n))
    the time.sleep ( 2) # increased latency can see the effect

# Create two threads 
T1 = threading.Thread (target = xiaohua, args = (2 ,))
t2=threading.Thread(target=xiaohua,args=(3,))

# Start two threads 
t1.start ()
t2.start()

After the program has executed the main thread from top to bottom, at t1, t2 after the two thread starts, in parallel with the main thread, seize CPU resources. We increased internal xiaohua function under two seconds of delay in order to perform our normal emptying function xiaohua first print second print soon have to wait two seconds at this time we are using a multi-threaded so these two lines simultaneously print the results.

python-Queue Module

Python的Queue模块中提供了同步的、线程安全的队列类，包括FIFO（先入先出)队列Queue，LIFO（后入先出）队列LifoQueue，和优先级队列PriorityQueue。这些队列都实现了锁原语，能够在多线程中直接使用。可以使用队列来实现线程间的同步。

Queue.qsize() 返回队列的大小
Queue.empty() 如果队列为空，返回True,反之False
Queue.full() 如果队列满了，返回True,反之False
Queue.full 与 maxsize 大小对应
Queue.get([block[, timeout]])获取队列，timeout等待时间
Queue.get_nowait() 相当Queue.get(False)
Queue.put(item) 写入队列，timeout等待时间
Queue.put_nowait(item) 相当Queue.put(item, False)
Queue.task_done() 在完成一项工作之后，Queue.task_done()函数向任务已经完成的队列发送一个信号
Queue.join() 实际上意味着等到队列为空，再执行别的操作

Queue模块-FIFO队列

class Queue.Queue(maxsize=0)

FIFO即First in First Out,先进先出。Queue提供了一个基本的FIFO容器，使用方法很简单,maxsize是个整数，指明了队列中能存放的数据个数的上限。一旦达到上限，插入会导致阻塞，直到队列中的数据被消费掉。如果maxsize小于或者等于0，队列大小没有限制。

import queue
Queue=queue.Queue()

for i in range(6):
    Queue.put(i)
    
while not Queue.empty():
    print(Queue.get())

执行结果

Queue模块-LIFO队列

class Queue.LifoQueue(maxsize=0)

LIFO即Last in First Out,后进先出。与栈的类似，使用也很简单,maxsize用法同上

import queue
Queue=queue.LifoQueue()

for i in range(6):
    Queue.put(i)
    
while not Queue.empty():
    print(Queue.get())

执行结果:

综合运用

# coding=utf-8
import threading, queue, time, urllib
from  urllib import request
baseUrl = 'http://www.pythontab.com/html/pythonjichu/'
urlQueue = queue.Queue()

for i in range(2,10):
    url=baseUrl+str(i)+'.html'
    urlQueue.put(url)
    print(url)

def fetchUrl(urlQueue):
    while True:
        try:
            url=urlQueue.get_nowait()#相当queue.get(False)
            i=urlQueue.qsize()#返回队列的大小
        except Exception as e:
                break
        print('当前线程名%s,url:%s'%(threading.currentThread().name,url));
        try:
            response=urllib.request.urlopen(url)
            responseCode=response.getcode()
        except Exception as e:
            continue
        if responseCode==200:
             #抓取内容的数据处理可以放到这里
#            #为了突出效果， 设置延时
            time.sleep(1)


if __name__=='__main__':
    startTime=time.time()
    threads=[]
    threadNum=4
    
    for i in range(0,threadNum):
        t=threading.Thread(target=fetchUrl,args=(urlQueue,))
        threads.append(t) #将线程加入到容器
        print(threads)
    
    for t in threads:
        t.start()
    for t in threads:
        t.join()
       
    endTime=time.time()
    print('time=%s'%(endTime-startTime))

参考学习:https://www.cnblogs.com/tkqasn/p/5700281.html

https://www.runoob.com/python/python-multithreading.html

https://www.cnblogs.com/itogo/p/5635629.html

python- network security programming fifth day (threading multi-threaded module & Queue module)

Threading module

Guess you like