1. What is Tornado
Tornado is a Python web framework and asynchronous networking library originally developed by FriendFeed. By using non-blocking network I/O, Tornado can scale to thousands of open connections, making it ideal for long polling, WebSockets, and other applications that require a long-lived connection with each user. The purpose of Tornado is to solve the stability and performance problems in high concurrency scenarios. Tornado provides an HttpServer. In Tornado 6.x version, the latest asynchronous IO library is used, which greatly improves the performance of the server. It uses non-blocking network I/O and solves the C10k problem (meaning, it can handle more than 10,000 concurrent connections on a single machine).
Tornado, Django, and Flask are called the three major frameworks for Python Web development. The obvious difference between Tornado and the other two frameworks is its non-blocking network IO. Tornado and Flask have some things in common. The first choice is that they are both "micro" and lightweight, in line with the current microservice architecture design. There is no template engine provided by default in Tornado, which means that you use Tornado more often API interface, also known as RESTful API. Of course you can use third-party templating engines to extend your application. But I still suggest that you use Tornado to develop pure back-end applications better.
What types of applications can we develop with Tornado in our daily work? The following are the scenarios I often use:
- As an asynchronous Http client to send requests or write asynchronous crawlers.
- As a WebSocket network framework to develop a Websocket server program, it is often called a chat room application.
- As a high-concurrency, high-performance HttpServer to provide services, combined with uwsgi to build a better server.
- As a web framework, to develop high-concurrency, high-performance web applications or interfaces.
- Used as a coroutine library to build coroutine concurrent applications.
2. Small demo to quickly get started with Tornado
This section will use a few simple examples to get you started with Tornado development. The author's development environment is Linux system, Python 3.6+, MySQL 5.7, and Redis 6.0. Note that the Python 3.6+ development environment is required in this section, and Mac or Windows can be used as the development system. MySQL and Redis are not needed in this section for the time being, and will be used in later development, so they can be installed later, and the version numbers should be as consistent as possible. If there is an exception caused by different versions, please Baidu or Google to solve it yourself.
Install Tornado
pip install tornado==6.0.4
Write a small crawler using Tornado asynchronous HTTP client.
The main code is as follows:
# async_http_cilent.py
import time
from datetime import timedelta
from html.parser import HTMLParser
from urllib.parse import urljoin, urldefrag
from tornado import gen, httpclient, ioloop, queues
base_url = "http://www.tornadoweb.org/en/stable/"
concurrency = 10
async def get_links_from_url(url):
"""Download the page at `url` and parse it for links.
Returned links have had the fragment after `#` removed, and have been made
absolute so, e.g. the URL 'gen.html#tornado.gen.coroutine' becomes
'http://www.tornadoweb.org/en/stable/gen.html'.
"""
response = await httpclient.AsyncHTTPClient().fetch(url)
print("fetched %s" % url)
html = response.body.decode(errors="ignore")
return [urljoin(url, remove_fragment(new_url)) for new_url in get_links(html)]
def remove_fragment(url):
pure_url, frag = urldefrag(url)
return pure_url
def get_links(html):
class URLSeeker(HTMLParser):
def __init__(self):
HTMLParser.__init__(self)
self.urls = []
def handle_starttag(self, tag, attrs):
href = dict(attrs).get("href")
if href and tag == "a":
self.urls.append(href)
url_seeker = URLSeeker()
url_seeker.feed(html)
return url_seeker.urls
async def main():
q = queues.Queue()
start = time.time()
fetching, fetched = set(), set()
async def fetch_url(current_url):
if current_url in fetching:
return
print("fetching %s" % current_url)
fetching.add(current_url)
urls = await get_links_from_url(current_url)
fetched.add(current_url)
for new_url in urls:
# Only follow links beneath the base URL
if new_url.startswith(base_url):
await q.put(new_url)
async def worker():
async for url in q:
if url is None:
return
try:
await fetch_url(url)
except Exception as e:
print("Exception: %s %s" % (e, url))
finally:
q.task_done()
await q.put(base_url)
# Start workers, then wait for the work queue to be empty.
workers = gen.multi([worker() for _ in range(concurrency)])
await q.join(timeout=timedelta(seconds=300))
assert fetching == fetched
print("Done in %d seconds, fetched %s URLs." % (time.time() - start, len(fetched)))
# Signal all the workers to exit.
for _ in range(concurrency):
await q.put(None)
await workers
if __name__ == "__main__":
io_loop = ioloop.IOLoop.current()
io_loop.run_sync(main)
Run the service (note: if using a virtual environment, run the command in the virtual environment):
python async_http_cilent.py
3. Write a small web application using the Tornado web framework
This section will use Tornado's web framework to write a small server application, which is also the prototype of the RESTful API interface. This section will implement an interface, the path is /test/ When requesting this path, the application will return a string of JSON strings, mainly The massage is Hello World.
# tornado_server.py
import tornado.ioloop
import tornado.web
import asyncio
import time
class BaseHandler(tornado.web.RequestHandler):
'''
请求的基类，继承自 Tornado 的 web 框架的 RequestHandler
'''
def __init__(self, application, request, **kwargs):
super().__init__(application, request, **kwargs)
print('[%s]' % (time.strftime("%Y-%m-%d %H:%M:%S")),request.version,request.remote_ip,request.method,request.uri)
class TestHandler(BaseHandler):
async def get(self, *args, **kwargs):
res_format = {
"message": "ok", "errorCode": 0, "data": {
}}
try:
await asyncio.sleep(2) # 会将程序暂停两秒
res = mul.delay(3,5)
res_format['message'] = 'Hello World'
return self.finish(res_format)
except Exception as e:
print('出现异常：%s' % str(e))
return self.finish({
"message": "出现无法预料的异常：{}".format(str(e)), "errorCode": 1, "data": {
}})
def make_app():
return tornado.web.Application([
(r"/test/", TestHandler),
], debug=True)
if __name__ == "__main__":
print("""Wellcome...
Starting development server at http://%s:%s/
Quit the server with CTRL+C.""" % ('127.0.0.1', '8080'))
app = make_app()
app.listen(8080) # 设置监听的端口号，默认监听地址是 localhost
tornado.ioloop.IOLoop.current().start()
Run the service:
python tornado_server.py
Test with the curl command:
$ curl -H "Content-Type: application/json" http://127.0.0.1:8080/test/
$ {
"message": "Hello World", "errorCode": 0, "data": {
}}
Test with Postman:
