Open source address: https://github.com/leebingbin/Python3.WebAPP.Blog
1. Build a development environment
First, confirm whether the Python version installed on the system is 3.6.x:
Install Python on your Mac. If you are using a Mac, the system is OS X 10.8~10.10, then the Python version that comes with the system is 2.7. To install the latest Python 3.6, there are two ways:
Method 1: Download the Python 3.6 installer from the Python official website, double-click to run and install;
Method 2: If Homebrew is installed, install it directly through the command brew install python3.
Then, use pip to install the third-party libraries needed to develop Web App:
Asynchronous framework aiohttp:
Front-end template engine jinja2:
MySQL 5.x database, download and install from the official website. After installation, please remember the root password.
MySQL's Python asynchronous driver aiomysql:
Project structure
Select a working directory, then, we establish the following directory structure:
blog-python3-webapp/ <-- 根目录
|
+- backup/ <-- 备份目录
|
+- conf/ <-- 配置文件
|
+- dist/ <-- 打包目录
|
+- www/ <-- Web目录，存放.py文件
| |
| +- static/ <-- 存放静态文件
| |
| +- templates/ <-- 存放模板文件
|
+- ios/ <-- 存放iOS App工程
|
+- android/ <-- 存放Android App工程
|
+- LICENSE <-- 代码LICENSEAfter creating the directory structure of the project, create a git repository and synchronize it to GitHub (address: https://github.com/leebingbin/Python3.WebAPP.Blog).
Since our blog-python3-webapp is based on asyncio, we use aiohttp to write a basic app.py to test whether the asynchronous web service environment is normal.
Before testing, the third-party module asyncio needs to be installed. asyncio is the standard library introduced in Python 3.4, with built-in support for asynchronous IO directly. Python 3.4 supports asynchronous IO programming and provides asyncio library, but in 3.4, @asyncio.coroutine and yield from are used, which is indistinguishable from the original generator keyword yield. After 3.5, async (meaning coroutine) is used. ) and the await keyword, which makes the distinction much easier. Since the web framework uses aiohttp based on asyncio, which is an asynchronous model based on coroutines. In a coroutine, ordinary synchronous IO operations cannot be called, because all users are served by one thread, and the execution speed of the coroutine must be very fast to handle the requests of a large number of users. Time-consuming IO operations cannot be called synchronously in coroutines, otherwise, the system cannot respond to any other users while waiting for an IO operation.
A principle of asynchronous programming: Once the decision is made to use asynchrony, every layer of the system must be asynchronous. Fortunately, aiomysql provides asynchronous IO drivers for MySQL databases.
But before that, MySQL needs to be installed; after all, there are many places in the web app that need to access the database. Install with dmg
1. After the download is complete, double-click to open
2. Right-click, select Open, and then follow the prompts to install it foolishly; but when a MYSQL Installer prompt box pops up, be sure to open the memo, copy and paste, and write down the password of the pop-up box (this is the password of your MySQL root account)
3. Open the MySQL service
4. Configure the path
Open .bash_profile with a text editor, add PATH=$PATH:/usr/local/mysql/bin and save it. I'm used to using Atom as a text editor, or editing directly with the built-in vim. Enter the source ~/.bash_profile path on the command line to configure it.
5. Log in and modify the password
Login - mysql -u root -p login, enter the previously saved password
Change password - use SET PASSWORD FOR 'root'@'localhost' = PASSWORD('root') ;
change Password. If your version is relatively new (for example, I installed version 5.7.x), the following prompt will appear. At this time, the password has been updated, but there will be a warning. If you want to view warnings, you can use SHOW WARNINGS.
6. How to uninstall MySQL under Mac (if the installation is wrong or the password is forgotten, it is better to uninstall and reinstall)
There are installation files in the DMG format installation of mysql under Mac, but there is no uninstall file... It is very depressing.
Stop all mysql related processes first.
sudo rm /usr/local/mysql
sudo rm -rf /usr/local/mysql*
sudo rm -rf /Library/StartupItems/MySQLCOM
sudo rm -rf /Library/PreferencePanes/My*
#如果电脑上没有这个文件，跳过这步骤就好
#如果有打开文件，去掉MYSQLCOM=-YES-保存即可
vim /etc/hostconfig (and removed the line MYSQLCOM=-YES-)
rm -rf ~/Library/PreferencePanes/My*
sudo rm -rf /Library/Receipts/mysql*
sudo rm -rf /Library/Receipts/MySQL*
sudo rm -rf /var/db/receipts/com.mysql.*2. Write an ORM in Python
After setting up the environment, let's write an ORM in Python!
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
__author__ = '[email protected]'
import asyncio, logging
import aiomysql
def log(sql, args=()):
logging.info('SQL: %s ' % sql)
#创建连接池
async def create_pool(loop, **kw):
logging.info('create database connection pool...')
global __pool
__pool = await aiomysql.create_pool(
host = kw.get('host', 'localhost'),
port = kw.get('port', 3306),
user = kw['root'],
password = kw['toor'],
db = kw['db'],
charset = kw.get('charset', 'utf8mb4'), #utf8mb4是utf8的超集,完全兼容utf8,支持emoji直接插入存储
autocommit = kw.get('autocommit', True),
maxsize = kw.get('maxsizze', 10),
minsize = kw.get('minsize', 1),
loop = loop
)
#CRUD之R
async def select(sql, args, size=None):
log(sql, args)
global __pool
async with __pool.get() as conn:
async with conn.cursor(aiomysql.DictCursor) as cur:
await cur.execute(sql.replace('?', '%s'), args or ()) #SQL语句的占位符是?，而MySQL的占位符是%s，select()函数在内部自动替换。防止SQL注入攻击:注意要始终坚持使用带参数的SQL，而不是自己拼接SQL字符串
if size:
rs = await cur.fetchmany(size) #获取最多指定数量的记录
else:
rs = await cur.fetchall() #获取所有记录
logging.info('rows returned: %s' % len(rs))
return rs
#定义通用的execute()函数,用于执行CRUD方法
async def execute(sql, args, autocommit=True):
log(sql)
async with __pool.get() as conn:
if not autocommit:
await conn.begin()
try:
async with conn.cursor(aiomysql.DictCursor) as cur:
await cur.execute(sql.replace('?', '%s'), args)
affected = cur.rowcount
if not autocommit:
await conn.commit()
except BaseException as e:
if not autocommit:
await conn.rollback()
raise
return affected
def create_args_string(num):
L = []
for n in range(num):
L.append('?')
return ','.join(L)
class Field(object):
def __init__(self, name, column_type, primary_key, default):
self.name = name
self.colum_type = column_type
self.primary_key = primary_key
self.default = default
def __str__(self):
return '<%s , $s:%s>' % (self.__class__.__name__, self.colum_type, self.name)
class StringField(Field):
def __init__(self, name=None, primary_key=False, default=None, ddl='varchar(100)'):
super().__init__(name, ddl, primary_key, default)
class BooleanField(Field):
def __init__(self, name=None, default=False):
super().__init__(name, 'boolean', False, default)
class IntegerField(Field):
def __init__(self, name=None, primary_key=False, default=0):
super().__init__(name, 'bigint', primary_key, default)
class FloatField(Field):
def __init__(self, name=None, primary_key=False, default=0.0):
super().__init__(name, 'real', primary_key, default)
class TextField(Field):
def __init__(self, name=None, default=None):
super().__init__(name, 'text', False, default)
class ModeMetaclass(type):
def __new__(cls, name, bases, attrs):
if name == 'Model':
return type.__new__(cls, name, bases, attrs)
tableName = attrs.get('__table__', None) or name
logging.info('found model: %s (table: %s)' % (name, tableName))
mappings = dict()
fields = []
primaryKey = None
for k, v in attrs.items():
if isinstance(v, Field):
logging.info(' found mapping: %s ==> %s' % (k, v))
mappings[k] = v
if v.primary_key:
# 找到主键
if primaryKey:
raise StandardErroe('Duplicate primary key for field: %s' % k)
primaryKey = k
else:
fields.append(k)
if not primaryKey:
raise StandardError('Primary key not found.')
for k in mappings.keys():
attrs.pop(k)
escaped_fields = list(map(lambda f: '`%s`' % f, fields))
attrs['__mappings__'] = mappings # 保存属性和列的映射关系
attrs['__table__'] = tableName
attrs['__primary_key__'] = primaryKey # 主键属性名
attrs['__fields__'] = fields # 除主键外的属性名
attrs['__select__'] = 'select `%s`, %s from `%s`' % (primaryKey, ', '.join(escaped_fields), tableName)
attrs['__insert__'] = 'insert into `%s` (%s, `%s`) values (%s)' % (tableName, ', '.join(escaped_fields), primaryKey, create_args_string(len(escaped_fields) + 1))
attrs['__update__'] = 'update `%s` set %s where `%s` = ?' % (tableName, ', '.join(map(lambda f: '`%s` = ?' % (mappings.get(f).name or f), fields)), primaryKey)
attrs['__delete__'] = 'delete from `%s` where `%s` = ?' % (tableName, primaryKey)
return type.__new__(cls, name, bases, attrs)
class Model(dict, metaclass=ModelMetaclass):
def __init__(self, **kw):
super(Model, self).__init__(**kw)
def __getattr__(self, key):
try:
return self[key]
except KeyError:
raise AttributeError(r"'Model' object has no attribute '%s'" % key)
def __setattr__(self, key, value):
self[key] = value
def getValue(self, key):
return getattr(self, key, None)
def getValueOrDefault(self, key):
value = getattr(self, key, None)
if value is None:
field = self.__mappings__[key]
if field.default is not None:
value = field.default() if callable(field.default) else field.default
logging.debug('using default value for %s : %s' % (key, str(value)))
setattr(self, key, value)
return value
@classmethod
async def findall(cls, where=None, args=None, **kw):
' find objects by where clause.'
sql = [cls.__slect__]
if where:
sql.append('where')
sql.append(where)
if args is None:
args = []
orderBy = kw.get('orderBy', None)
if orderBy:
sql.append('order by')
sql.append(orderBy)
limit = kw.get('limit', None)
if limit is not None:
sql.append('limit')
if isinstance(limit, int):
sql.append('?')
args.append(limit)
elif isinstance(limit, tuple) and len(limit) == 2:
sql.append('?, ?')
args.extend(limit)
else:
raise ValueError('Invalid limit value: %s' % str(limit))
rs = await select(' '.join(sql), args)
return [cls(**r) for r in rs]
@classmethod
async def findNumber(cls, selectField, where=None, args=None):
' find number by select and where. '
sql = ['select %s _num_ from `%s`' % (selectField, cls.__table__)]
if where:
sql.append('where')
sql.append(where)
rs = await select(' '.join(sql), args, 1)
if len(rs) == 0:
return None
return rs[0]['_num_']
@classmethod
async def find(cls, pk):
' find object by primary key. '
rs = await select('%s where `%s`=?' % (cls.__select__, cls.__primary_key__), [pk], 1)
if len(rs) == 0:
return None
return cls(**rs[0])
async def save(self):
args = list(map(self.getValueOrDefault, self.__fields__))
args.append(self.getValueOrDefault(self.__primary_key__))
rows = await execute(self.__inset__, args)
if rows != 1:
logging.warning('failed to update by primary key: affected rows: %s' % rows)
async def remove(self):
args = [self.getValue(self.__primary_key__)]
rows = await execute(self.__delete__, args)
if rows != 1:
logging.warning('failed to remove by primary key: affected rows: %s' % rows)
With ORM, we can use Model to represent the three tables required by Python3.WebAPP.Blog :
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
'''
Models for user, blog, comment.
'''
__author__ = '[email protected]'
import time, uuid
from orm import Model, StringField, BooleanField, FloatField, TextField
def next_id():
return '%015%s000' % (int(time.time() * 1000), uuid.uuid4().hex)
class User(Model):
__table__ = 'users'
id = StringField(primary_key=True, default=next_id(), ddl='varchar(50)')
email = StringField(ddl='varchar(50)')
passwd = StringField(ddl='varchar(50)')
admin = BooleanField()
name = StringField(ddl='varchar(50)')
image = StringField(ddl='varchar(500)')
created_at = FloatField(default=time.time)
class Blog(Model):
__table__ = 'blogs'
id = StringField(primary_key=True, default=next_id, ddl='varchar(50)')
user_id = StringField(ddl='varchar(50)')
user_name = StringField(ddl='varchar(50)')
user_image = StringField(ddl='varchar(500)')
name = StringField(ddl='varchar(50)')
summary = StringField(ddl='varchar(200)')
content = TextField()
created_at = FloatField(default=time.time)
class Comment(Model):
__table__ = 'comments'
id = StringField(primary_key=True, default=next_id, ddl='varchar(50)')
blog_id = StringField(ddl='varchar(50)')
user_id = StringField(ddl='varchar(50)')
user_name = StringField(ddl='varchar(50)')
user_image = StringField(ddl='varchar(500)')
content = TextField()
created_at = FloatField(default=time.time)
3. Writing a Web Framework
Before starting web development, we need to write a web framework.
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
__author__ = '[email protected]'
import asyncio, os, inspect, logging, functools
from urllib import parse
from aiohttp import web
from apis import APIError
def get(path):
'''
Define decorator @get('/path')
'''
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kw):
return func(*args, **kw)
wrapper.__method__ = 'GET'
wrapper.__route__ = path
return wrapper
return decorator
def post(path):
'''
Define decorator @post('/path')
'''
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kw):
return func(*args, **kw)
wrapper.__method__ = 'POST'
wrapper.__route__ = path
return wrapper
return decorator
def get_required_kw_args(fn):
args = []
params = inspect.signature(fn).parameters
for name, param in params.items():
if param.kind == inspect.Parameter.KEYWORD_ONLY and param.default == inspect.Parameter.empty:
args.append(name)
return tuple(args)
def get_named_kw_args(fn):
args = []
params = inspect.signature(fn).parameters
for name, param in params.items():
if param.kind == inspect.Parameter.KEYWORD_ONLY:
args.append(name)
return tuple(args)
def has_named_kw_args(fn):
params = inspect.signature(fn).parameters
for name, param in params.items():
if param.kind == inspect.Parameter.KEYWORD_ONLY:
return True
def has_var_kw_arg(fn):
params = inspect.signature(fn).parameters
for name, param in params.items():
if param.kind == inspect.Parameter.VAR_KEYWORD:
return True
def has_request_arg(fn):
sig = inspect.signature(fn)
params = sig.parameters
found = False
for name, param in params.items():
if name == 'request':
found = True
continue
if found and (param.kind != inspect.Parameter.VAR_POSITIONAL and param.kind != inspect.Parameter.KEYWORD_ONLY and param.kind != inspect.Parameter.VAR_KEYWORD):
raise ValueError('request parameter must be the last named parameter in function: %s%s' % (fn.__name__, str(sig)))
return found
class RequestHandler(object):
def __init__(self, app, fn):
self._app = app
self._func = fn
self._has_request_arg = has_request_arg(fn)
self._has_var_kw_arg = has_var_kw_arg(fn)
self._has_named_kw_args = has_named_kw_args(fn)
self._named_kw_args = get_named_kw_args(fn)
self._required_kw_args = get_required_kw_args(fn)
async def __call__(self, request):
kw = None
if self._has_var_kw_arg or self._has_named_kw_args or self._required_kw_args:
if request.method == 'POST':
if not request.content_type:
return web.HTTPBadRequest('Missing Content-Type.')
ct = request.content_type.lower()
if ct.startswith('application/json'):
params = await request.json()
if not isinstance(params, dict):
return web.HTTPBadRequest('JSON body must be object.')
kw = params
elif ct.startswith('application/x-www-form-urlencoded') or ct.startswith('multipart/form-data'):
params = await request.post()
kw = dict(**params)
else:
return web.HTTPBadRequest('Unsupported Content-Type: %s' % request.content_type)
if request.method == 'GET':
qs = request.query_string
if qs:
kw = dict()
for k, v in parse.parse_qs(qs, True).items():
kw[k] = v[0]
if kw is None:
kw = dict(**request.match_info)
else:
if not self._has_var_kw_arg and self._named_kw_args:
# remove all unamed kw:
copy = dict()
for name in self._named_kw_args:
if name in kw:
copy[name] = kw[name]
kw = copy
# check named arg:
for k, v in request.match_info.items():
if k in kw:
logging.warning('Duplicate arg name in named arg and kw args: %s' % k)
kw[k] = v
if self._has_request_arg:
kw['request'] = request
# check required kw:
if self._required_kw_args:
for name in self._required_kw_args:
if not name in kw:
return web.HTTPBadRequest('Missing argument: %s' % name)
logging.info('call with args: %s' % str(kw))
try:
r = await self._func(**kw)
return r
except APIError as e:
return dict(error=e.error, data=e.data, message=e.message)
def add_static(app):
path = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'static')
app.router.add_static('/static/', path)
logging.info('add static %s => %s' % ('/static/', path))
def add_route(app, fn):
method = getattr(fn, '__method__', None)
path = getattr(fn, '__route__', None)
if path is None or method is None:
raise ValueError('@get or @post not defined in %s.' % str(fn))
if not asyncio.iscoroutinefunction(fn) and not inspect.isgeneratorfunction(fn):
fn = asyncio.coroutine(fn)
logging.info('add route %s %s => %s(%s)' % (method, path, fn.__name__, ', '.join(inspect.signature(fn).parameters.keys())))
app.router.add_route(method, path, RequestHandler(app, fn))
def add_routes(app, module_name):
n = module_name.rfind('.')
if n == (-1):
mod = __import__(module_name, globals(), locals())
else:
name = module_name[n+1:]
mod = getattr(__import__(module_name[:n], globals(), locals(), [name]), name)
for attr in dir(mod):
if attr.startswith('_'):
continue
fn = getattr(mod, attr)
if callable(fn):
method = getattr(fn, '__method__', None)
path = getattr(fn, '__route__', None)
if method and path:
add_route(app, fn)
Write the configuration file config_default.py:
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
__author__ = '[email protected]'
'''
Default configurations.
'''
# 'secret': secret 是必需的选项，这是用于签名会话ID cookie的密钥。这可以是单个密钥的字符串或多个秘密的数组(只有第一个元素将用于签名会话ID cookie)而在验证请求中的签名时，将考虑所有元素
# 另外, 考虑到安全性, 这个密钥是不建议存储在的程序中的. 最好的方法是存储在你的系统环境变量中, 通过 os.getenv(key, default=None) 获得
configs = {
'debug': True,
'db': {
'host': '127.0.0.1',
'port': 3306,
'user': 'root',
'password': 'toor',
'db': 'db'
},
'session': {
'secret': '[email protected]'
}
}
Write the configuration file config_override.py:
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
__author__ = '[email protected]'
'''
Override configurations.
'''
configs = {
'db': {
'host': '127.0.0.1'
}
}
With config_default.py as the standard configuration of the development environment and config_override.py as the standard configuration of the production environment, we can easily develop locally and deploy the application to the server at any time. The application reads the configuration file needs to read from config_override.py first. In order to simplify reading configuration files, all configurations can be read into a unified config.py:
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
__author__ = '[email protected]'
'''
Configuration
'''
import config_default
class Dict(dict):
'''
Simple dict but support access as x.y style.
'''
def __init__(self, names=(), values=(), **kw):
super(Dict, self).__init__(**kw)
for k, v in zip(names, values):
self[k] = v
def __getattr__(self, key):
try:
return self[key]
except KeyError:
raise AttributeError(r"'Dict' object has no attribute '%s'" % key)
def __setattr__(self, key, value):
self[key] = value
def merge(defaults, override):
r = {}
for k, v in defaults.items():
if k in override:
if isinstance(v, dict):
r[k] = merge(v, override[k])
else:
r[k] = override[k]
else:
r[k] = v
return r
def toDict(d):
D = Dict()
for k, v in d.items():
D[k] = toDict(v) if isinstance(v, dict) else v
return D
configs = config_default.configs
try:
import config_override
configs = merge(configs, config_override.configs)
except ImportError:
pass
configs = toDict(configs)
This article is an original article by the blogger, please indicate the source for reprinting!
https://my.oschina.net/u/3375733/blog/