ORM Django base in six of locks and transactions

A lock

Row-level locking

　　select_for_update (nowait = False, skip_locked = False) # attention must be used inside a transaction, as to how to open a transaction, we see the following transactions a.

　　Query returns a set of rows locked until the end of the transaction, if the database support, it will generate a SELECT ... FOR UPDATE statement.

　　for example:

entries = Entry.objects.select_for_update().filter(author=request.user)  #加互斥锁，由于mysql在查询时自动加的是共享锁，所以我们可以手动加上互斥锁。create、update、delete操作时，mysql自动加行级互斥锁

　　All matching rows are locked until the end of the transaction. This means that data can be locked to prevent other transactions from modifying.

　　Under normal circumstances, if other matters related to locked rows, this query will be blocked until the lock is released. If you do not want to make this inquiry blocked, use select_for_update (nowait = True). If the other transaction holds a conflicting lock, mutex, then the query will throw an exception DatabaseError. You can also use select_for_update (skip_locked = True) ignore locked rows. nowait and skip_locked are mutually exclusive, while setting causes ValueError

　　Currently, postgresql, oracle and mysql database back-end support select_for_update (). However, MySQL does not support the nowait and skip_locked parameters.

　　Use these options are not supported database backend (such as MySQL) will nowait = True or skip_locked = True converted to select_for_update () will result in an DatabaseError exception, which can prevent the code from terminated unexpectedly.

Table lock

class LockingManager(models.Manager):
    """ Add lock/unlock functionality to manager.

    Example::

        class Job(models.Model): #其实不用这么负载，直接在orm创建表的时候，给这个表定义一个lock和unlock方法，借助django提供的connection模块来发送锁表的原生sql语句和解锁的原生sql语句就可以了，不用外层的这个LckingManager(model.Manager)类

            manager = LockingManager()

            counter = models.IntegerField(null=True, default=0)

            @staticmethod
            def do_atomic_update(job_id)
                ''' Updates job integer, keeping it below 5 '''
                try:
                    # Ensure only one HTTP request can do this update at once.
                    Job.objects.lock()

                    job = Job.object.get(id=job_id)
                    # If we don't lock the tables two simultanous
                    # requests might both increase the counter
                    # going over 5
                    if job.counter < 5:
                        job.counter += 1                                        
                        job.save()

                finally:
                    Job.objects.unlock()


    """    

    def lock(self):
        """ Lock table. 

        Locks the object model table so that atomic update is possible.
        Simulatenous database access request pend until the lock is unlock()'ed.

        Note: If you need to lock multiple tables, you need to do lock them
        all in one SQL clause and this function is not enough. To avoid
        dead lock, all tables must be locked in the same order.

        See http://dev.mysql.com/doc/refman/5.0/en/lock-tables.html
        """
        cursor = connection.cursor()
        table = self.model._meta.db_table
        logger.debug("Locking table %s" % table)
        cursor.execute("LOCK TABLES %s WRITE" % table)
        row = cursor.fetchone()
        return row

    def unlock(self):
        """ Unlock the table. """
        cursor = connection.cursor()
        table = self.model._meta.db_table
        cursor.execute("UNLOCK TABLES")
        row = cursor.fetchone()
        return row  

Two affairs

　About MySQL transaction, mysql blog I have made it very clear, and then we take a look at Django is if you do a transaction. There are a variety of ways to add transaction before django1.8 version, in the form of middleware, in the form of function decorators, context manager in the form of (global) and so on, but after a lot of methods to update the version 1.8, the following we only say the latest:

Global open 1

　　In the Web application, is commonly used in transaction processing mode are wrapped each request in a transaction. This feature is very simple to use, you only need to configure its entry ATOMIC_REQUESTS set to True.

　　It works like this: When the request came, Django will open a transaction before calling the view method. If the request was properly prepared and correctly returns the result, Django will commit the transaction. Otherwise, Django will roll back the transaction.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'mxshop',
        'HOST': '127.0.0.1',
        'PORT': '3306',
        'USER': 'root',
        'PASSWORD': '123',
        'OPTIONS': {
            "init_command": "SET default_storage_engine='INNODB'",
　　　　　　　#'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", #配置开启严格sql模式


        }
        "ATOMIC_REQUESTS": True, #全局开启事务，绑定的是http请求响应整个过程
        "AUTOCOMMIT":False, #全局取消自动提交，慎用
    }，
　　'other':{
　　　　'ENGINE': 'django.db.backends.mysql', 
            ......
　　} #还可以配置其他数据库
}

　　This approach is above all a unified sql http requests are placed in a corresponding transaction execution (either all succeed, or all else fails). Is global configuration, if you want to turn on the water for a http request (and then customize the transaction), you can use non_atomic_requests decorator, then he can not control the affairs

from django.db import transaction

@transaction.non_atomic_requests
def my_view(request):
    do_stuff()

@transaction.non_atomic_requests(using='other')
def my_other_view(request):
    do_stuff_on_the_other_database()

　　But Django documentation says, is not recommended. Because if the transaction with an HTTP request to bind together, but view is dependent on the efficiency of application and database queries to the database lock contention current situation. When the flow rate up, performance would be affected, knowing that you will do

　　So the following is recommended this way, through transaction.atomicto more explicit control transactions. allows us atomic block, when executed, provide guarantees atoms in the database level. If the code block is successfully completed, a corresponding change will be submitted to the database commit; if an exception is encountered during execution, it will roll back all changes according to the code segment.

2 topical affairs

　　　Atomic ( a using = None, that the savepoint = True ) [Source], parameters: using = 'other', that is, when you operate other databases, to take effect this transaction, we look at the above database configuration, in addition to default, as well as a other the default is the default. savepoint mean open transaction save point, I recommend a look at the database blog affairs segment of the inside of the explanation save point.

Atomicity is a property database transaction. Use atomic, we can create a block of code includes atomic. Upon completion of block normal operation, all the modifications are committed to the database. Conversely, if there is an exception, the changes will be rolled back.

　　　　Manage the atomic block may also code embedded to the method. In this case, even if the internal code block normal operation, if external code block throws an exception, then it is no way to submit its changes to the database.

　　　　Usage 1: function to do the decorators to use　

from django.db import transaction

@transaction.atomic
def viewfunc(request):
    # This code executes inside a transaction.
    do_stuff()

　　Usage 2: used as a context manager, in fact, saved set points affairs

from django.db import transaction

def viewfunc(request):
    # This code executes in autocommit mode (Django's default).
    do_stuff()

    with transaction.atomic():   #保存点
        # This code executes inside a transaction.
        do_more_stuff()

    do_other_stuff()

　　Once the atomic block into try / except, the natural integrity errors will be disposed of, such as the following example:

from django.db import IntegrityError, transaction

@transaction.atomic
def viewfunc(request):
    create_parent()

    try:
        with transaction.atomic():
            generate_relationships()
    except IntegrityError:
        handle_exception()

    add_children()

　　Usage 3: may also be nested, nested transaction context manager function of the transaction, the transaction context manager nested transaction context manager and the like. The following are examples of nested functions context:

from django.db import IntegrityError, transaction

@transaction.atomic
def viewfunc(request):
    create_parent()

    try:
        with transaction.atomic():
            generate_relationships()
　　　　　　　#other_task()  #还要注意一点，如果你在事务里面写了别的操作，只有这些操作全部完成之后，事务才会commit，也就是说，如果你这个任务是查询上面更改的数据表里面的数据，那么看到的还是事务提交之前的数据。
    except IntegrityError:
        handle_exception()

    add_children()

　　In this case, even if generate_relationships () code to break the data integrity constraints, you can still perform database operations in add_children (), and change the create_parent () produced is also effective. Note that before calling handle_exception (), modify () in generate_relationships had been rolled back safe. Therefore, if necessary, you can still operate the database in an exception handler.

尽量不要在atomic代码块中捕获异常

  因为当atomic块中的代码执行完的时候，Django会根据代码正常运行来执行相应的提交或者回滚操作。如果在atomic代码块里面捕捉并处理了异常，就有可能隐盖代码本身的错误，从而可能会有一些意料之外的不愉快事情发生。

  担心主要集中在DatabaseError和它的子类（如IntegrityError）。如果这种异常真的发生了，事务就会被破坏掉，而Django会在代码运行完后执行回滚操作。如果你试图在回滚前执行一些数据库操作，Django会抛出TransactionManagementError。通常你会在一个ORM相关的信号处理器抛出异常时遇到这个行为。

捕获异常的正确方式正如上面atomic代码块所示。如果有必要，添加额外的atomic代码块来做这件事情，也就是事务嵌套。这么做的好处是：当异常发生时，它能明确地告诉你那些操作需要回滚，而那些是不需要的。

　　　　In order to ensure atomicity, atomic also banned some API. Like trying to commit, roll back the transaction, and changes automatically submitted to the database connection status of these operations, the atomic block of code are not promises, otherwise it will throw an exception.

　　Here is Django's transaction management code:

• Opening a transaction enters the outermost atomic block;
• Create savepoint atomic block access to the interior;
• When you exit the release of internal atomic or roll back the transaction; pay attention if there are nested, the inner layer of the transaction is not submitted, it may release (normal end) or rollback
• When the transaction commits or rolls back the outermost atomic block exit;

　　　　You can save the point parameter is set to False to disable the internal code block to create a save point. If an exception occurs, Django executed after exiting the block when the first parent roll back, if there is a save point, save this position will roll back to the point, otherwise it is rolled back to the outermost block of code. The outer layer of the transaction is still able to guarantee atomicity. However, this option should only be used to save the overhead of a larger point of time. After all, it has a drawback: destroys error handling mechanism described above.

　　Note: Transaction only for the operation of the database layer for transaction management, it can not be understood as a python operation of transaction management

def example_view(request):
    tag = False
    with transaction.atomic():
        tag = True
        change_obj() # 修改对象变量
        obj.save()
        raise DataError
    print("tag = ",tag) #结果是True，也就是说在事务中的python变量赋值，即便是事务回滚了，这个赋值也是成功的

　　Also note : If you configure global affairs, local affairs, and it may be a conflict, you may find that your local after the transaction is complete, if your function inside sql in addition to other problems, that is, not in this context management other sql function inside the local affairs within the scope of the package's a problem, your local transactions is not submitted on because the global rolls back all the sql requests and responses involved, so it is recommended after try not to project to configure global affairs, to get through local affairs, of course, look at your business scenario.

Other methods of transaction

@transaction.atomic
def viewfunc(request):

  a.save()
  # open transaction now contains a.save()
  sid = transaction.savepoint()  #创建保存点

  b.save()
  # open transaction now contains a.save() and b.save()

  if want_to_keep_b:
      transaction.savepoint_commit(sid) #提交保存点
      # open transaction still contains a.save() and b.save()
  else:
      transaction.savepoint_rollback(sid)  #回滚保存点
      # open transaction now contains only a.save()

  transaction.commit() #手动提交事务，默认是自动提交的，也就是说如果你没有设置取消自动提交，那么这句话不用写，如果你配置了那个AUTOCOMMIT=False，那么就需要自己手动进行提交。

　　In order to ensure transaction isolation, we can also combine the above lock is achieved, that is to say in the affairs inside the query, we use select_for_update display locking means to ensure the isolation, the lock will be released after the end of the transaction, for example, :(To understanding)

@transaction.atomic ## 轻松开启事务
def handle(self):
    ## 测试是否存在此用户
    try:
        ## 锁定被查询行直到事务结束
        user = 
    User.objects.select_for_update().get(open_id=self.user.open_id)
        #other sql 语句
    except User.DoesNotExist:
        raise BaseError(-1, 'User does not exist.')
    

　　By Django external python script to test the transaction:

import os

if __name__ == '__main__':
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "BMS.settings")
    import django
    django.setup()

    import datetime
    from app01 import models

    try:
        from django.db import transaction
        with transaction.atomic():
            new_publisher = models.Publisher.objects.create(name="火星出版社")
            models.Book.objects.create(title="橘子物语", publish_date=datetime.date.today(), publisher_id=10)  # 指定一个不存在的出版社id
    except Exception as e:
        print(str(e))

　　Some say the following principles set it matters little:

　　　　1. Transaction short holding
　　　　2. avoid transaction ROLLBACK
　　　　3. avoided that the savepoint
　　　　4. By default, depending on the pessimistic lock
　　　　5. demanding transaction throughput considerations optimistic locking
　　　　The display transaction statement open
　　　　row 7. Lock better, lock time as short as possible