Django- query optimization

Table data:

from django.db import models
 
class Province(models.Model):
    name = models.CharField(max_length=10)
    def __str__(self):
        return self.name
 
class City(models.Model):
    name = models.CharField(max_length=5)
    province = models.ForeignKey(Province)
    def __str__(self):
        return self.name
 
class Person(models.Model):
    firstname  = models.CharField(max_length=10)
    lastname   = models.CharField(max_length=10)
    visitation = models.ManyToManyField(City, related_name = "visitor")
    hometown   = models.ForeignKey(City, related_name = "birth")
    living     = models.ForeignKey(City, related_name = "citizen")
    def __str__(self):
        return self.firstname + self.lastname

一、select_related

For one field (OneToOneField) and foreign key fields (ForeignKey), you may be used to optimize QuerySet select_related

Select_related QuerySet that after use of () function, Django will obtain the corresponding outer key corresponding to the object , so that when no longer needed after the database query.

Simple query:

citys = City.objects.all()
for c in citys:
   print (c.province)

Implementation of the above SQL statement causes multiple queries

SQL

SELECT `QSOptimize_city`.`id`, `QSOptimize_city`.`name`, `QSOptimize_city`.`province_id`
FROM `QSOptimize_city`
 
SELECT `QSOptimize_province`.`id`, `QSOptimize_province`.`name` 
FROM `QSOptimize_province`
WHERE `QSOptimize_province`.`id` = 1 ;
 
SELECT `QSOptimize_province`.`id`, `QSOptimize_province`.`name` 
FROM `QSOptimize_province`
WHERE `QSOptimize_province`.`id` = 2 ;
 
SELECT `QSOptimize_province`.`id`, `QSOptimize_province`.`name` 
FROM `QSOptimize_province`
WHERE `QSOptimize_province`.`id` = 1 ;

Use select_related:

citys = City.objects.select_related().all()
for c in citys:
  print (c.province)

SQL query only once, greatly reducing the number of SQL queries

SQL

SELECT `QSOptimize_city`.`id`, `QSOptimize_city`.`name`, 
`QSOptimize_city`.`province_id`, `QSOptimize_province`.`id`, `QSOptimize_province`.`name` 
FROM`QSOptimize_city` 
INNER JOIN `QSOptimize_province` ON (`QSOptimize_city`.`province_id` = `QSOptimize_province`.`id`) ;

Django used to fulfill the request INNER JOIN

Many foreign key query

select_related () accepts a variable length parameters, each parameter is a foreign key needs to be acquired field names, field names and foreign key foreign key, foreign key foreign key foreign key .... To select a foreign key foreign key requires the use of two underscores "__" to connect.

Note: Not specified foreign key is not added to the result

1.7 ago

zhangs=Person.objects.select_related('hometown__province','living__province')
.get(firstname="张",lastname="三")
zhangs.hometown.province
zhangs.living.province

After 1.7 supports chaining

zhangs=Person.objects.select_related('hometown__province')
.select_related('living__province').get(firstname="张",lastname="三")
zhangs.hometown.province
zhangs.living.province

Depth inquiry depth

Django will recursively traverse all specified OneToOneField and ForeignKey in depth. In this example illustrates:

zhangs = Person.objects.select_related(depth = d)
# d=1  相当于 select_related('hometown','living')
# d=2  相当于 select_related('hometown__province','living__province')

No arguments

This indicates that the requested Django as deep as possible select_related ()

Note: Django you do not know what the actual use of the field, so will all the fields are caught in, which will result in unnecessary waste and affect performance

summary:

1. And many-to-one relationship between the main needle optimization
2. The use of SQL JOIN statement optimization, be optimized by reducing the number of SQL queries and improve performance.
3. By variable length parameter specifies the need select_related field name. You can also use double underscore "__" Connecting to achieve the specified field names recursive queries.
4. No specified field will not be cached, not specified depth will not be cached, if you want to access, then Django will conduct SQL query again.
5. May be specified by the depth of recursion depth parameter, Django automatically cache all the fields within the specified depth. If you want to access outside the specified field of depth, Django will conduct SQL query again.
6. Accepts no arguments call, Django will try deep recursive query all the fields. But note that there is waste and limit the performance of Django recursive.
7. Django> = 1.7, chained calls select_related equivalent to using a variable-length parameters. Django <1.7, chained calls will lead to failure select_related front, leaving only the last one.

二、prefetch_related

For many to many fields (ManyToManyField) and many fields, you may be used prefetch_related () to optimize. Perhaps you will say, no one called OneToManyField thing ah. In fact, ForeignKey is a many-to-field, but the field is ForeignKey is associated with many of the field.

prefetch_related () and select_related () is to reduce the number of SQL queries, but not the same way to achieve. The latter is by JOIN statements, to solve the problem in a SQL query. But for many relationships, use of tables JOIN to get SQL statement will be very long, it will lead to increased memory footprint and increase the running time of SQL statements. prefetch_related () solution is, respectively, a query for each table , and then deal with the relationship between them in Python

zhangs = Person.objects.prefetch_related('visitation').get(firstname="张",lastname="三")
for city in zhangs.visitation.all() :
  print(city) 

Prefetch objects

1. A Prefetch object can only assign a prefetch operation.
2. Prefetch objects prefetch_related field specifies the manner and the same parameters, the field name is done by double underline connected.
3. QuerySet prefetch can be specified by using parameters manually queryset.
   You can specify the prefetch attribute name by to_attr parameters.
4. Prefetch objects and lookups string specified parameters can be mixed.

summary:

1. prefetch_related main pin-to-many and many-to optimize the relationship.
2. prefetch_related respectively acquire the contents of each table, and then to optimize the relationship between them through with Python.
3. You can specify the field names need select_related by variable-length parameters. Select_related targeting and features are the same.
4. In Django> = 1.7 can be achieved through complex queries Prefetch objects.
5. As a parameter prefetch_related, Prefetch objects and strings can be mixed.
6. prefetch_related chain will call the corresponding prefetch added to it, rather than replacing, it seems that there is no distinction based on the different versions.
7. You can be emptied before prefetch_related by passing None.

Reference: https://blog.csdn.net/cugbabybear/article/details/38342793