Django connects multiple databases

original intention

In order to separate the data of different businesses and fall into different libraries, usedjango to connect multiple databases.

set up

# settings.py
DATABASES = {
    
    
    "default": {
    
    },
    "users": {
    
    
        "NAME": "user_data",
        "ENGINE": "django.db.backends.mysql",
        "USER": "mysql_user",
        "PASSWORD": "superS3cret",
    },
    "customers": {
    
    
        "NAME": "customer_data",
        "ENGINE": "django.db.backends.mysql",
        "USER": "mysql_cust",
        "PASSWORD": "veryPriv@ate",
    },
}

• DjangoUsedefaultdatabase if no other option
• defaultNo need, but an empty dictionary must be set

Synchronize

Setting up multiple databases requires synchronization multiple times. It's more troublesome here.

python3 manage.py
python manage.py migrate message。# 迁移到default库
python manage.py migrate message  --database=message_db

Migration will migrate all table images of all models to the two libraries. Do not expect to migrate all model sub-table storage at once with the built-in instructions.

Database Routing

class MessageRouter:
    """
    A router to control all database operations on models in the
    auth and contenttypes applications.
    """

    route_app_labels = {
    
     "message", "data"}
    

    def db_for_read(self, model, **hints):
        """
        Attempts to read auth and contenttypes models go to message_db.
        """


        if model._meta.app_label in self.route_app_labels:
            return "message_db"
            
            
        return None

    def db_for_write(self, model, **hints):
        """
        Attempts to write auth and contenttypes models go to message_db.
        """
        if model._meta.app_label in self.route_app_labels:

            return "message_db"
        return None

    def allow_relation(self, obj1, obj2, **hints):
        """
        Allow relations if a model in the auth or contenttypes apps is
        involved.
        """

        if (
            obj1._meta.app_label in self.route_app_labels
            or obj2._meta.app_label in self.route_app_labels
        ):
            return True
        return None

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        """
        Make sure the auth and contenttypes apps only appear in the
        'message_db' database.
        """
        if app_label in self.route_app_labels:

           
            return db == "message_db"
       
        return None

You can set the read, write, and migration permissions of the database through Django's path middleware.
needs to be added in settings.py

DATABASE_ROUTERS = ["path.to.MessageRouter"]
# my instance is ["message.MessageRouter.MessageRouter"]

fake migration

If you encounter a situation where you do not want to migrate to library A, but you are prompted all the time, you can use fake migration.

py manage.py migrate  <app> --fake

Summarize

Data sharding (also known as database sharding or database partitioning) is a strategy for breaking up a large database into smaller, more manageable components called "shards" or "partitions." This approach has several potential benefits:

1. Improve performance and response time
   Read and write separation: By allocating read operations and write operations to different databases or shards, the throughput and response time of the system can be improved.
   Parallel processing: Sharding allows multiple queries and transactions to be executed in parallel on different databases or shards, thereby improving performance.
2. Scalability
   Horizontal scalability: When the amount of data and access increases, the system can be easily expanded by adding more databases or shards instead of replacing existing hardware (This is called vertical scaling).
3. Simplified backup and recovery
   Faster backups: Small database shards make backups easier and faster.
   Partial recovery: If a problem occurs with one shard, you can restore only that specific shard without affecting the entire system.
4. High availability and failover
   Redundancy: Data can be stored redundantly in multiple shards or databases, thereby increasing data availability.
   Fault isolation: If a problem occurs in one database or shard, it will not affect other shards, thus reducing the scope of the failure.
5. Data locality
   Optimizing queries: Data sharding can be optimized based on the access pattern of the application, for example, by geographical location or time, thereby improving query performance.
6. Management and Maintenance
   Distributed Management: Small, decentralized databases are generally easier to manage and maintain.
   Version control and updates: Database shards can be updated or modified one by one, rather than updating the entire database at once.
7. Cost Effectiveness
   Hardware Cost: By using shards, you can store and process data in a distributed manner on cheap commodity hardware, rather than relying on a single, expensive, high-performance server. .
   Although data sharding has many benefits, it also brings some challenges, such as data consistency, complex queries and transaction processing. Therefore, the needs and characteristics of your application need to be carefully evaluated before deciding to adopt this strategy.

But it will be more troublesome, as two databases must be migrated every time.