Learn python (6) database programming from scratch with me: MongoDB database

foreword

Looking back, I talked about python syntax programming, compulsory introductory basics and network programming, multi-thread/multi-process/coroutine, etc. Yesterday and today I talked about database programming MySQL, and Redis is the third MongoDB article today. I haven’t read it before. There is no need to turn forward, the series of articles have been sorted out:

1. Learn python from scratch with me (1) Compulsory programming grammar
2. Learn python from scratch with me (2) Network programming
3. Learn python from scratch with me (3) Multi-thread/multi-process/ Coroutine
4. Learn python from scratch with me (4) Database programming: MySQL database
5. Learn python from scratch with me (5) Database programming: Redis database

This article talks about: python database programming: MongoDB database

This series of articles is based on the following learning routes. Due to the large content:

Learn python from scratch to advanced advanced roadmap

Pay attention to the official account: python technology training camp , learn advanced step by step

Python resources suitable for zero-based learning and advanced people:

① Tencent certified python complete project practical tutorial notes PDF
② More than a dozen major manufacturers python interview topic PDF
③ Python full set of video tutorials (zero foundation-advanced advanced JS reverse)
④ Hundreds of project actual combat + source code + notes
⑤ Programming grammar - machine learning -Full-stack development-data analysis-crawler-APP reverse engineering and other full set of projects + documents
⑥ Exchange and study
⑦ Want to take part-time orders

Next chapter: Machine Learning Module

1. A brief introduction to MongoDB

Python MongoDB is a non-relational database, also known as NoSQL database. Data storage solutions designed to provide high performance, high availability, and scalability. Python MongoDB stores data using the JSON format, which makes it ideal for handling large amounts of unstructured data.

Key features of Python MongoDB include :

High performance : Python MongoDB uses memory mapping technology, which can read and write data quickly.
High availability : Python MongoDB supports master-slave replication and sharding technology, which can ensure data reliability and availability.
Scalability : Python MongoDB can easily scale to multiple nodes to meet growing data demands.
Flexibility : Python MongoDB supports dynamic schema, which can change the data structure at any time as needed.
Ease of use : Python MongoDB provides an easy-to-use API for easy data manipulation.

The installation of Python MongoDB is very simple, just use the pip command to install it. After the installation is complete, you can use the pymongo library to connect and operate the MongoDB database.

Python MongoDB has a wide range of application scenarios, especially in big data processing and real-time data analysis. It can be used to store various types of unstructured data such as log data, social media data, IoT data, etc. At the same time, Python MongoDB can also be integrated with other data storage solutions (such as relational databases) to meet different data processing needs.

In summary, Python MongoDB is a high-performance, highly available, and scalable data storage solution that can help developers easily handle large amounts of unstructured data.

2. Basic commands about database

MongoDB is a non-relational database that uses documents to store data instead of tables. Python is a popular programming language that can interact with MongoDB databases. In Python, we can use the PyMongo library to connect and operate the MongoDB database.

In MongoDB, we can use the following basic commands to manage the database:

create database

In MongoDB, we can use the use command to create a new database. For example, to create a mydatabasedatabase named we can use the following command:

use mydatabase

If the database does not exist, MongoDB will create a new one. If the database already exists, MongoDB will return a reference to that database.

show database

To display all databases in MongoDB, we can use the following command:

show dbs

This will list all databases in MongoDB.

delete database

To delete a database in MongoDB, we can use the following command:

db.dropDatabase()

This will delete the currently selected database.

switch database

To switch to another database in MongoDB, we can use the following command:

use anotherdatabase

This will switch to anotherdatabasethe database named .

show current database

To display the currently selected database, we can use the following command:

db

This will return a reference to the currently selected database.

In short, the interaction between Python and the MongoDB database is very convenient, and PyMongothe library can be used to connect and operate the MongoDB database. In Python, we can use basic commands to manage databases, such as creating, displaying, deleting, and switching databases.

3. Basic commands about collections

Basic commands of Python MongoDB database about collections

Collections are an important concept in MongoDB when working with the MongoDB database in Python. Collections are similar to tables in relational databases, but MongoDB is a document database, so documents are stored in collections.

The following are the basic commands of the Python MongoDB database on collections :

create collection

To create a collection, you can use MongoDB's db.create_collection()methods. In Python, collections can be created using methods of classes pymongoin the library .Collectioninsert_one()

import pymongo

# 连接MongoDB数据库
client = pymongo.MongoClient("mongodb://localhost:27017/")

# 创建一个名为mycollection的集合
mydb = client["mydatabase"]
mycol = mydb["mycollection"]

insert document

To insert documents into a collection, you can use MongoDB's db.collection.insert()methods. In Python, you can use methods of classes pymongoin the library to insert documents.Collectioninsert_one()

import pymongo

# 连接MongoDB数据库
client = pymongo.MongoClient("mongodb://localhost:27017/")

# 获取mycollection集合
mydb = client["mydatabase"]
mycol = mydb["mycollection"]

# 插入一条文档
mydict = { "name": "John", "address": "Highway 37" }
x = mycol.insert_one(mydict)

query document

To query documents in a collection, you can use MongoDB's db.collection.find()methods. In Python, documents can be queried using methods of classes pymongoin the library .Collectionfind()

import pymongo

# 连接MongoDB数据库
client = pymongo.MongoClient("mongodb://localhost:27017/")

# 获取mycollection集合
mydb = client["mydatabase"]
mycol = mydb["mycollection"]

# 查询所有文档
for x in mycol.find():
  print(x)

update document

To update documents in a collection, you can use MongoDB's db.collection.update()methods. In Python, you can use methods of classes pymongoin the library to update documents.Collectionupdate_one()

import pymongo

# 连接MongoDB数据库
client = pymongo.MongoClient("mongodb://localhost:27017/")

# 获取mycollection集合
mydb = client["mydatabase"]
mycol = mydb["mycollection"]

# 更新一条文档
myquery = { "address": "Highway 37" }
newvalues = { "$set": { "address": "Park Lane 38" } }
mycol.update_one(myquery, newvalues)

delete document

To delete documents in a collection, you can use MongoDB's db.collection.remove()methods. In Python, you can use methods of classes pymongoin the library to delete documents.Collectiondelete_one()

import pymongo

# 连接MongoDB数据库
client = pymongo.MongoClient("mongodb://localhost:27017/")

# 获取mycollection集合
mydb = client["mydatabase"]
mycol = mydb["mycollection"]

# 删除一条文档
myquery = { "address": "Park Lane 38" }
mycol.delete_one(myquery)

4. Data type

The MongoDB database supports the following data types :

String (string): store text data, the most commonly used data type.
Integer (integer): store integers.
Boolean (Boolean): stores a Boolean value (true/false).
Double (double-precision floating-point): Stores floating-point numbers.
Min/Max keys: Minimum and maximum values for comparing values.
Arrays (array): used to store arrays or lists.
Timestamp: Records when a document was modified or added.
Object (object): for embedded documents.
Null (empty value): Used to represent a null value or a field that does not exist.
Symbol: Like a string, but for a specific context.
Date (date): used to store date and time.
Object ID (object ID): used to uniquely identify the document.
Binary data (binary data): used to store binary data, such as pictures, audio, etc.
Code (code): used to store JavaScript code.
Regular expression: used to store regular expressions.

5. Data Operation

There are many ways to operate the MongoDB database in Python. You can use the official pymongolibrary or a third-party library mongoengine.

The following are pymongothe basic data operations of the library:

Connect to the database

import pymongo

client = pymongo.MongoClient('mongodb://localhost:27017/')
db = client['testdb']

insert data

collection = db['testcollection']
data = {'name': 'Tom', 'age': 18}
collection.insert_one(data)

Query data

# 查询所有数据
result = collection.find()
for item in result:
    print(item)

# 查询指定条件的数据
result = collection.find({'name': 'Tom'})
for item in result:
    print(item)

update data

# 更新一条数据
collection.update_one({'name': 'Tom'}, {'$set': {'age': 20}})

# 更新多条数据
collection.update_many({'name': 'Tom'}, {'$set': {'age': 20}})

delete data

# 删除一条数据
collection.delete_one({'name': 'Tom'})

# 删除多条数据
collection.delete_many({'name': 'Tom'})

aggregation operation

# 统计数据总数
count = collection.count_documents({})

# 按条件分组统计
result = collection.aggregate([
    {'$group': {'_id': '$name', 'count': {'$sum': 1}}}
])
for item in result:
    print(item)

The above is pymongothe basic data operation of the library, and the operation mode of other libraries is similar

6. Data backup and recovery

Backing up and restoring the MongoDB database in Python can use mongodumpthe and mongorestorecommand, or use pymongothe functions in the library to operate.

Backup and restore MongoDB databases using mongodumpand commandsmongorestore

Backup the MongoDB database:

mongodump --host <host> --port <port> --db <database> --out <backup_directory>

Among them, is the IP address or host name of the MongoDB server, is the port number of the MongoDB server, the default is 27017, is the name of the database to be backed up, and <backup_directory> is the directory where the backup file is stored.

Restore the MongoDB database:

mongorestore --host <host> --port <port> --db <database> <backup_directory>

Among them, , , and <backup_directory> have the same meaning as the backup command.

pymongoBackup and restore MongoDB databases using the library

Backup the MongoDB database:

import pymongo
import datetime

client = pymongo.MongoClient("<mongodb_uri>")
db = client["<database>"]
collection = db["<collection>"]

backup = collection.find()

with open("<backup_file>", "w") as f:
    for doc in backup:
        f.write(str(doc) + "\n")

Among them, <mongodb_uri> is the URI of the MongoDB server, and is the name of the database and collection to be backed up, and <backup_file> is the path of the backup file.

Restore the MongoDB database:

import pymongo

client = pymongo.MongoClient("<mongodb_uri>")
db = client["<database>"]
collection = db["<collection>"]

with open("<backup_file>", "r") as f:
    for line in f:
        doc = eval(line.strip())
        collection.insert_one(doc)

Among them, <mongodb_uri>,, and <backup_file> have the same meaning as the backup command. Note that this method is only suitable for backing up and restoring small databases. For large databases, it is recommended to use mongodumpand mongorestorecommands to operate.

7. Aggregation operation

MongoDB's aggregation operation refers to the process of grouping, filtering, sorting, computing, and other operations on multiple documents to generate a new document collection. The pymongo library can be used in Python to perform MongoDB aggregation operations.

Here is an example of a simple aggregation operation, assuming we have a collection called students that contains each student's name, age, and grade:

from pymongo import MongoClient

client = MongoClient()
db = client['test']
collection = db['students']

pipeline = [
    {"$group": {"_id": "$age", "avg_score": {"$avg": "$score"}}},
    {"$sort": {"_id": 1}}
]

result = collection.aggregate(pipeline)

for doc in result:
    print(doc)

In the above code, we used MongoDB's group and group and avg operators to calculate the average grade for each age group, and used the $sort operator to sort by age in ascending order. The final result will be a collection of documents containing average grades for each age group.

In addition to group and group and avg operators, MongoDB also provides many other aggregation operators, such as match, match, project, sort, sort, limit, etc., which can be combined according to specific needs.

8. Index

An index in MongoDB is a data structure used to improve query performance. In MongoDB, various types of indexes can be used, including single-field indexes, compound indexes, full-text indexes, etc.

The following is an example of using Python to manipulate MongoDB indexes:

Create a single-field index

from pymongo import MongoClient

client = MongoClient()
db = client.test_database
collection = db.test_collection

# 创建单字段索引
collection.create_index("name")

# 查询是否创建成功
print(collection.index_information())

Create composite index

from pymongo import MongoClient

client = MongoClient()
db = client.test_database
collection = db.test_collection

# 创建复合索引
collection.create_index([("name", 1), ("age", -1)])

# 查询是否创建成功
print(collection.index_information())

Create full text index

from pymongo import MongoClient

client = MongoClient()
db = client.test_database
collection = db.test_collection

# 创建全文索引
collection.create_index([("content", "text")])

# 查询是否创建成功
print(collection.index_information())

query using index

from pymongo import MongoClient

client = MongoClient()
db = client.test_database
collection = db.test_collection

# 查询使用索引
result = collection.find({"name": "Tom"}).explain()["executionStats"]
print(result)

The above is an example of Python operating MongoDB indexes. Different types of indexes can be selected according to actual needs to improve query performance.

Nine, pymongo operation

To operate the MongoDB database in Python, you need to use pymongothe library. The following are pymongothe common operations of the library:

Connect to MongoDB database

import pymongo

client = pymongo.MongoClient('mongodb://localhost:27017/')

select database

db = client['database_name']

select collection

collection = db['collection_name']