MongoDB-how to import csv data into a table in mongodb database

background introduction

The background is that the development suddenly asked me if I could import data to the database, and then only need a few columns of data.

My first thought was: read the csv file with a python script, splice the content into a text in json format, and then import it in a script. Later, I found that the GUI tool I used can directly import data into the database.

Implementation process

Since there are tools that can be imported directly, it must be imported with ready-made tools. For tools, I choose here: NoSQLBooster, official website download address: https://nosqlbooster.com/downloads

The operation steps are as follows:

1. To import data to which database, first select the database, then right-click, select Import Collections, and then select the Import from JSON and CSV files option

2. Select which columns to import on the page, and the field types of the corresponding columns, and then click Execute:

The contents of the csv file selected here are as follows:

The effect after importing the database:

If you need to reorder the fields after importing data, you can use the following statement:

格式：
db.集合名称.update({}, {$rename:{"旧键名称":"新键名称"}}, false, true)
举例：将import_test表中的age字段改为大写的AGE字段：
db.import_test.update({}, {$rename:{"age":"AGE"}}, false, true)

I don’t know if my careful friends have noticed two problems with the data imported above:

1. The data involves Chinese, if it is imported, it will be garbled

2. If the csv file is similar to the mobile phone number, if the format is not set in the csv, the value will change after importing it.

Since the above tool imports may sometimes be unreliable, then let's try the way of using python scripts:

Not much nonsense, just upload the code directly: ( Note that when reading the file, you must specify the encoding, otherwise there may be unknown exceptions such as reading errors/garbled characters )

import pandas as pd
from pymongo import MongoClient


# 读取CSV文件
df = pd.read_csv('import_test.csv',encoding='GBK')
print(df.to_dict('records'))
# 连接MongoDB
client = MongoClient('localhost', 27017)
db = client['study']
collection = db['import_test']


# 将数据写入MongoDB
collection.insert_many(df.to_dict('records'))

operation result: