Initialize and import json file data based on docker (docker-compose) mongodb

Recently, when I was cut leeks, I encountered a problem, that is, how to import some data into it when initializing the mongodb container, and import it from a file with a json suffix. I rummaged through the domestic platforms, but I couldn't solve anything. I read the official usage of mongo in dockerhub, and it was vague. So, I went to stackoverflow and google to search, and then I finally figured it out after tossing around for a while. Here are two related links:

• Initialize MongoDB running on a Docker container
• https://time-is-life.fun/how-to-seed-data-into-mongo-with-docker

Refer to the above two links to basically solve it, but I encountered a few pitfalls, so record it

premise

dockerhub mongo

Three environment variables:

1. MONGO_INITDB_ROOT_USERNAME
2. MONGO_INITDB_ROOT_PASSWORD
3. MONGO_INITDB_DATABASE

The first two are easy to understand, and the third one looks at the sophistry of the official website:

This variable allows you to specify the name of a database to be used for creation scripts in /docker-entrypoint-initdb.d/*.js (see Initializing a fresh instance below). MongoDB is fundamentally designed for “create on first use” , so if you do not insert data with your JavaScript files, then no database is created.
That is, this database will only be generated if you do an insert operation in the /docker-entrypoint-initdb.d/*.js file. (mongodb can execute .js files and .sh files under the /docker-entrypoint-initdb.d folder）

Then the official website on how to initialize the database instance:

Initializing a fresh instance
When a container is started for the first time it will execute files with extensions .sh and .js that are found in /docker-entrypoint-initdb.d. Files will be executed in alphabetical order. .js files will be executed by mongo using the database specified by the MONGO_INITDB_DATABASE variable, if it is present, or test otherwise. You may also switch databases within the .js script.

hands-on final version

ok, after reading the two links and official website documents, I made the following configuration:
First, look at the directory structure.

Here, note that when there are multiple sh and js files in the /docker-entrypoint-initdb.d folder, they are in alphabetical order Executed, so here js must be executed before sh, first create the database and collection, and then import the json data.

docker-compose.ymlAs follows (this is useless MONGO_INITDB_DATABASE, because $MONGO_INITDB_DATABASEit will report an error when added to the sh file, and it has not been resolved yet, you can try it yourself, you can see here: How can I pass environment variables to mongo docker-entrypoint-initdb.d?） :

...
  jiefang-mongo:
    container_name: jiefang-mongo
    image: mongo
    build:
      context: ./mongo
      dockerfile: dockerfile
    ports:
      - "27017:27017"
    volumes:
      - ./mongo/conf/mongod.conf:/etc/mongod.conf
      - ./mongo/data:/data/db
      - ./mongo/init:/docker-entrypoint-initdb.d
...

Mongo is dockerfileas follows (it doesn’t work, it mainly caches the original image, if you make the image directly in compose, you need to pull the original image again every time you change it):

FROM mongo
MAINTAINER bty

01-init.jsThe file is as follows:

// Create user
dbAdmin = db.getSiblingDB("admin");
dbAdmin.createUser({
    
    
  user: "mongo",
  // 希望有机会值10个btc
  pwd: "10btc",
  roles: [{
    
     role: "userAdminAnyDatabase", db: "admin" }],
  mechanisms: ["SCRAM-SHA-1"],
});

// Authenticate user
dbAdmin.auth({
    
    
  user: "mongo",
  pwd: "10btc",
  mechanisms: ["SCRAM-SHA-1"],
  digestPassword: true,
});

// Create DB and collection
db = new Mongo().getDB("jiefangProj");
db.createCollection("t_xcp", {
    
     capped: false });

02-init.shas follows:

echo "########### Loading data to Mongo DB ###########"
mongoimport --jsonArray --db jiefangProj --file /docker-entrypoint-initdb.d/data.json --collection t_xcp
echo "########### Mongo DB data Loaded ###########"

Pit point

This 02-init.shmade me miserable, at first it was:

echo "########### Loading data to Mongo DB ###########"

mongoimport --jsonArray --db jiefangProj --collection t_xcp --file /docker-entrypoint-initdb.d/data.json 
echo "########### Mongo DB data Loaded ###########"

Then an error was reported, saying that \nit was not a command, but it was caused by spaces and newlines, and then I made the sh file compact

echo "########### Loading data to Mongo DB ###########"
mongoimport --jsonArray --db jiefangProj --collection t_xcp --file /docker-entrypoint-initdb.d/data.json 
echo "########### Mongo DB data Loaded ###########"

Then I keep reporting an error: I can't find this file or folder, and I have been doing it all afternoon.

I changed the parameter again:

echo "########### Loading data to Mongo DB ###########"
mongoimport --db jiefangProj --collection t_xcp --file /docker-entrypoint-initdb.d/data.json --jsonArray
echo "########### Mongo DB data Loaded ###########"

Then I also reported an error, but the error was different, saying that jsonArray was not recognized, so I used my smart brain to think whether there was a problem with the sh file format, but I still couldn't find the reason after working for a long time.
Finally, after dinner, I changed the parameter position casually:

echo "########### Loading data to Mongo DB ###########"
mongoimport --jsonArray --db jiefangProj --file /docker-entrypoint-initdb.d/data.json --collection t_xcp
echo "########### Mongo DB data Loaded ###########"

It worked!

Postscript, in fact, it still doesn’t work. In the end, I can’t delete the two echo commands, and only keep which mongoimport. This is really no problem.

---

After a few days, I found that there was a problem with data security, so I improved as follows:

data stolen

It was found that the mongodb data was always deleted, and there was a message displayed in a new collection. This person's English is not very good, and there are too many wrong sentences:

All your data is a backed up. You must pay 0.01 BTC to 12nx7Q6FAczH6yfhaEMJRmvRkavDTcwUJK 48 hours for recover it. After 48 hours expiration we will leaked and exposed all your data. In case of refusal to pay, we will contact the General Data Protection Regulation, GDPR and notify them that you store user data in an open form and is not safe. Under the rules of the law, you face a heavy fine or arrest and your base dump will be dropped from our server! You can buy bitcoin here, does not take much time to buy https://localbitcoins.com with this guide https://localbitcoins.com/guides/how-to-buy-bitcoins After paying write to me in the mail with your DB IP:
[email protected]
[email protected]

I was also drunk, and found that the password I set before was useless, and I could log in directly. The reason is that the authentication login is not turned on. After it is turned on, the mongod.conffile is as follows:

# mongod.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# Where and how to store data.
storage:
  dbPath: /data/db
  journal:
    enabled: true
#  engine:
#  wiredTiger:

# where to write logging data.
systemLog:
  logAppend: true
  destination: file
  logRotate: reopen
  path: "/var/log/mongodb/mongod.log"
# network interfaces
net:
  port: 27017
  # 0.0.0.0就是可以远程登陆，不限制bind
  bindIp: 0.0.0.0


# how the process runs
processManagement:
  timeZoneInfo: /usr/share/zoneinfo

security:
  # 这里，验证登陆要打开
  authorization: enabled
# 慢查询日志
operationProfiling:
  mode: slowOp
  slowOpThresholdMs: 100
  

Then I fiddled a few times and found that the above methods of initializing the database and creating users always have permission problems, such as not being able to view the contents of the jiefangProj database. Finally, I made a reference to a question and answer on StackOverflow to solve it.
How to create a DB for MongoDB container on start up?

Then I sort out my entire requirements:

1. Create ROOT user for mongodb
2. Create a new database jiefangproj
3. Create a separate user for the jiefangproj database
4. Import the json file into the t_xcp collection of jiefangproj

OK, here is my operation

docker-compose.ymlas follows:

version : '3.8'
services:
  jiefang-mongo:
    container_name: jiefang-mongo
    image: mongo
    build:
      context: ./mongo
      dockerfile: dockerfile
    ports:
      - "27017:27017"
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: rootpwd
      MONGO_INITDB_DATABASE: jiefangproj # js文件的操作会在该数据库下完成
    volumes:
      - ./mongo/conf:/etc/mongo # 配置文件
      - ./mongo/log:/var/log/mongodb # 日志目录挂载
      - ./mongo/data:/data/db # 数据目录挂载
      - ./mongo/init:/docker-entrypoint-initdb.d # 初始化操作
    command: --config /etc/mongo/mongod.conf # 指定上面的配置文件运行

The files in the init folder are shown in the figure above.

jsThe file is as follows

db.createUser({
    
    
  user: "username",
  pwd: "password",
  // 这个role有很多内置的角色
  // 具体参考https://www.mongodb.com/docs/manual/core/authorization/
  roles: [{
    
     role: "dbOwner", db: "jiefangproj" }],
  mechanisms: ["SCRAM-SHA-1"],
});

shThe file is as follows:

mongoimport --jsonArray --db jiefangproj  --collection t_xcp --file /docker-entrypoint-initdb.d/data.json