The use of mongoDB for smart forms

I met a more complex module in my work last year: smart forms. The smart form is actually the same as the questionnaire star. The customer designs the content style of the form according to their needs, then publishes it to collect the data filled in by everyone, and finally conducts statistical analysis and generates a chart.

The first problem with smart forms was the design of the database. The uncertainty of the form fields and the flexibility of the database requirements are very large. There are three main solutions after thinking:

1. Dynamically generate database tables according to user-designed form content

2. Use a very large number of fields to accommodate the maximum number of fields that customers may require.

3. In-memory database.

First of all, because the number of customers is not too small, the first method of dynamically generating tables was rejected, for fear that the database would be too burdened. The second one felt a bit too stupid haha, and then the third one was chosen. Due to the need for statistical analysis, mongDB was chosen as the final solution.

Next, I will summarize how to integrate mongoDB into spring mvc at work, because this module actually has a lot to do, and only give some simple examples related to mongoDB:

1. Needless to say, import the coordinates into pom.xml and choose the version by yourself

<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-mongodb</artifactId>
<version>1.10.1.RELEASE</version>
</dependency>
       <dependency>
           <groupId>org.mongodb</groupId>
           <artifactId>mongo-java-driver</artifactId>
           <version>3.2.2</version>
       </dependency>

2. Configure mongoDB data source

1. Configure <context:property-placeholder location="classpath:application.properties" ignore-unresolvable="true"/> in applicationContext.xml to set read external resources to facilitate configuration database information ignore-unresolvable="true" if already If there is an external resource configured, this property needs to be set, otherwise only one is read by default.

2. Introduce mongoBD configuration Introduce <import resource="spring-mongo.xml"/> in applicationContext to get a separate file.

3. Write the content of spring-mongo.xml

<context:property-placeholder location="classpath:mongo.properties" ignore-unresolvable="true"/>

<mongo:mongo-client replica-set="${mongo.hostport}" credentials="${mongo.username}:${mongo.password}@${mongo.dbname}" id="mongo">
<mongo:client-options connections-per-host="${mongo.connectionsPerHost}"
threads-allowed-to-block-for-connection-multiplier="${mongo.threadsAllowedToBlockForConnectionMultiplier}"
connect-timeout="${mongo.connectTimeout}" max-wait-time="${mongo.maxWaitTime}"
socket-timeout="${mongo.socketTimeout}"/>
</mongo:mongo-client>

<mongo:db-factory id="mongoDbFactory" dbname="${mongo.dbname} "mongo-ref="mongo"/>

<bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate">
<constructor-arg name="mongoDbFactory" ref=" mongoDbFactory"/>
</bean>


<mongo:repositories base-package="com.dg.ptm.model.mongo "/>//In fact, the mapping of database fields is not needed in the smart form, so the introduction here is just for learning at the time, and it is not really used.

3. Write the dao layer for ease of use. Because there are many, only a few are listed so that you can remember them in the future.

List one:

Interface method:

/**
* Perform a fuzzy query based on a field condition
* @param field field name
* @param param parameter value
* @param cls type
* @return
*/
public List fuzzySearchByOneParam(String field, String param, Class cls);

Implementation:

   @Override
   public List fuzzySearchByOneParam(String field, String param, Class cls) {
       return getMongoTemplate().find(new Query(Criteria.where(
               field).regex(".*?\\" +param+ ".*")), cls);
   }

List two:

Interface method:

/**
* Insert json format document
into the collection * @param jsonStr json to be inserted
* @param collectionName collection name
*/
public void insertJSONObject(String jsonStr, String collectionName) throws Exception;

Implementation:

   @Override
   public void insertJSONObject(String jsonStr, String collectionName) throws Exception{
       DBCollection collection = getMongoTemplate().getCollection(collectionName);
       BasicDBObject document = (BasicDBObject) JSON.parse(jsonStr);
       collection.insert(document);
   }

There is also a more complicated method of grouping statistics, but my own method is a bit dull. I hope that there is a better way to teach and provide a better method. There are still many deficiencies in the mongo pipeline.

List three

Interface method:

   /**
   * Group and query matching data according to the input retrieval conditions
   * @param query query condition
   * @param groupKey group field
   * @param operate operation name
   * @param selectName display field
   * @param collectionName document name
   * @return
   * @throws Exception
   */
   public String findJsonObjectAndGroupByQuery(BasicDBObject query, String groupKey, String operate, String selectName, String collectionName) throws Exception;

Implementation:

   public String findJsonObjectAndGroupByQuery(BasicDBObject query, String groupKey, String operate, String selectName, String collectionName)
           throws Exception {
       DBCollection collection = getMongoTemplate().getCollection(collectionName);
       DBObject keys = new BasicDBObject(groupKey, 1);
DBObject initial = new BasicDBObject("count", 0);
initial.put("result", 0);
String reduce = "";
String finalize = "";
if("avg".equals(operate)) { // 求平均数
   reduce = "function(doc,out){out.count++; out.result = out.result + Number(doc." + selectName + ");}";
   finalize = "function(out){out.result = out.result / out.count; return out;}";
} else if("sum".equals(operate)) { // 求总和
   reduce = "function(doc,out){out.count++; out.result = out.result + Number(doc." + selectName + ");}";
   finalize = "function(out){return out;}";
} else if("max".equals(operate)) { // 求最大
   reduce = "function(doc,out){out.count++; if(out.result < Number(doc." + selectName + "))out.result = Number(doc." + selectName + ");}";
   finalize = "function(out){return out;}";
} else if("min".equals(operate)) { // 求最小
   initial.put("result", Long.MAX_VALUE);
   reduce = "function(doc,out){out.count++; if(out.result >Number(doc." + selectName + "))out.result = Number(doc." + selectName + ");}";
   finalize = "function(out){if(out.result==" + Long.MAX_VALUE + ") {out.result = 0} return out;}";
} else if("count".equals(operate)) { // 求总数
   reduce = "function(doc,out){out.count++;}";
   finalize = "function(out){out.result = out.count; return out;}";
} else {
   throw new Exception();
}

BasicDBList dbList = (BasicDBList) collection.group(keys, query, initial, reduce, finalize);
ListIterator<Object> listIterator = dbList.listIterator();
Gson gson = new Gson();

List<Map> resultList = new ArrayList<>();
while(listIterator.hasNext()) {
   Map resultMap = gson.fromJson(String.valueOf(listIterator.next()), new TypeToken<Map>() {}.getType());
   resultList.add(resultMap);
}
       return gson.toJson(resultList);
   }

Many command line implementations of mongDB are very similar to js, so when I implement grouping, I actually use the implementation method similar to js language. I can't remember why it wasn't implemented using the Aggregation pipeline operation. . . I can't remember exactly what difficulties I encountered, so I switched to this method. It seems that due to the problem of data type, correct statistics could not be performed.

Supplement: Lastly, if you need to perform mapping operations using mongoDB, you need to declare the annotation @Document on the pojo, so that the id in your class will correspond to the default primary key _id in mongoDB, otherwise it will be considered a new field , that's it.

The use of mongoDB for smart forms

Guess you like