NoSQL, the full name of Not Only SQL, means not only SQL, but also refers to non-relational databases. NoSQL is based on key-value pairs, and does not need to be parsed by the SQL layer, there is no coupling between data, and the performance is very high.
Non-relational databases can be further subdivided as follows.
- Key-value storage database : Redis, Voldemort and Oracle BDB are represented.
- Column store database : Representatives include Cassandra, HBase, and Riak.
- Document database : Representatives include CouchDB and MongoDB.
- Graph database : Representatives include Neo4J, InfoGrid, and Infinite Graph.
For the crawler's data storage, a piece of data may fail to extract some fields and be missing, and the data may be adjusted at any time. In addition, there are nested relationships between data. If you use relational database storage, firstly, you need to build a table in advance, and secondly, if there is a data nesting relationship, you need to serialize it before it can be stored, which is very inconvenient. If you use a non-relational database, you can avoid some trouble and be simpler and more efficient.
In this section, we mainly introduce the data storage operations of MongoDB and Redis.