CouchDB
1 document storage
CouchDB server host is stored in a document database. Each document has a unique name in the database. CouchDB provides a RESTful HTTP API to read and update (add, edit, delete) database documentation.
CouchDB document is the main unit of data from any database fields and accessories. The document also includes metadata information maintained by the database system. Document field has a unique name and contain several types (text, number, boolean, lists, etc.) are. And the text size or number of elements is not limited.
CouchDB document update mode is no lock and optimization. The document is loaded by a client application for document editing, updating, storing and finally back to the database. If another client to edit the same document and save the first update. Then the client will get an error editing conflicts when storing. To resolve conflicting updates, the latest version of the document will open, edit, update will reapply and try again.
A single document update (add, edit, delete) is not an all or whole, either completely succeed or completely fail. Database does not support parallel storage or updated documents.
2ACID property
CouchDB file system layer, and having all of the characteristics submitted atomicity, consistency, isolation and durability (the ACID) properties. In the hard disk, CouchDB never covered or submitted data structure is connected. Ensure that the database files are always consistent, this is a "collapse only" design, where the CouchDB server does not perform the closing process, but simply terminates.
Documentation update (add, edit, delete) is serial, in addition to binary byte write in parallel. Database readers never be locked nor write operation is completed or wait for other readers when reading. Any number of clients can read a document need not be locked or parallel update is interrupted even if the same document. CouchDB read operation using multi-version concurrency control (MVCC) mode. See every client from the beginning to the end of the read operation is consistent database snapshots. This means that CouchDB can ensure transactional semantics of each document.
Documents are indexed by B-trees by their name and serial number. Examples of each update to a database will generate a new sequence number. Sequence ID in a database for later increments find the changes. Documents stored or deleted B-tree indexes will also be updated. Index update always occurs in the file (only added when updating) last.
Advantages of the document data is always convenient to pack and store rather than the many different database systems separated into multiple tables and rows. When the document is submitted to the hard disk, the document fields and metadata to serialize a document by one document (later contribute efficiently build view) packed into the buffer.
When CouchDB document is updated, all data and associated indexes will be flushed to the hard disk is always committed transactions in a manner such that the database remains in a consistent state. Submitted by two steps:
- All relevant documents and index data synchronization update and refresh to the hard disk.
- The updated first database to write two consecutive identical blocks to form the front file 4k, and then flushed to disk synchronization.
When a system crash or power step is closed, partially updated simply discard and restart. If a crash occurs in (the head of information submitted) Step two, still keep a copy of the same previous header, to ensure consistency of all previously submitted data. In addition to the header area, after a crash or power failure without the need for consistency check or repair.
3 Compression
By compression occasionally to reclaim wasted space. As planned, or when a database file exceeds a certain amount of wasted space, all activity data compression process will be cloned to the new file, and then discard the old file. Online databases remain fully throughout the process, and all updates and reading can be completed successfully. Only when all data has been copied and all users are transferred to the new file, just delete the old database files.
4 views
ACID properties only store and update process, but we also need the ability to display data of data, unlike SQL databases must be broken down to watch carefully interesting and useful way, the data in CouchDB to small storage mechanism in the document. CouchDB document to become more flexible and they have their own implicit structure. This relieves the table bidirectional replication mode and the most difficult problems and pitfalls that contains the data.
However, in addition to acting as a file server outside the beautiful, simple document model for data storage and sharing too simple, not build a real-world application in which - it simply can not meet our expectations and expectations. We want to slice and dice, and see our data in many different ways. Now we need a filter, organize and report has not been broken down into the form of a method of data.
4.1 view model
To address add structure back to unstructured and semi-structured data problems, CouchDB integrates a view model. And the polymerization is a view reporting database document, and constructed in the polymerization as needed, and coupled to the reporting database document. Because the view is dynamically constructed, and will not affect the basic document, so you can use different views of the same data as needed representation.
View definition is strictly virtual, database and displays only the current instance of the document, so that it is separated from the data displayed and is compatible with the copy. CouchDB view is special design document definition, and may be cross-database instance (e.g., a conventional document) copy, so data can be copied only in CouchDB, the application and the entire design may be copied.
4.2JavaScript View feature
A view is defined using JavaScript functionality, which acts as a functional part of the map in the map-reduce system. CouchDB view the document as function parameters, and any calculations necessary to determine the view provided by the data (if any). It can be based on a single document to add a multi-line view, you can also do not add any rows.
View index 4.3
View the actual document content is dynamic representation of the database, and CouchDB can easily create useful views of data. However, the generated contain hundreds of thousands or millions of documents in a database view is a waste of time and resources, but not every time the system from scratch to do.
In order to maintain rapid query view, the view engine maintains an index of its view and update the database to reflect the change in its increments. CouchDB's core design has been optimized to a large extent around the needs of views and their indexes efficient, incremental created.
Views and their functions are defined in a special "design" documents and design documents can contain any number of uniquely named view feature. When a user opens a view and automatically updates its index, all views of the same design documents are indexed as a group.
View builder database sequence ID to determine whether the group is completely synchronized with the database view. If not, then view the document database engine checks all changes since the last refresh (arranged in order packaged). Read the document in order of appearance in the disk file, thereby reducing the frequency and cost of disk head search.
You can simultaneously read and query views, but also can refresh the view. If the client is a large content slowly out of view, you can open and refresh the same view, without blocking the first client to another client. Client reader if this applies to any number of them can read and query views while at the same time to refresh the index for other clients, without giving the reader a problem.
When the view through your engine "map" and "Zoom" function processing documents, their view from the front row of the value of the index is deleted (if present). If you choose to view a document by function, the function result is inserted as a new row into the view.
After the view index changes are written to disk, updates are always added to the end of the file, in order to reduce disk head during disk seek time to submit and make sure crashes and power failures will not lead to index corruption. If a crash occurs while updating the index view, it is not a complete index update will be lost and gradually rebuild its state previously submitted.
5 Security and Authentication
In order to protect personnel can read and update the document, CouchDB has a simple reader to access and update validate the model, the model can be extended to implement custom security model.
5.1 Administrator Access
CouchDB database instance with administrator accounts. Administrator account can create additional administrator account and update design documents. Design document that contains view definitions and special formulas and other special document conventional fields and the Blob.
5.2 update verification
After the document is written to disk, you can use a JavaScript function to dynamically validate it, in order to achieve security and data validation. When a document is verified by all standard formula that will allow updates to continue. If the verification fails, updates will be discontinued, the user will receive an error response to the client.
User credentials and documents are updated as input validation formula, you can implement a custom security model by verifying document updates the user's permission.
Basic "only Author" Achieving update document model is very simple, which is listed to verify documentation updates check the user in the "Author" field of an existing document. You can also use more dynamic model, such as checking individual user account configuration file permissions.
For real-time usage and perform updates replicated updates are validated to ensure security and data validation shared in a distributed system.
6 Distributed update and replicate
CouchDB is a distributed database system nodes based. It allows users to access and update the server and share the same data when disconnected. These changes can then be bi-directional replication.
Store, view and security model CouchDB document is intended to work together to make a real efficient and reliable two-way replication. Documents and design can be copied, thereby allowing a complete database application (including application design, logic and data) to the portable computer for off-line use, or copied to a remote office connected to slow or unreliable, difficult shared server data.
Replication process is incremental. At the database level, just copy the document since the last inspection copy updated. If for reasons such as network problems or crashes copy fails at any step, then the next copy will restart at the last checkpoint.
You can create and maintain a partial replica. Replication may be filtered through a JavaScript function, so that only a particular document or copy documents that meet certain conditions. This can allow users to use offline large shared database applications subsets for their own use, while maintaining normal interaction with the application and the data subset.
6.1 Conflict
Collision detection and management is a key issue for any distributed editing system. CouchDB storage system will edit the conflict as a common state, rather than the exception state. Conflict management model is simple and "lossless" while preserving the semantics of a single document and allow decentralized conflict resolution.
CouchDB database allows any number of documents in the presence of conflicts at the same time, each database instance definitively determine which documents are "winners", which is a conflict. Only the winner documents can be displayed in the view, and the "lost" the conflict can still access and remain in the database until the database during compression to remove or cleared. Since the conflict document is still a regular document, they are the same as regular document copying, and to follow the same safety and validation rules.
When the distributed editing conflicts, each copy of the database will see the same winning version, and each has the opportunity to resolve the conflict. Resolve the conflict can be done manually, it can be done automatically by the agent and the nature of the data conflict. The system enables distributed conflict resolution possible, while maintaining a single document semantics of the database.
Even if multiple users connect or disconnect the proxy attempts to resolve the same conflict, conflict management also continue to function. If the solution of the conflict leads to more conflict, the system will be treated in the same way they determine the winner of the same on each machine, and maintain a single document semantics.
6.2 Application
Copy only the basic model, with almost no extra work can make many of the traditional single-server distributed database applications. CouchDB is designed to copy immediately for basic database applications, but also can be extended for a more detailed and fully functional use.
Database with little work, you can build distributed document management application with a fine safety and integrity revision history. Documentation can be implemented using the incremental update to field and Blob replication, wherein the replicated updates almost actual editing difference ( "difference") as an efficient and incremental.