Features of sqlite

1. ACID transactions [1] 

2. Zero configuration – no need to install and manage configuration

3. A complete database stored in a single disk file

4. Database files can be freely shared between machines with different byte orders

5. Database support Size up to 2TB

6. Small enough, roughly 130K lines of C code, 4.43M

7. Faster than some popular databases in most common database operations

8. Simple, easy API

9. Includes TCL bindings, and support via Wrapper Bindings for other languages10.

Well-annotated source code, and has more than 90% test coverage11

. Independent: no additional dependencies12

. The source code is completely open source, you can use it for any purpose, including selling

it13. Supports many Various development languages, C, C++, PHP, Perl, Java, C#, Python, Ruby, etc. In-

memory database: the new favorite of data management in the era of big data

At the 2012 China System Architect Conference, the author once did a survey on big data, In one of the survey results of "Among the many technology trends, what are the new technologies for data management that you are concerned about?", "In-memory database" became the most concerned new technology after "Distributed Storage and Computing". Technology. The reason why the in-memory database has received more and more attention is closely related to the leap in performance and the continuous improvement of cost performance.

An in-memory database, as the name suggests, is a database that directly operates data in memory. Compared with disk, the speed of reading and writing data in memory is several orders of magnitude higher. Saving data in memory can greatly improve the performance of applications compared to accessing from disk. At the same time, the in-memory database abandoned the traditional method of disk data management, redesigned the architecture based on all data in memory, and made corresponding improvements in data caching, fast algorithms, and parallel operations, so the data processing speed is faster than traditional The data processing speed of the database is much faster, generally more than 10 times. The biggest feature of an in-memory database is that its "master copy" or "working version" is resident in memory, that is, active transactions only deal with the in-memory copy of the real-time in-memory database.

Similarities and Differences Between In-

Memory The goal of in-memory databases is to increase throughput and reduce latency by using memory for data storage. This is different from traditional database management systems that use disk storage, where in-memory data is faster than disk-based databases due to simpler internal optimization algorithms and fewer CPU instructions to execute. Accessing in-memory data can improve responsiveness, and in-memory databases are generally used for applications with high response time requirements, such as transactions, telecommunications, and defense systems. Because of this characteristic of in-memory databases, these databases use more memory than disk database products. The specific differences are as follows:

1. The traditional database system is a relational database, and the purpose of developing this database is to process permanent and stable data. Relational database emphasizes maintaining the integrity and consistency of data, but it is difficult to take into account the timing constraints of data and its processing, and cannot meet the needs of real-time applications in industrial production management, because real-time transactions require the system to accurately predict the running time of transactions.

2. For disk databases, due to disk access, data transfer between internal and external memory, buffer management, queuing, and lock delays, the actual average execution time of transactions is very different from the estimated worst-case execution time. The entire database or its main "work" part is put into memory, so that there is no I/O during the execution of each transaction, then the system can more accurately estimate and arrange the running time of the transaction, so that it has better dynamic predictability Provides strong support, and also lays the foundation for implementing transaction timing constraints. This is the main reason for the emergence of in-memory databases.

3. The data processed by the in-memory database is usually "short-lived", that is, it has a certain valid time. When outdated, new data is generated, and the current decision-making derivation becomes invalid. Therefore, in practical applications, an in-memory database is used to process data with strong real-time business logic. The traditional database is designed to process permanent and stable data, and its performance goals are high system throughput and low cost, and the real-time nature of data processing is relatively less considered. In practical applications, the characteristics of traditional databases are used to store data with relatively low real-time requirements.

In practical applications, these two databases are often used in combination, rather than replacing traditional databases with in-memory databases.

·SQLite

SQLite is a lightweight database, it occupies very low resources, and can be combined with many programming languages, but the supported SQL statements are not inferior to other open source databases. Its design goal is embedded, and it has been used in many embedded products. It occupies very low resources. In embedded devices, only a few hundred K of memory may be enough. It can support mainstream operating systems such as Windows/Linux/Unix, and can be combined with many programming languages, such as Tcl, PHP, Java, etc., as well as ODBC interfaces. In terms of database management systems, its processing speed is faster than all of them.

· Redis

Redis is an open source, high-performance key-value store (key-value store). It is often referred to as a data structure server. Redis keys can include data types such as strings, hashes, lists, sets, and sorted sets. For these data types, you can perform atomic operations. For example: appending to strings; incrementing the value in the hash; adding elements to the list; computing the intersection, union, and difference of sets, etc.


SQLite differs from most other SQL database engines because its primary design goal is simplicity:

•Easy to manage


•Easy to use


•Easy to embed in other large programs


•Easy to maintain and configure
Many people like SQLite for its small size and speed. But these features are only some of its advantages, users will also find that SQLite is very stable. Excellent Stability stems from its simplicity. The simpler it is, the less likely it is to make mistakes. In addition to the above-mentioned simplicity, compactness and stability, the most important thing is that SQLite strives to be simple.


Simplicity can be said to be an advantage in a database engine , but it can also be a disadvantage, depending on what you want to do. To achieve simplicity, SQLite omits some features that people find useful, such as high concurrency, strict access control, rich built-in functions, stored procedures , complex SQL language features, XML and Java extensions, huge trillion-level data measurements, etc. If you need to use these features and don't mind their complexity, then SQLite may not be for you. SQLite It is not intended to be an enterprise database engine, nor is it intended to compete with Oracle or PostgreSQL.


From experience, SQLite is suitable for the following situations: When you are more interested in the simple management, use and maintenance of the database, rather than the enterprise class Using SQLite is a wise choice when the database provides countless complex functions. It has also been proven that in many cases it has become clear that simplicity is the best choice.


SQLite Best Trial


Site•Website
As a database engine SQLite is suitable for websites with small and medium traffic (that is, 99.9% of websites). How much website traffic SQLite can handle depends on how much pressure the website's database has. Generally speaking, if a website has less than 100,000 hits times/day, SQLite can run normally. 100,000 times/day is a conservative estimate, not an accurate upper limit. It turns out that SQLite can still run normally even at 10 times the above traffic.

•Embedded Equipment and application software


Because the SQLite database requires almost no management, SQLite is a good choice for devices or services that run unattended or without human technical support. SQLite is well suited for mobile phones, PDAs, set-top boxes, and other instruments. As an embedded database it also works well in client programs.

• The application file format


SQLite has been a great success as a native disk file format for desktop applications. For example financial analysis tools, CAD packages, archives Management programs, etc. General database open operations need to call the sqlite3_open() function, and mark the start point of an explicit local transaction (BEGIN TRANSACTION) to ensure that the contents of the file are obtained in an exclusive manner. The file save will perform a commit (COMMIT ) at the same time marks another explicit local transaction starting point. The role of this transaction processing is to ensure that the updates to the application data files are atomic, durable, independent and consistent.


Some temporary triggers can be added to the database, Used to record all changes in a temporary undo/redo log table. These changes can be rolled back when the user presses the undo/redo button. Apply this technique to achieve an infinite level of undo/redo Redo functionality requires very little coding.

• Replaces some special file formats
Many programs use the fopen(), fread(), or fwrite() functions to create and manage some custom files for storing data. Using SQLite instead of these custom file formats would be a good choice.




• Internal or a temporary database
for programs that have large amounts of data that need to be filtered and sorted differently, as opposed to writing code that does the same thing, if you read the data into an in-memory SQLite database, then use join queries and ORDER BY clauses It is usually simpler and faster to extract the required data in a certain order and arrangement. Using the embedded SQLite database as described above will make the program more flexible, because adding new columns or indexes does not require rewriting any query statements





•Command line data set analysis
tool Experienced SQL users can use the SQLite command line program to analyze a variety of mixed data sets . The original data can be imported from a CSV (comma separated value file) file, and then sliced ​​to generate countless Comprehensive data reporting. Possible uses include website log analysis, sports statistics analysis, editing planning criteria, analyzing test results.

Of course you can also do the same with an enterprise-level client/server database. In this case use The benefits of SQLite are: SQLite is easier to deploy and the resulting database is a separate file that you can store on a floppy or USB stick or email to a colleague.





• As an enterprise database during Demo or Beta Alternatives
If you're writing a client program that uses an enterprise database engine, it makes sense to use a general-purpose database backend that allows you to connect to different SQL database engines. The bigger point is to make the SQLite database engine static Connect to the client program to embed SQLite as a hybrid database support. This way the client program can use the SQLite database file for independent testing or verification.





•Database teaching
Because the installation and use of SQLite is very simple (the installation process is almost negligible, you only need to copy the SQLite source code or sqlite.exe executable file to the target host, and then run it directly), so it is very suitable for explaining SQL statements. They can easily create a database of their liking, and then email the teacher to annotate or rate it. For advanced students who are interested in how to implement a relational database management system (RDBMS), it is modular in design and has great Well commented and documented SQLite source code will give them a good foundation. This is not to say that SQLite is an exact model of how other database engines are implemented, but it is good for students to understand how SQLite works quickly and thus to master other databases System design and implementation principles.





• Experimenting with extensions to the SQL language
SQLite's simple and modular design makes it an excellent prototyping platform for testing database language features or new ideas.

Where are other relational database management systems suitable for use? (RDBMS)

• Client/Server Programs
If you have many client programs accessing a shared database over the network, you should consider using a client/server database instead of SQLite. SQLite can work over a network file system, but because And most network file systems have delays, so the execution efficiency will not be very high. In addition, most network file systems have bugs in implementing logical file locks (including Unix and Windows). If the file lock does not work properly , it may occur that two or more client programs change the same part of the same database at the same time, resulting in database errors. Because these problems are inherently bugs in the execution of the file system, there is no way for SQLite to avoid them. .Good


experience tells us that SQLite should be avoided in situations where many computers need to access the same database at the same time through a network file system.




• High traffic website
SQLite usually works well as a backend database for a website. But if your website gets so much traffic that you start to consider a distributed database deployment, then you shouldn't hesitate to consider using an enterprise-class database A client/server database instead of SQLite.




• Very large datasets
when you start a transaction in SQLite (transactions are generated before any write operations occur, not necessarily explicit calls to BEGIN...COMMIT) , the database engine will have to allocate a small chunk of dirty pages (file buffer pages) to help itself manage rollback operations. SQLite requires 256 bytes per 1MB database file. For small databases this space is not much, but when the database When it grows to gigabytes, the size of the buffered pages can be quite large. If you need to store or modify tens of gigabytes of data, you should consider using other database engines.





• High concurrent access to
SQLite for the entire database The file is read/write locked. This means that if any process reads a part of the database, no other process can write to any part of the database. Likewise, if any process is The database is written to, and no other process can read any part of the database. For most cases this is not a problem. In these cases each program's use of the database is brief and not exclusive, This lock will exist at most ten milliseconds. But if some programs require high concurrency, then these programs need to find other solutions.