Qt database application 15 - general database synchronization

I. Introduction

The main function of database synchronization is to synchronize the local database records to the remote database. The database type is not limited. For example, the local is a sqlite database, the remote can be a mysql database, the local is a mysql database, and the remote can also be a postgresql database, as long as the database is set when you set it up. This is equivalent to uploading and synchronizing database records through a program. Another mechanism is to set up a hot backup directly on the database. Individuals are still accustomed to using a program to control them independently. For example, you can customize which table to synchronize. Which part of the data needs to be synchronized.

There is an application scenario where many clients on site are connected to various sensor IoT devices (now there is a high-level name called pan-sensing), and the client is responsible for collecting the data of these devices, which are recorded locally, and then the records can be synchronized to the remote. You need to provide an app, web page, applet, etc. to get the corresponding data from the database. If there is no process of uploading to the cloud, it will be difficult to get the data. The cloud usually has an IP of the public network. The address is fixed, so that the app or the applet can directly send the request and reply with the corresponding data. In addition to being able to get the data, you can also send the corresponding operation to execute the command back control. For example, a separate table stores the corresponding command, and the specific command rules are customized. During the synchronization process, the client also actively queries whether the command table has records. Take it out and execute it according to the agreed rules. After the execution is successful, the corresponding record will be deleted.

Features of database communication management thread class:

1. The database type can be set, and multiple database types are supported.
2. Database types include but are not limited to odbc, sqlite, mysql, postgresql, sqlserver, oracle, NPC Jincang, etc.
3. The database connection information can be set, including host address, user information, etc.
4. With an automatic reconnection mechanism, you can set whether to check the connection and check the interval.
5. Supports a single SQL statement queue, which is generally used to query and return data, and execute one at a time.
6. Supports multiple SQL statement queues, which are generally used for remote submission of data, and multiple executions are performed each time one is inserted.
7. Support batch sql statement queue, generally used to update data in batches, insert multiple executions at a time.
8. The maximum number of queues can be set to limit the set of SQL statements that are queued for processing.
9. Send out print information, error information, and query results through signals.

Some experience summary about Qt database related development:
https://qtchina.blog.csdn.net/article/details/119022424

2. Features

1. At the same time, it supports a variety of databases such as odbc, sqlite, mysql, postgresql, sqlserver, oracle, NPC Jincang, etc.
2. A single database class can manage local database communication, as well as support remote database communication, etc.
3. The database thread supports the execution of various SQL statements, including single and batch.
4. All class printing information, error information, and execution results in the component are signaled.
5. Integrate the general page-turning class of the database (responsible for specific processing logic), and match the paging navigation control (responsible for appearance) to form a super awesome page-turning control.
6. Integrated database automatic cleaning class, setting the maximum number of records to automatically clean up early data in the background.
7. Integrate custom delegate classes to support checkboxes, text boxes, drop-down boxes, date boxes, spinners, progress bars, etc.
8. At the same time support Qt4-Qt6, test any version of Qt4.6 to Qt6.3, any system and compiler.
9. This component runs at least tens of thousands of sites 7 times 24 hours a day, 360 days without failure, with commercial-level quality assurance.
10. Each class corresponds to a complete and detailed usage example, with detailed annotations, which is very suitable for reading and learning.
11. It can be run as a standalone program, such as automatically cleaning up old data and syncing data to the cloud.
12. All threads are processed, no interface is stuck, and the database is automatically reconnected.
13. In the normal test situation, the sqlite database, the database generator inserts 1000 records per second for about 0.003 seconds, while the automatic cleaning data class deletes 1000 records per second for about 0.13 seconds, and different threads do not interfere with each other.

3. Experience address

1. Experience address: https://pan.baidu.com/s/1ZxG-oyUKe286LPMPxOrO2A Extraction code: o05q File name: bin_dbtool.zip
2. Domestic site: https://gitee.com/feiyangqingyun
3. International site: https://github.com/feiyangqingyun
4. Personal homepage: https://blog.csdn.net/feiyangqingyun
5. Know the homepage: https://www.zhihu.com/people/feiyangqingyun/

Fourth, the effect map

5. Relevant code

#include "frmdbupload.h"
#include "ui_frmdbupload.h"
#include "quihelper.h"
#include "dbconnthread.h"

frmDbUpload::frmDbUpload(QWidget *parent) : QWidget(parent), ui(new Ui::frmDbUpload)
{
    
    
    ui->setupUi(this);
    this->initForm();
    this->initConfig();
}

frmDbUpload::~frmDbUpload()
{
    
    
    delete ui;
}

void frmDbUpload::initForm()
{
    
    
    ui->frame->setFixedWidth(AppConfig::RightWidth);

    maxCount = 100;
    currentCount = 0;

    timer = new QTimer(this);
    timer->setInterval(1000);
    connect(timer, SIGNAL(timeout()), this, SLOT(on_btnDo_clicked()));

    //实例化数据库通信类
    dbConn = new DbConnThread(this);
    dbConn->setDbFlag("云端");
    connect(dbConn, SIGNAL(debug(QString)), this, SLOT(debug(QString)));
    connect(dbConn, SIGNAL(error(QString)), this, SLOT(error(QString)));
}

void frmDbUpload::initConfig()
{
    
    
    ui->cboxDbType->addItems(DbHelper::getDbType());
    ui->cboxDbType->setCurrentIndex(ui->cboxDbType->findText(AppConfig::DbType5));
    connect(ui->cboxDbType, SIGNAL(currentIndexChanged(int)), this, SLOT(saveConfig()));

    ui->txtDbName->setText(AppConfig::DbName5);
    connect(ui->txtDbName, SIGNAL(textChanged(QString)), this, SLOT(saveConfig()));

    ui->txtHostName->setText(AppConfig::HostName5);
    connect(ui->txtHostName, SIGNAL(textChanged(QString)), this, SLOT(saveConfig()));

    ui->txtHostPort->setText(QString::number(AppConfig::HostPort5));
    connect(ui->txtHostPort, SIGNAL(textChanged(QString)), this, SLOT(saveConfig()));

    ui->txtUserName->setText(AppConfig::UserName5);
    connect(ui->txtUserName, SIGNAL(textChanged(QString)), this, SLOT(saveConfig()));

    ui->txtUserPwd->setText(AppConfig::UserPwd5);
    connect(ui->txtUserPwd, SIGNAL(textChanged(QString)), this, SLOT(saveConfig()));
}

void frmDbUpload::saveConfig()
{
    
    
    AppConfig::DbType5 = ui->cboxDbType->currentText();
    AppConfig::DbName5 = ui->txtDbName->text();
    AppConfig::HostName5 = ui->txtHostName->text();
    AppConfig::HostPort5 = ui->txtHostPort->text().toInt();
    AppConfig::UserName5 = ui->txtUserName->text();
    AppConfig::UserPwd5 = ui->txtUserPwd->text();
    AppConfig::writeConfig();
}

void frmDbUpload::debug(const QString &msg)
{
    
    
    DbHelper::appendMsg(ui->txtMain, maxCount, currentCount, 0, msg);
}

void frmDbUpload::error(const QString &msg)
{
    
    
    DbHelper::appendMsg(ui->txtMain, maxCount, currentCount, 1, msg);
}

void frmDbUpload::on_btnOpen_clicked()
{
    
    
    if (ui->btnOpen->text() == "打开数据库") {
    
    
        DbInfo dbInfo;
        dbInfo.connName = this->objectName();
        dbInfo.dbName = AppConfig::DbName5;
        dbInfo.hostName = AppConfig::HostName5;
        dbInfo.hostPort = AppConfig::HostPort5;
        dbInfo.userName = AppConfig::UserName5;
        dbInfo.userPwd = AppConfig::UserPwd5;

        QString dbType = AppConfig::DbType5.toUpper();
        if (dbType == "SQLITE") {
    
    
            dbInfo.dbName = DbHelper::getDbDefaultFile();
        }

        dbConn->setConnInfo(DbHelper::getDbType(dbType), dbInfo);
        if (dbConn->openDb()) {
    
    
            dbConn->start();
            ui->btnOpen->setText("关闭数据库");
        } else {
    
    
            QString error = dbConn->getDatabase().lastError().text();
            QUIHelper::showMessageBoxError("打开数据库失败!\n" + error, 3);
        }
    } else {
    
    
        dbConn->stop();
        dbConn->closeDb();
        ui->btnOpen->setText("打开数据库");
        on_btnClear_clicked();
    }

    QTimer::singleShot(100, this, SLOT(on_btnInit_clicked()));
    QTimer::singleShot(1000, this, SLOT(on_btnStart_clicked()));
}

void frmDbUpload::on_btnCopy_clicked()
{
    
    
    //将数据库设置参数一键粘贴过来
    ui->cboxDbType->setCurrentIndex(ui->cboxDbType->findText(AppConfig::LocalDbType));
    ui->txtDbName->setText(AppConfig::LocalDbName);
    ui->txtHostName->setText(AppConfig::LocalHostName);
    ui->txtHostPort->setText(QString::number(AppConfig::LocalHostPort));
    ui->txtUserName->setText(AppConfig::LocalUserName);
    ui->txtUserPwd->setText(AppConfig::LocalUserPwd);
}

void frmDbUpload::on_btnInit_clicked()
{
    
    
    if (!dbConn->getOk()) {
    
    
        return;
    }

    //填充默认设备的初始数据
    QStringList sqls;
    //首先插入删除表语句
    sqls << "delete from NodeData";
    //再插入新建语句
    for (int i = 1; i < 10; i++) {
    
    
        QString sql1 = QString("insert into NodeData(PositionID,NodeValue,NodeStatus,SaveTime)");
        QString sql2 = QString("values('AT-1000%1','0',0,'%2')").arg(i).arg(DATETIME);
        sqls << (sql1 + " " + sql2);
    }

    dbConn->initDb(sqls);
}

void frmDbUpload::on_btnDo_clicked()
{
    
    
    if (!dbConn->getOk()) {
    
    
        return;
    }

    //在软件中相应的更新语句(增加/删除/修改)插入到队列
    //这里就举例更新值和时间
    for (int i = 1; i < 10; i++) {
    
    
        int value1 = rand() % 100;
        int value2 = rand() % 5;
        QString sql = QString("update NodeData set NodeValue='%1',NodeStatus='%2',SaveTime='%3' where PositionID='AT-1000%4'")
                      .arg(value1).arg(value2).arg(DATETIME).arg(i);
        dbConn->append(sql);
    }
}

void frmDbUpload::on_btnStart_clicked()
{
    
    
    if (ui->btnStart->text() == "启动服务") {
    
    
        on_btnDo_clicked();
        timer->start();
        ui->btnStart->setText("停止服务");
    } else {
    
    
        timer->stop();
        ui->btnStart->setText("启动服务");
    }
}

void frmDbUpload::on_btnClear_clicked()
{
    
    
    DbHelper::appendMsg(ui->txtMain, maxCount, currentCount, 0, "", true);
}