[C#] SqlBulkCopy batch adding Note that the DataTable must be in the same order as the list, otherwise an error will be reported, and the auto-increment column will be turned off

In the previous article, I was testing a batch adding operation, and found that the consistent addition was unsuccessful. The final analysis was that the order of the field columns was inconsistent.

1. Inconsistent column names

1.1. Error message

When debugging, the following error message was consistently prompted. After comparing the data and types, no problems were found. After analysis, the first column is a guid string, and the first field of the table is an integer with an auto-increment id, which means Can explain the following prompt message: The
given value of String type from the data source cannot be converted to the type int of the specified target column.

1.2. Solution

When using SqlBulkCopy to add DataTables in batches, it is very important to ensure that the columns of the DataTable are consistent with those of the database table.

The following are the requirements that must be met:

1) Match the column names: the column names in the DataTable must be consistent with the column names in the database table. Make sure the capitalization and spelling of the column names are correct.

2) Number of columns: The number of columns in the DataTable must be consistent with the number of columns in the database table.

3) Data type matching: The data type of the column in the DataTable must match the data type of the column in the database table. Make sure the data types are compatible and can be converted correctly.

If the columns of the DataTable do not match the columns of the database table, the SqlBulkCopy operation may fail, or cause data insertion errors.

If you want to change the name of the DataTable column, you can use ColumnMapping for mapping.

using (SqlConnection connection = new SqlConnection(connectionString))
{
    
    
    connection.Open();

    // 创建一个 DataTable 并填充数据
    DataTable dataTable = CreateDataTable();

    // 创建一个 SqlBulkCopy 对象
    using (SqlBulkCopy bulkCopy = new SqlBulkCopy(connection))
    {
    
    
        bulkCopy.DestinationTableName = "YourDestinationTableName";

        // 映射 DataTable 的列到数据库表的列
        bulkCopy.ColumnMappings.Add("SourceColumn1", "DestinationColumn1");
        bulkCopy.ColumnMappings.Add("SourceColumn2", "DestinationColumn2");

        // 执行批量插入
        bulkCopy.WriteToServer(dataTable);
    }
}

In the preceding code, map the names of the DataTable columns to the column names of the target table, making sure that the number and data types of the mapped columns match correctly.

To sum up, in order to successfully add DataTable in batches, ensure that the columns of the DataTable are consistent with those of the database table. ColumnMappings can also be used to map column names to operate when column names are inconsistent.

2. Close the auto-increment column

When adding DataTable data in batches, if you need to deal with the self-incrementing ID column, you can take the following two common methods:

2.1, does not contain columns

Disable the self-increment feature of the target table:
Disable the self-increment feature on the ID column of the target table, and then do not include the ID column of the source table in the mapping when SqlBulkCopy operates. In this way, the database engine will automatically assign a new auto-increment ID to each row.

using (SqlConnection connection = new SqlConnection(connectionString))
{
    
    
    connection.Open();

    // 创建一个 DataTable 并填充数据
    DataTable dataTable = CreateDataTable();

    // 创建一个 SqlBulkCopy 对象
    using (SqlBulkCopy bulkCopy = new SqlBulkCopy(connection))
    {
    
    
        bulkCopy.DestinationTableName = "YourDestinationTableName";

        // 不包括源表的 ID 列在映射中
        bulkCopy.ColumnMappings.Add("Column1", "Column1");
        bulkCopy.ColumnMappings.Add("Column2", "Column2");

        // 执行批量插入
        bulkCopy.WriteToServer(dataTable);
    }
}

In the above code, it is assumed that the ID column in the source table does not need to be mapped as the ID column of the target table in SqlBulkCopy. The database engine generates auto-incremented ID values ​​for the target table.

2.2. Features closed

Turn off the auto-increment feature in DataTable:
If you want to keep the auto-increment ID column of the source table and insert it into the target table, but avoid conflicting with the auto-increment ID of the target table, you can turn off the auto-increment feature in DataTable.

// 创建一个 DataTable 并填充数据
DataTable dataTable = CreateDataTable();

// 关闭 ID 列的自增特性
dataTable.Columns["ID"].AutoIncrement = false;
dataTable.Columns["ID"].ReadOnly = true;

// 接下来使用 SqlBulkCopy 进行批量插入操作

After turning off the auto-increment feature of the ID column, SqlBulkCopy will not process the ID column during batch insert, but will insert it into the target table according to the value in the source table.

You need to choose a suitable method to deal with the problem of auto-incrementing ID columns according to your specific needs. Note that when using SqlBulkCopy for bulk insert, make sure the database table structure and source table have the same data types and column names, and map the columns correctly to avoid any data insertion issues