再战mysql 数据去重

年初时，写过一篇去重的，在小表中还能用用，在大表中真的是效率低下，现在给了一次优化
https://www.cnblogs.com/jarjune/p/8328013.html

继上一篇文章

方法三：

DELIMITER //

DROP PROCEDURE IF EXISTS delete_rows_2;

CREATE PROCEDURE delete_rows_2(IN TABLENAME VARCHAR(50), IN FIELDNAMES VARCHAR(100), IN AUTOFIELD VARCHAR(50))
BEGIN

DECLARE DELETE_TABLE_ROWS_SQL VARCHAR(1000);

SET DELETE_TABLE_ROWS_SQL = CONCAT('
        DELETE 
        FROM 
            ', TABLENAME ,' 
        WHERE 
            (', FIELDNAMES ,') IN ( 
                SELECT ', FIELDNAMES ,' 
                FROM (
                    SELECT 
                        ', FIELDNAMES ,' 
                    FROM 
                        ', TABLENAME ,' 
                    GROUP BY 
                        ', FIELDNAMES ,' 
                    HAVING 
                        COUNT(1) > 1 
                ) t1
            ) 
        AND ', AUTOFIELD ,' NOT IN ( 
            SELECT ', AUTOFIELD ,' 
            FROM (
                SELECT 
                    MAX(', AUTOFIELD ,') ', AUTOFIELD ,' 
                FROM 
                    ', TABLENAME ,'
                GROUP BY 
                    ', FIELDNAMES ,' 
                HAVING 
                    COUNT(1) > 1 
                ) t2
            )
');

SET @DELETE_TABLE_ROWS_SQL = DELETE_TABLE_ROWS_SQL;

PREPARE DELETE_TABLE_ROWS_SQL_PRE FROM @DELETE_TABLE_ROWS_SQL;
EXECUTE DELETE_TABLE_ROWS_SQL_PRE;

END//

DELIMITER ;

CALL delete_rows_1('表名', '字段1,字段2,字段3...', '主键(唯一)字段');

之后发现删除的效率还是挺低，又优化成

方法三(优化)：

DELIMITER //

DROP PROCEDURE IF EXISTS delete_rows_2;

CREATE PROCEDURE delete_rows_2(IN TABLENAME VARCHAR(50), IN FIELDNAMES VARCHAR(100), IN AUTOFIELD VARCHAR(50))
BEGIN

DECLARE DELETE_TABLE_ROWS_SQL VARCHAR(1000);

SET DELETE_TABLE_ROWS_SQL = CONCAT('
        DELETE 
        FROM 
            ', TABLENAME ,' 
        WHERE 
            ', AUTOFIELD ,' IN ( 
                SELECT 
                    ', AUTOFIELD ,' 
                FROM
                    (
                    SELECT 
                        ', AUTOFIELD ,' 
                    FROM 
                        ', TABLENAME ,' 
                    WHERE 
                        (', FIELDNAMES ,') IN ( 
                            SELECT 
                                ', FIELDNAMES ,' 
                            FROM 
                                ', TABLENAME ,' 
                            GROUP BY 
                                ', FIELDNAMES ,' 
                            HAVING 
                                COUNT(1) > 1 
                        ) 
                    AND ', AUTOFIELD ,' NOT IN ( 
                        SELECT 
                            MAX(', AUTOFIELD ,') 
                        FROM 
                            ', TABLENAME ,'
                        GROUP BY 
                            ', FIELDNAMES ,' 
                        HAVING 
                            COUNT(1) > 1 
                    ) 
                ) t2 
            ) 
    ');

SET @DELETE_TABLE_ROWS_SQL = DELETE_TABLE_ROWS_SQL;

PREPARE DELETE_TABLE_ROWS_SQL_PRE FROM @DELETE_TABLE_ROWS_SQL;
EXECUTE DELETE_TABLE_ROWS_SQL_PRE;

END//

DELIMITER ;

CALL delete_rows_2('表名', '字段1,字段2,字段3...', '主键字段');

综上，方法三(优化)是目前在用的去重。