MySQL to handle duplicate data
There may be some duplicate records in MySQL data tables, in some cases we allow duplicate data, but sometimes we need to delete the duplicate data.
This chapter we will introduce how to prevent data table duplicate data and how to delete duplicate data in the data table.
Prevent duplicate data appears in the table
You can set the specified field is a MySQL data table PRIMARY KEY (primary key) or UNIQUE (unique) to ensure the uniqueness of the index data.
Let's try an example: under the table without indexes and primary keys, so the table allows multiple duplicate records appear.
CREATE TABLE person_tbl
(
first_name CHAR(20),
last_name CHAR(20),
sex CHAR(10)
);
If you want to set the table fields first_name, last_name data can not be repeated, you can set up a dual primary key mode to set the unique nature of the data, if you set up a dual primary key, then the default value of that key can not be NULL, can be set to NOT NULL. As follows:
CREATE TABLE person_tbl
(
first_name CHAR(20) NOT NULL,
last_name CHAR(20) NOT NULL,
sex CHAR(10),
PRIMARY KEY (last_name, first_name)
);
If we set up a unique index, then when inserting duplicate data, SQL statement will not be executed successfully, and throw wrong.
Difference INSERT IGNORE INTO and INSERT INTO is INSERT IGNORE ignore the data already in the database, if no database data, insert new data, if there is data, then skip this data. This will retain the data already in the database, the purpose of the data inserted in the gap.
The following example uses the INSERT IGNORE INTO, no error after execution, will not insert duplicate data to the data table:
mysql> INSERT IGNORE INTO person_tbl (last_name, first_name)
-> VALUES( 'Jay', 'Thomas');
Query OK, 1 row affected (0.00 sec)
mysql> INSERT IGNORE INTO person_tbl (last_name, first_name)
-> VALUES( 'Jay', 'Thomas');
Query OK, 0 rows affected (0.00 sec)
INSERT IGNORE INTO data when inserted, is provided in the uniqueness of the recording, if the insert duplicate data will not return an error, it returns only as a warning. The REPLACE INTO into the same record if the primary or unique present, the first deleted. Then insert a new record.
Another method of setting data is only to add a UNIQUE index, as follows:
CREATE TABLE person_tbl
(
first_name CHAR(20) NOT NULL,
last_name CHAR(20) NOT NULL,
sex CHAR(10)
UNIQUE (last_name, first_name)
);
Statistics duplicate data
The following tables we will repeat the number of records in the first_name and last_name:
mysql> SELECT COUNT(*) as repetitions, last_name, first_name
-> FROM person_tbl
-> GROUP BY last_name, first_name
-> HAVING repetitions > 1;
The above query will return person_tbl table number of duplicate records. Under normal circumstances, the query duplicate values, do the following:
Determine which column contains values may be repeated.
In the column selection list using the COUNT (*) those listed in the column.
Columns listed in the GROUP BY clause.
HAVING clause sets the number of repetitions is greater than 1.
Filter duplicate data
If you need to duplicate data can not be read using the DISTINCT keyword in the SELECT statement to filter duplicate data.
mysql> SELECT DISTINCT last_name, first_name
-> FROM person_tbl
-> ORDER BY last_name;
You can also use the GROUP BY to read the data in the table does not duplicate data:
mysql> SELECT last_name, first_name
-> FROM person_tbl
-> GROUP BY (last_name, first_name);
Delete duplicate data
If you want to dedupe data in the table, you can use the following SQL statement:
mysql> CREATE TABLE tmp SELECT last_name, first_name, sex
-> FROM person_tbl;
-> GROUP BY (last_name, first_name);
mysql> DROP TABLE person_tbl;
mysql> ALTER TABLE tmp RENAME TO person_tbl;
Of course you can also add INDEX (index) and PRIMAY KEY (primary key) in such a simple way to remove duplicate data table records in the table. Methods as below:
mysql> ALTER IGNORE TABLE person_tbl
-> ADD PRIMARY KEY (last_name, first_name);
This article reprinted from: w3cschool (edit: Leilin Peng Source: network intrusion deleted)