One million data warehousing

? < PHP
 $ SQL = "INSERT INTO the Users (username, kinfname, fationname, usertioe) values" ;
    // php7.2 manual packaging method to obtain a random number (method in search RAND) 
    function generateRandStr ( $ length ) {
         $ randstr = " " ;
         for ( $ I = 0; $ I < $ length ; $ I ++ ) {
             $ randnum = the mt_rand (0,61 );
             IF ( $ randnum <10 ) {
                 $ randstr . = CHR ( $ randnum +48); 
            } The else  IF ( $ randnum <36 ) {
                 $ randstr . = CHR ( $ randnum +55 ); 
            } the else {
                 $ randstr =. CHR ( $ randnum +61 ); 
            } 
        } 
        return  $ randstr ; 
    } 
// set the timeout time 
the set_time_limit (3600 );
 the ini_set ( 'the memory_limit', '1024M' );
 // upper half sql statement 
$ sql="insert into users(username,kinfname,fationname,usertime) values ";
for ($i=0;$i<1000000;$i++){
    $rand=generateRandStr(6);
    $name=generateRandStr(10);
    $time=time();
    //循环后半截sql语句 拼接
    $sql.="('".$i."[email protected]','".$rand."','".$name."','".$time."'),";
, 0, -1$ SQL(substr=$ SQLremove the last comma//
}
);
 // connect to the database 
$ Link = "MySQL: Host = 127.0.0.1; dbname = 1703plus" ;
 $ PDO = new new the PDO ( $ Link , 'the root', 'the root' );
 // execute statement 
$ PDO -> Exec ( $ SQL );
 ?>

 

 

 

 

 

-------------------------------

 

Run the code Note that, except where the operation code is marked red also note the following

10W storage of data given as follows:

The reason is: max_allowed_packet setting mysql is too small to record write failed

The solution is:

1, view the current maximum number of M support

show VARIABLES like '%max_allowed_packet%';

He expressed support for the 1M, so need to be modified

 

2, Review: set global max_allowed_packet = 20 * 1024 * 1024 is changed here 20M

 

So 100W of data storage should be how to deal with it?

 

Just set set global max_allowed_packet = 100 * 1024 * 1024 to

After setting To turn off Navicat re-enter the execution to see results

 

 

----------------------------------------

 

Slow queries on state show VARIABLES LIKE 'slow_query_log%';

Open Set slow query SET GLOBAL slow_query_log = 'ON';

Set slow query more than 1 second recording SET GLOBAL long_query_time = 1

After setting To turn off Navicat re-enter the execution to see results

 

----------------------------------------

For the optimization of one million data table consider indexing, if the query is still a little long time, consider the storage engine into InnoDb