sparksql and mysql performance comparison

The following is the elapsed time for the same query statement at various data levels

select type,count(*) as count from test group by type order by count desc;

 

mysql 600W 3s

sparksql 550W 5s

 

mysql 1000W 5.4s

sparksql 1100W 6.3s

 

mysql 1900W 9.9s

sparksql 2000W 8.7s

It can be seen that when the amount of data is relatively large, the advantages of spark are reflected.

Note: The spark cluster server is configured with dual-core 4G memory. The cluster is configured with 4CPU and 4G memory

Additional sets of data

mysql 2700W 14.4s

mysql 4885W 25.2s

mysql 7900W 40.8s

mysql 1E 52.2s

 

spark 2000W 12.1s

spark 4185W 15.3s

spark 6278W 20.4s

spark 8370W 24.3s

spark 1E 28.1s