BenchmarkSQL简介+简用

版权声明:本文为博主原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。
本文链接: https://blog.csdn.net/ziptop/article/details/52183280

1、要求

内嵌TPCC测试脚本,支持PG、oracle、mysql等数据库的压力测试,使用前需要安装jdk,原来支持linux和windows上测试,自2.3.5版本以来不再支持windows上的压力测试,所以若想在Windows上测试,需要用2.3.5版本。

2、描述

每个仓库负责十个区域的供货,每个区域 3000 个客户服务,每个仓库维护 100000 种商品的库存纪录,每个仓库至多有10个终端

BenchmarkSQL数据量:
每个仓库数据量大约为76823.04KB即约为76M
所以 Warehouse=14时,数据量大约为1G
Warehouse=140时,数据量大约为10G

想要加载500G左右的数据,warehouse大约为5000即可,除了表数据还有索引
记一次加载时间:
128G内存、机械硬盘、7000warehouse
19:20开始加载数据,大约在第三天17:00左右加载完数据(近46h),然后开始建索引,直到第四天

测试模拟5中事务处理,新订单、支付操作、订单状态查询、发货、库存状态查询;TPC-C基准测试针对一种模拟订单录入与销售环境测量每分钟商业事务(tpmC)吞吐量。

3、简用

(1)修改prop.pg

driver=org.postgresql.Driver
conn=jdbc:postgresql://localhost:5866/benchdb
user=bench
password=bench

warehouses=10
terminals=1
//To run specified transactions per terminal- runMins must equal zero
runTxnsPerTerminal=0
//To run for specified minutes- runTxnsPerTerminal must equal zero
runMins=5
//Number of total transactions per minute
limitTxnsPerMin=300

(2)创建测试表

[highgo@localhost run]$ ./runSQL.sh props.pg sqlTableCreates          ------步骤一
DROP SCHEMA IF EXISTS benchmarksql CASCADE;
CREATE SCHEMA benchmarksql;
create table benchmarksql.warehouse (
w_id        integer   not null,
w_ytd       decimal(12,2),
w_tax       decimal(4,4),
w_name      varchar(10),
w_street_1  varchar(20),
w_street_2  varchar(20),
w_city      varchar(20),
w_state     char(2),
w_zip       char(9)
);
create table benchmarksql.district (
d_w_id       integer       not null,
d_id         integer       not null,
d_ytd        decimal(12,2),
d_tax        decimal(4,4),
d_next_o_id  integer,
d_name       varchar(10),
d_street_1   varchar(20),
d_street_2   varchar(20),
d_city       varchar(20),
d_state      char(2),
d_zip        char(9)
);
create table benchmarksql.customer (
c_w_id         integer        not null,
c_d_id         integer        not null,
c_id           integer        not null,
c_discount     decimal(4,4),
c_credit       char(2),
c_last         varchar(16),
c_first        varchar(16),
c_credit_lim   decimal(12,2),
c_balance      decimal(12,2),
c_ytd_payment  float,
c_payment_cnt  integer,
c_delivery_cnt integer,
c_street_1     varchar(20),
c_street_2     varchar(20),
c_city         varchar(20),
c_state        char(2),
c_zip          char(9),
c_phone        char(16),
c_since        timestamp,
c_middle       char(2),
c_data         varchar(500)
);
create sequence hist_id_seq;
create table benchmarksql.history (
hist_id  integer not null default nextval('hist_id_seq') primary key,
h_c_id   integer,
h_c_d_id integer,
h_c_w_id integer,
h_d_id   integer,
h_w_id   integer,
h_date   timestamp,
h_amount decimal(6,2),
h_data   varchar(24)
);
create table benchmarksql.oorder (
o_w_id       integer      not null,
o_d_id       integer      not null,
o_id         integer      not null,
o_c_id       integer,
o_carrier_id integer,
o_ol_cnt     decimal(2,0),
o_all_local  decimal(1,0),
o_entry_d    timestamp
);
create table benchmarksql.new_order (
no_w_id  integer   not null,
no_d_id  integer   not null,
no_o_id  integer   not null
);
create table benchmarksql.order_line (
ol_w_id         integer   not null,
ol_d_id         integer   not null,
ol_o_id         integer   not null,
ol_number       integer   not null,
ol_i_id         integer   not null,
ol_delivery_d   timestamp,
ol_amount       decimal(6,2),
ol_supply_w_id  integer,
ol_quantity     decimal(2,0),
ol_dist_info    char(24)
);
create table benchmarksql.stock (
s_w_id       integer       not null,
s_i_id       integer       not null,
s_quantity   decimal(4,0),
s_ytd        decimal(8,2),
s_order_cnt  integer,
s_remote_cnt integer,
s_data       varchar(50),
s_dist_01    char(24),
s_dist_02    char(24),
s_dist_03    char(24),
s_dist_04    char(24),
s_dist_05    char(24),
s_dist_06    char(24),
s_dist_07    char(24),
s_dist_08    char(24),
s_dist_09    char(24),
s_dist_10    char(24)
);
create table benchmarksql.item (
i_id     integer      not null,
i_name   varchar(24),
i_price  decimal(5,2),
i_data   varchar(50),
i_im_id  integer
);

(3)导入数据

①直接导入

[highgo@localhost run]$ ./runLoader.sh props.pg numWarehouses 10          ------步骤二
Starting BenchmarkSQL LoadData
----------------- Initialization -------------------
numWarehouses
10
driver=org.postgresql.Driver
conn=jdbc:postgresql://localhost:5866/benchdb
user=bench
password=******

------------- LoadData StartTime = Wed Aug 10 18:58:49 PDT 2016-------------

Start Whse Load for 10 Whses @ Wed Aug 10 18:58:49 PDT 2016 ...
Elasped Time(ms): 0.029
End Whse Load @  Wed Aug 10 18:58:49 PDT 2016

Start Item Load for 100000 Items @ Wed Aug 10 18:58:49 PDT 2016 ...
  Elasped Time(ms): 0.379       Writing record 10000 of 100000
  Elasped Time(ms): 0.856       Writing record 20000 of 100000
  Elasped Time(ms): 0.729       Writing record 30000 of 100000
  Elasped Time(ms): 0.52        Writing record 40000 of 100000
  Elasped Time(ms): 0.576       Writing record 50000 of 100000
  Elasped Time(ms): 0.281       Writing record 60000 of 100000
  Elasped Time(ms): 0.36        Writing record 70000 of 100000
  Elasped Time(ms): 0.269       Writing record 80000 of 100000
  Elasped Time(ms): 0.251       Writing record 90000 of 100000
  ……
  ……

②可以将数据导出到csv文件中,以后可从这些文件将数据更快的多次导入数据库

[highgo@localhost run]$ ./runLoader.sh props.pg numWarehouses 10  fileLocation /home/highgo/Desktop/BMcsv/  (注意,目录最后BMcsv后/一定存在,不然会在Desktop目录下以BMcsv为文件名生成文件)
Starting BenchmarkSQL LoadData
----------------- Initialization -------------------
numWarehouses
10
/home/highgo/Desktop/BMcsv
driver=org.postgresql.Driver
conn=jdbc:postgresql://localhost:5866/benchdb
user=bench
password=******

------------- LoadData StartTime = Wed Aug 10 19:16:22 PDT 2016-------------

Start Whse Load for 10 Whses @ Wed Aug 10 19:16:22 PDT 2016 ...
Elasped Time(ms): 0.179
End Whse Load @  Wed Aug 10 19:16:22 PDT 2016

Start Item Load for 100000 Items @ Wed Aug 10 19:16:22 PDT 2016 ...
  Elasped Time(ms): 0.2         Writing record 10000 of 100000
  ……
  ……

10仓库的数据量,导出到文件大小
[highgo@localhost BMcsv]$ ll -h
total 696M
-rw-rw-r–. 1 highgo highgo 19M Aug 10 22:55 cust-hist.csv
-rw-rw-r–. 1 highgo highgo 162M Aug 10 22:55 customer.csv
-rw-rw-r–. 1 highgo highgo 8.8K Aug 10 22:54 district.csv
-rw-rw-r–. 1 highgo highgo 7.0M Aug 10 22:54 item.csv
-rw-rw-r–. 1 highgo highgo 809K Aug 10 22:55 new-order.csv
-rw-rw-r–. 1 highgo highgo 13M Aug 10 22:55 order.csv
-rw-rw-r–. 1 highgo highgo 212M Aug 10 22:55 order-line.csv
-rw-rw-r–. 1 highgo highgo 283M Aug 10 22:54 stock.csv
-rw-rw-r–. 1 highgo highgo 829 Aug 10 22:54 warehouse.csv

将数据加载到数据库,首先修改sqlTableCopies中文件位置为导出目录,如:

copy benchmarksql.warehouse
  (w_id, w_ytd, w_tax, w_name, w_street_1, w_street_2, w_city, w_state, w_zip)  
  from '/home/highgo/Desktop/BMcsv/warehouse.csv' WITH CSV;
[highgo@localhost run]$ ./runSQL.sh props.pg sqlTableCopies
copy benchmarksql.warehouse
(w_id, w_ytd, w_tax, w_name, w_street_1, w_street_2, w_city, w_state, w_zip)
from '/home/highgo/Desktop/BMcsv/warehouse.csv' WITH CSV;
copy benchmarksql.item
(i_id, i_name, i_price, i_data, i_im_id)
from '/home/highgo/Desktop/BMcsv/item.csv' WITH CSV;
copy benchmarksql.stock
……
……

(4)创建索引

[highgo@localhost run]$ ./runSQL.sh props.pg sqlIndexCreates          ------步骤三
alter table benchmarksql.warehouse add constraint pk_warehouse
primary key (w_id);
alter table benchmarksql.district add constraint pk_district
primary key (d_w_id, d_id);
alter table benchmarksql.customer add constraint pk_customer
primary key (c_w_id, c_d_id, c_id);
……
……

(5)执行压测(仓库数为10,数据量大约为696M)

[highgo@localhost run]$ ./runBenchmark.sh props.pg           ------步骤四
2016-08-10 23:16:43,365  INFO - Term-00, 
2016-08-10 23:16:43,383  INFO - Term-00, +-------------------------------------------------------------+
2016-08-10 23:16:43,383  INFO - Term-00,      BenchmarkSQL v4.1
2016-08-10 23:16:43,383  INFO - Term-00, +-------------------------------------------------------------+
2016-08-10 23:16:43,383  INFO - Term-00,  (c) 2003, Raul Barbosa
2016-08-10 23:16:43,383  INFO - Term-00,  (c) 2004-2014, Denis Lussier
2016-08-10 23:16:43,384  INFO - Term-00, +-------------------------------------------------------------+
2016-08-10 23:16:43,384  INFO - Term-00, 
2016-08-10 23:16:43,384  INFO - Term-00, driver=org.postgresql.Driver
2016-08-10 23:16:43,384  INFO - Term-00, conn=jdbc:postgresql://localhost:5866/benchdb
2016-08-10 23:16:43,384  INFO - Term-00, user=bench
2016-08-10 23:16:43,385  INFO - Term-00, 
2016-08-10 23:16:43,385  INFO - Term-00, warehouses=10
2016-08-10 23:16:43,385  INFO - Term-00, terminals=1
2016-08-10 23:16:43,385  INFO - Term-00, runMins=5
2016-08-10 23:16:43,386  INFO - Term-00, limitTxnsPerMin=300
2016-08-10 23:16:43,386  INFO - Term-00, 
2016-08-10 23:16:43,386  INFO - Term-00, newOrderWeight=45
2016-08-10 23:16:43,386  INFO - Term-00, paymentWeight=43
2016-08-10 23:16:43,387  INFO - Term-00, orderStatusWeight=4
2016-08-10 23:16:43,387  INFO - Term-00, deliveryWeight=4
2016-08-10 23:16:43,387  INFO - Term-00, stockLevelWeight=4
2016-08-10 23:16:43,387  INFO - Term-00, 
Term-00, Running Average tpmTOTAL: 339.46    Current tpmTOTAL: 12    Memory Usag
……
……
Term-00, Running Average tpmTOTAL: 281.66    Current tpmTOTAL: 7740    Memory UsTerm-00, Running Average tpmTOTAL: 281.43    Current tpmTOTAL: 7740    Memory UsTerm-00, Running Average tpmTOTAL: 281.67    Current tpmTOTAL: 7740    Memory UsTerm-00, Running Average tpmTOTAL: 281.67    Current tpmTOTAL: 7740    Memory UsTerm-00, Running Average tpmTOTAL: 281.71    Current tpmTOTAL: 7740    Memory Usage2016-08-10 23:21:44,425  INFO - Term-00,     Current tpmTOTAL: 9672    Memory Usage: 3MB / 15MB
2016-08-10 23:21:44,426  INFO - Term-00, 
2016-08-10 23:21:44,426  INFO - Term-00, Measured tpmC (NewOrders) = 124.11
2016-08-10 23:21:44,426  INFO - Term-00, Measured tpmTOTAL = 284.0
2016-08-10 23:21:44,426  INFO - Term-00, Session Start     = 2016-08-10 23:16:44
2016-08-10 23:21:44,426  INFO - Term-00, Session End       = 2016-08-10 23:21:44
2016-08-10 23:21:44,426  INFO - Term-00, Transaction Count = 1420

猜你喜欢

转载自blog.csdn.net/ziptop/article/details/52183280
今日推荐