Table of contents
-
- 1. Introduction
- 2. Build the system environment
- 3. Unzip the test file
- 5. Initialize environment variables
- 6. Copy and modify the cfg file
- 7. Bring in the parameters according to the test needs and execute the test
- 8. Introduction to the test sub-project
- Nine, test process error handling
- 10. Other test software related links
1. Introduction
This article mainly uses SPEC2006 to test on the ARM platform. The Linux system used is Debian. The test process is described through specific operation examples. SPEC2006 is a new generation of SPEC's industry-standard CPU test benchmark suite. Focus on testing the system's processor, memory subsystem, and compiler. This benchmark suite includes the SPECint benchmark and the SPECfp benchmark. The SPECint2006 benchmark contains 12 different benchmarks and the SPECfp2006 benchmark contains 19 different benchmarks. SPEC designed this suite to provide a relatively standard calculation-intensive, high-performance cross-hardware CPU testing tool. The SPEC CPU 2006 benchmark has several different ways to measure computer performance. One way is a measure of how fast a computer can complete a single task, another way is a measure of throughput, capacity or rate.
Overview of the overall process:
1. Build the system environment;
2. Decompress the test file;
3. Execute the installation script;
4. Initialize the environment variables;
5. Copy and modify the cfg file
;
2. Build the system environment
Install the linux operating system and the corresponding language compiler such as: gcc gcc-g++ gcc-gfortran;
[root@localhost /]# apt-get install gcc gcc-g++ gcc-gfortran -y
3. Unzip the test file
Upload the speccpu2006 installation package to the operating system, unzip it and enter the test directory
[root@localhost /]# tar -xvf speccpu2006-v1.0.1.tar -C .
[root@localhost /speccpu2006-v1.0.1]# cd speccpu2006-v1.0.1
##4. Execute the installation script
[root@localhost /speccpu2006-v1.0.1]# ./install.sh //如遇没有权限执行脚本，请确保是否为root用户，同时给所以文件加入可执行权限
[root@localhost /speccpu2006-v1.0.1]# chmod -R a+x *
5. Initialize environment variables
[root@localhost /speccpu2006-v1.0.1]# source shrc
6. Copy and modify the cfg file
[root@localhost /speccpu2006-v1.0.1/config]# cd config/
[root@localhost /speccpu2006-v1.0.1/config]# cp arm64.cfg test.cfg //根据架构选择不通的cfg文件
//It is necessary to modify the cfg configuration, such as hardware configuration, compilation parameters and compilation parameters of each sub-item, (because I am in the test process, because of the compiler version, the compilation error is caused, so I need to modify the sub-item compilation parameters, later Chapter will talk), the specific picture is as follows:
7. Bring in the parameters according to the test needs and execute the test
7.1 Test scope of SPEC2006
1. Single project test
runspec -c test.cfg 429.mcf --rate 4 -n 1 --noreportable
2. Int integer test
runspec -c test.cfg int --rate 8 -n 1 -size=ref --noreportable --output_format all
3, fp floating point number
runspec -c test.cfg fp --rate 8 -n 1 -size=ref --noreportable --output_format all
4. Full project test
runspec -c test.cfg --rate 8 --noreportable --size=ref --tune=base -o text,screen,pdf -I --iterations=1 all
7.2 Detailed explanation of parameters
1) ref: test scale (the test scale has test, ref, and train, and the minimum running time of test is the shortest. If you want to test the correctness of the compiler, you can use the test scale, but you can use ref when you want to test the performance) 2) iterations=3
times : The number of tests (test once when the compiler performance is stable), can also be written as n 3;
if you want to get the result of weighted statistics, which is often referred to as the spec score of the cpu, you need to set iterations to be greater than or equal to 3.
3) All test range: Indicates fp and int tests (replace all with int or fp for fixed-point and floating-point tests; replace all with file numbers such as 456/444, indicating that a test item is tested separately)
for The SPEC test of the CPU, the default is all, but if you are not satisfied with the test score of a certain item, you can perform a separate test by specifying the number of this test item. The test will also get a score, which can be used when doing individual performance tuning.
4) Output format: -o text, screen, pdf means that the test is completed, and the format of the generated report is txt, screen display and pdf format respectively, and the saving directory is in the spec decompression directory result.
5) noreportable && reportable: Indicates to detect/not detect whether the generated binary file has been modified.
--reportable If the original generated binary file is modified, the runtime will automatically recompile the generated binary file to ensure that the running program is the original program.
6) Number of test cores: -r sets the number of CPU cores to be tested;
7) Test mode: -tune = base benchmark test; -tune = peak peak test; -tune defaults to base, you can choose base, peak, or all, Report first It is base, followed by peak.
8) -I: Indicates that during the test, if an error is reported, skip the error and continue the test;
8. Introduction to the test sub-project
| sub-item | language | illustrate |
|---|---|---|
| 400. perlbench PERL programming language | ANSI C | The payload consists of three scripts: the main payload is the spam detection software SpamAssassin, one is the email-to-HTML converter MHonArc, and the last one is specdiff |
| 401.bzip2 compression | ANSI C | The load includes six parts: two small JPEG images, a program, several source program files packaged in a tar, an HTML file, mixed files including compressed highly compressible files and not very compressible files, test Three different compression levels are used for compression and decompression |
| 403. gcc C compiler | C | Compiled 9 sets of C code |
| 429.mcf combinatorial optimization | ANSI C w/ libm | MCF is a program for single-station vehicle scheduling in large-scale public transportation. 429.mcf requires about 860/1700MB of memory when running on 32/64-bit models |
| 445.gobmk Artificial Intelligence: Go | C | Go |
| 456.hmmer gene sequence search | C | Gene sequence search using HMMS (Hidden Markov Models, Hidden Markov Model) gene identification method |
| 458.sjeng Artificial Intelligence: Chess | ANSI C | International chess |
| 462.libquantum Physics: Quantum Computing | ISO/IEC 9899:1999(“C99”) | libquantum is a library file for simulating quantum computers, used for research on quantum computer applications |
| 464.h264ref video compression | C | H.264 encode two YUV format source files using two configurations |
| 471.omnetpp discrete event simulation | C++ | Including about 8000 computers and 900 switches/hubs, and a large CSMA/CD protocol Ethernet simulation with a mix of various rates from 10Mb to 1000Mb |
| 473.astar Pathfinding Algorithm | C++ | Implemented three different versions of the 2D pathfinding algorithm A* |
| 483. xalancbmk XML processing | C++ | Conversion of XML document/XSL table to HTML document |
| 410.bwaves fluid mechanics | Fortran 77 | Simulation Calculation of Shock Wave in Three-dimensional Transonic Viscous Flow |
| 416.gamess quantum chemistry | Fortran | Calculation of three SCF self-consistent fields: cytosine molecules, water and Cu2+ ions, triazolium ions |
| 433.milc quantum mechanics | C | Simulation of 4D SU(3) lattice gauge theory for the study of QCD quantum chromodynamics, quarks and gluons |
| 434.zeusmp Physics: Computational Fluid Dynamics | Fortran 77/REAL*8 | Used to calculate fluid dynamics and magnetohydrodynamics under ideal, non-relativistic conditions, 434.zeusmp simulates and calculates 3D shock waves in a unified magnetic field |
| 435.gromacs biochemistry/molecular mechanics | C & Fortran | GROMACS is a molecular mechanics calculation suite, but can also be used for non-biological systems, 435.gromacs simulates the structure of protein lysozyme in an aqueous and ionic solution under various experimental means such as nuclear magnetic resonance X-ray irradiation |
| 436.cactus ADM Physics: General Relativity | Fortran 90, ANSI C | 436.cactusADM solves the Einstein evolution equation whose space-time curvature is determined by the internal matter. The Einstein evolution equation consists of 10 second-order nonlinear partial differential equations decomposed by the standard ADM 3+1. |
| 437.leslie 3d fluid mechanics | Fortran 90 | LESlie3d is a computational fluid dynamics program used to calculate turbulence. 437.leslie3d calculates a time-stratified mixed fluid such as fuel injected into a combustion chamber. |
| 444.namd biology/molecules | C++ | NAMD is a parallel computing program for large biomolecular systems, 444.namd simulates AI apoprotein composed of 92224 atoms |
| 447.dealII Finite Element Analysis | C++ w/Boost lib | deal.II is a C++ library for adaptive finite element and error estimation. 447.dealII solves the Helmholtz equation with very coefficients. It uses an adaptive method based on binary weighted error estimation to generate the best grid. , the equation is solved in 3 dimensions |
| 450.soplex linear programming, optimization | ANSI C++ | SoPlex solves linear equations using the simplex algorithm |
| 453. povray image ray tracing | ISO C++ | POV-Ray is a ray tracing rendering software, 453.povray renders a 1280x1024 anti-aliased chess board image |
| 454. Calculix Structural Mechanics | Fortran 90 & C w/SPOOLES code | CalculiX is a finite element analysis software for linear and nonlinear three-dimensional structural mechanics, 454.calculix calculates the stress and deformation of a high-speed rotating compression disc under the action of centrifugal force |
| 459.Gems FDTD Computational Electromagnetics | Fortran 90 | 459.GemsFDTD uses the FDTD (Finite Difference Time Domain) method to solve Maxwell's equations in the three-dimensional time domain, and calculates the radar cross section of an ideal conductor |
| 465.tonto quantum chemistry | Fortran 95 | Tonto is an object-oriented quantum chemistry program package, 465.tonto calculation is oriented to quantum crystallography, it is based on a constrained molecular Hartree-Fock wave function that conforms to X-ray diffraction experimental data |
| 470.lbm Hydrodynamics | ANSI C | 470.lbm uses LBM (lattice Boltzmann method) to simulate incompressible fluid, which simulates two situations: shear-driven fluid and pipeline fluid similar to piston push, and the test contains 3000 steps |
| 481.wrf weather forecast | Fortran 90 & C | 481.wrf calculates the data of NCAR based on the WRF (Weather Research and Forecastin) model, and the data includes data from UTC 2001.06.11 to UTC 2001.06.12 at three-hour intervals |
| 482. sphinx3 speech recognition | C | Speech Recognition |
Nine, test process error handling
(1) Error building 416.gamess
solution: set the compilation parameters of 416.gamess in Portability Flags in the configuration file cfg
416.gamess=default=default=default:
FPORTABILITY = -funconstrained-commons -std=legacy
（2）Error building 447.dealII
解决方法：在配置文件cfg中的Portability Flags中设置447.dealII的编译参数
447.dealII=default=default=default:
CXXPORTABILITY = -fpermissive
（3） Error building 450.soplex
解决方法：在配置文件cfg中的Portability Flags中设置450.soplex的编译参数
450.soplex=default=default=default:
CXXPORTABILITY = -std=c++03
（4）Error building 481.wrf
解决方法：在配置文件cfg中的Portability Flags中设置481.wrf的编译参数
481.wrf=default=default=default:
CPORTABILITY = -DSPEC_CPU_CASE_FLAG -DSPEC_CPU_LINUX
FPORTABILITY = -std=legacy
（5）编译时报错
specmake build 2> make.err | tee make.out
/usr/bin/gcc -c -o av.o -DSPEC_CPU -DNDEBUG -DPERL_CORE -O2 -fno-strict-aliasing -DSPEC_CPU_LP64 -DSPEC_CPU_LINUX_X64 av.c
In file included from /usr/include/stdio.h:936:0,
from perlio.h:65,
from iperlsys.h:51,
from perl.h:2585,
from av.c:22:
/usr/include/x86_64-linux-gnu/bits/stdio2.h: In function ‘sprintf’:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:33:10: error: invalid use of ‘__builtin_va_arg_pack ()’
return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1,
^
/usr/include/x86_64-linux-gnu/bits/stdio2.h: In function ‘snprintf’:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:64:10: error: invalid use of ‘__builtin_va_arg_pack ()’
return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
^
/usr/include/x86_64-linux-gnu/bits/stdio2.h: In function ‘fprintf’:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:97:10: error: invalid use of ‘__builtin_va_arg_pack ()’
return __fprintf_chk (__stream, __USE_FORTIFY_LEVEL - 1, __fmt,
^
/usr/include/x86_64-linux-gnu/bits/stdio2.h: In function ‘printf’:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:104:10: error: invalid use of ‘__builtin_va_arg_pack ()’
return __printf_chk (__USE_FORTIFY_LEVEL - 1, __fmt, __va_arg_pack ());
^
/usr/include/x86_64-linux-gnu/bits/stdio2.h: In function ‘dprintf’:
原因是spec与gcc 版本不对应 ，gcc 相对于spec2006 来说gcc 版本过高，所以gcc降级
（6）运行上报错：runspec --config mytest.cfg --action=onlyrun --tune=all --reportable int //通过–action 指定 onlyrun 不再编译
*** Error during benchmark setup for 429.mcf
Setting up 445.gobmk ref base gcc43-64bit default: ERROR: Cannot create run directory for 445.gobmk: mkdir /home/nfs/zhaoan/spec/cpu2006/benchspec/CPU2006/445.gobmk/run/run_base_ref_gcc43-64bit.0005: Permission denied at /home/nfs/zhaoan/spec/cpu2006/bin/benchmark.pm line 2073
*** Error during benchmark setup for 445.gobmk
Setting up 456.hmmer ref base gcc43-64bit default: ERROR: Cannot create run directory for 456.hmmer: mkdir /home/nfs/zhaoan/spec/cpu2006/benchspec/CPU2006/456.hmmer/run/run_base_ref_gcc43-64bit.0005: Permission denied at /home/nfs/zhaoan/spec/cpu2006/bin/benchmark.pm line 2073
*** Error during benchmark setup for 456.hmmer
Setting up 458.sjeng ref base gcc43-64bit default: ERROR: Cannot create run directory for 458.sjeng: mkdir /home/nfs/zhaoan/spec/cpu2006/benchspec/CPU2006/458.sjeng/run/run_base_ref_gcc43-64bit.0005: Permission denied at /home/nfs/zhaoan/spec/cpu2006/bin/benchmark.pm line 2073
原因：由于在docker容器中是使用root权限编译的，在138上面再执行时用户是：wxsc ,所以权限有问题
修改：sudo chown -R wxsc:wxsc ./*
（7）运行时报错
Running (#2) 445.gobmk ref base gcc43-64bit default
/home/nfs/zhaoan/spec/cpu2006/bin/specinvoke -d /home/nfs/zhaoan/spec/cpu2006/benchspec/CPU2006/445.gobmk/run/run_base_ref_gcc43-64bit.0006 -e speccmds.err -o speccmds.stdout -f speccmds.cmd -C -q
Socket error Event: 32 Error: 10053.
Connection closing…Socket close.
Connection closed by foreign host.
Disconnected from remote host(192.167.253.138) at 08:50:57.
解决方法：改为后台执行即可
（8）运行时报错
1548641594.38: Contents of nph3.err
1548641594.38: ****************************************
1548641594.38: …/run_base_ref_gcc43-64bit.0009/hmmer_base.gcc43-64bit: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.27’ not found (required by …/run_base_ref_gcc43-64bit.0009/hmmer_base.gcc43-64bit)
解决方法：编译时改为静态链接
使用的config 是 mytest.cfg ，则修改mytest.cfg文件将
COPTIMIZE = -O2 -fno-strict-aliasing
CXXOPTIMIZE = -O2 -fno-strict-aliasing
FOPTIMIZE = -O2 -fno-strict-aliasing
改为：
COPTIMIZE = -O2 -fno-strict-aliasing -static
CXXOPTIMIZE = -O2 -fno-strict-aliasing -static
FOPTIMIZE = -O2 -fno-strict-aliasing -static
（9）float编译时报错
1548726106.29: ****************************************
1548726106.29: Contents of exam29.err
1548726106.29: ****************************************
1548726106.29: STOP IN ABRT
1548726106.29: ****************************************
1548726106.29: *** Miscompare of exam29.out; for details see
1548726106.29: /cpu2006/benchspec/CPU2006/416.gamess/run/run_base_test_gcc43-64bit.0000/exam29.out.mis
1548726106.29: 0423: 1 0 0 -228.438026762 -228.438026762 0.585016182 0.000000000
1548726106.29: 1 0 0 -228.438026762 -228.438026762 2.057216987 0.000000000
1548726106.29: ^
1548726106.29: 0424: ---------------START SECOND ORDER SCF---------------
1548726106.29: 2 1 0 -13.398272306 215.039754456 0.019611730 2.138165837
1548726106.29: ^
1548726106.29: 0425: 2 1 0 -228.975183270 -0.537156507 0.141897260 0.111481465
1548726106.29: ---------------START SECOND ORDER SCF---------------
1548726106.29: ^
1548726106.29: 0426: 3 2 0 -229.014081256 -0.038897986 0.064028396 0.079634004
1548726106.29: 3 2 0 -13.404086244 -0.005813938 0.000644098 0.008801378
1548726106.29: ^
1548726106.29: 0427: 4 3 0 -229.036210679 -0.022129423 0.025840433 0.007697238
1548726106.29: 4 3 0 -13.404093666 -0.000007422 0.000028154 0.000448854
1548726106.30: ^
1548726106.30: 0428: 5 4 0 -229.036777024 -0.000566345 0.008639408 0.001874192
1548726106.30: 5 4 0 -13.404093683 -0.000000017 0.000000370 0.000004470
1548726106.30: ^
1548726106.30: 0429: 6 5 0 -229.036824895 -0.000047871 0.001734717 0.000979860
1548726106.30: 6 5 0 -13.404093683 -0.000000000 0.000000008 0.000000051
1548726106.30: ^
1548726106.30: 0431: 7 6 0 -229.036831711 -0.000006816 0.000315932 0.000297576
1548726106.30: -----------------
1548726106.30: ^
1548726106.30: 0432: 8 7 0 -229.036832335 -0.000000624 0.000112760 0.000083271
1548726106.30: DENSITY CONVERGED
1548726106.30: ^
1548726106.30: 0433: 9 8 0 -229.036832426 -0.000000092 0.000074046 0.000042607
1548726106.30: -----------------
1548726106.30: ^
1548726106.30: Benchmark Times:
1548726106.30: Start: Tue Jan 29 09:41:45 2019 (1548726105)
1548726106.30: Stop: Tue Jan 29 09:41:45 2019 (1548726105)
1548726106.30: Elapsed: 00:00:00 (0)
1548726106.30: Reported: 0 39727000 0.039727
1548726106.30: Error 416.gamess base test ratio=-0.00, runtime=0.039727, power=0.00w, temp=0.00 deg, humidity=0.00%
1548726106.30: Invalid run; unable to continue.
解决方法：
这个问题通过修改相应的config 中编译优化选项 将
COPTIMIZE = -O2 -fno-strict-aliasing -static
CXXOPTIMIZE = -O2 -fno-strict-aliasing -static
FOPTIMIZE = -O2 -fno-strict-aliasing -static
改为：
COPTIMIZE = -O0 -fno-strict-aliasing -static
CXXOPTIMIZE = -O0 -fno-strict-aliasing -static
FOPTIMIZE = -O0 -fno-strict-aliasing -static
注意：416测试时报错，# Error 416.gamess: Output miscompare，可以在416项中COPTIMIZE、CXXOPTIMIZE、FOPTIMIZE追加-ffpe-summary=none解决
View the commands executed by specific subitems:
cd /speccpu2006-v1.0.1/benchspec/CPU2006/416.gamess/run/build_base_x86_64_linux.0000
specinvoke -n -f speccmds.cmd
Note to clear the cache before testing:
echo 3 > /proc/sys /vm/drop_caches #Clean up the cache
If the test project fails to compile, you can test it by modifying the test source code or changing the version of gcc, c++ and other compilers
10. Other test software related links
1. FIO test hard disk performance parameters and example summary