com.amazonaws.AmazonClientException: Unable to verify integrity of data download. Client calculated content hash didn't match hash calculated by Amazon S3. The data may be corrupt.
at com.amazonaws.services.s3.internal.DigestValidationInputStream.validateMD5Digest(DigestValidationInputStream.java:79)
at com.amazonaws.services.s3.internal.DigestValidationInputStream.read(DigestValidationInputStream.java:61)
at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:72)
at com.amazonaws.services.s3.model.S3ObjectInputStream.read(S3ObjectInputStream.java:155)
at com.amazonaws.services.s3.model.S3ObjectInputStream.read(S3ObjectInputStream.java:147)
at com.intel.cosbench.driver.operator.Reader.copyLarge(Reader.java:120)
at com.intel.cosbench.driver.operator.Reader.doRead(Reader.java:92)
at com.intel.cosbench.driver.operator.Reader.operate(Reader.java:69)
at com.intel.cosbench.driver.operator.AbstractOperator.operate(AbstractOperator.java:76)
at com.intel.cosbench.driver.agent.WorkAgent.performOperation(WorkAgent.java:197)
at com.intel.cosbench.driver.agent.WorkAgent.doWork(WorkAgent.java:177)
at com.intel.cosbench.driver.agent.WorkAgent.execute(WorkAgent.java:134)
at com.intel.cosbench.driver.agent.AbstractAgent.call(AbstractAgent.java:44)
at com.intel.cosbench.driver.agent.AbstractAgent.call(AbstractAgent.java:1)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
================================================== stage: s5-cleanup ==================================================
---------------------------------- mission: null, driver: driver1 ----------------------------------
[N/A]================================================== stage: s6-dispose ==================================================
Solution:
Need to modify
- stop-all.sh stop cosbench process (controller, drivers)
- Edit the cosbench-start.sh file, find the java startup command line parameters, and add "-Dcom.amazonaws.services.s3.disableGetObjectMD5Validation=true" to disable the MD5 verification function of S3
echo "Launching osgi framwork ... "
/usr/bin/nohup java -Dcom.amazonaws.services.s3.disableGetObjectMD5Validation=true -Dcosbench.t
omcat.config=$TOMCAT_CONFIG -server -cp main/* org.eclipse.equinox.launcher.Main -configuration
$OSGI_CONFIG -console $OSGI_CONSOLE_PORT 1> $BOOT_LOG 2>&1 &- start-all.sh starts all cosbench processes.
[root@ceph01 0.4.2.c4]# ./stop-all.sh
Stopping cosbench controller ...
Successfully stopped cosbench controller.
======================================================
127.0.0.1:18088
18088
Stopping cosbench driver ...
Successfully stopped cosbench driver.
[root@ceph01 0.4.2.c4]# ./start-all.sh
Launching osgi framwork ...
Successfully launched osgi framework!
Booting cosbench driver ...
.
Starting cosbench-log_0.4.2 [OK]
Starting cosbench-tomcat_0.4.2 [OK]
Starting cosbench-config_0.4.2 [OK]
Starting cosbench-http_0.4.2 [OK]
Starting cosbench-cdmi-util_0.4.2 [OK]
Starting cosbench-core_0.4.2 [OK]
Starting cosbench-core-web_0.4.2 [OK]
Starting cosbench-api_0.4.2 [OK]
Starting cosbench-mock_0.4.2 [OK]
Starting cosbench-ampli_0.4.2 [OK]
Starting cosbench-swift_0.4.2 [OK]
Starting cosbench-keystone_0.4.2 [OK]
Starting cosbench-httpauth_0.4.2 [OK]
Starting cosbench-s3_0.4.2 [OK]
Starting cosbench-librados_0.4.2 [OK]
Starting cosbench-scality_0.4.2 [OK]
Starting cosbench-cdmi-swift_0.4.2 [OK]
Starting cosbench-cdmi-base_0.4.2 [OK]
Starting cosbench-driver_0.4.2 [OK]
Starting cosbench-driver-web_0.4.2 [OK]
Successfully started cosbench driver!
Listening on port 0.0.0.0/0.0.0.0:18089 ...
Persistence bundle starting...
Persistence bundle started.
----------------------------------------------
!!! Service will listen on web port: 18088 !!!
----------------------------------------------
======================================================
Launching osgi framwork ...
Successfully launched osgi framework!
Booting cosbench controller ...
.
Starting cosbench-log_0.4.2 [OK]
Starting cosbench-tomcat_0.4.2 [OK]
Starting cosbench-config_0.4.2 [OK]
Starting cosbench-core_0.4.2 [OK]
Starting cosbench-core-web_0.4.2 [OK]
Starting cosbench-controller_0.4.2 [OK]
Starting cosbench-controller-web_0.4.2 [OK]
Successfully started cosbench controller!
Listening on port 0.0.0.0/0.0.0.0:19089 ...
Persistence bundle starting...
Persistence bundle started.
----------------------------------------------
!!! Service will listen on web port: 19088 !!!
----------------------------------------------