TPorfiler Monitor System

TPorfiler Monitor System

Fetch the latest code
>git clone https://github.com/alibaba/TProfiler.git

Build the Final codes
>mvn assembly:assembly

Find and unzip the file in some place
>unzip TProfiler_1.0.1.zip

Link to my usually working place
>sudo ln -s /Users/carl/tool/tprofiler-1.0.1 /opt/tprofiler-1.0.1
>sudo ln -s /opt/tprofiler-1.0.1 /opt/tprofiler

Copy the jar and configurations files to project scripts directory
>cp /opt/tprofiler/lib/tprofiler-1.0.1.jar ./
>cp /opt/tprofiler/profile.properties ./

Place the libraries and configurations in the right place. The command to start the process may be like this
>java -Xms2G -Xmx2G -XX:MaxMetaspaceSize=256M -Dbuild.env=prod -Drunning_module=SCAN_FILE -jar target/feeds-2g-1.0-jar-with-dependencies.jar -javaagent:/opt/feeds-2g/scripts/tprofiler-1.0.1.jar -Dprofile.properties=/opt/feeds-2g/scripts/profile.properties

Prepare all ENV on Docker

Some changes on Dockerfile
#Prepare S3 command tool
RUN        yum --enablerepo=extras -y install epel-release
RUN        yum -y update
RUN        yum -y install python-pip
RUN        pip install --upgrade pip
RUN        pip install --upgrade --user awscli
ENV         PATH="~/.local/bin:${PATH}"

ADD     scripts/tprofiler-1.0.1.jar /share/
ADD     scripts/profile.properties /share/

I want to have s3 command there, then I can upload my logging to S3 in docker container.

Changes on the start.sh Command
-javaagent:/share/tprofiler-1.0.1.jar -Dprofile.properties=/share/profile.properties \

It will generate a lot of logging files in the right place as follow:
tmethod.log
tmethod.log.2017-05-12
…snip…
tprofiler.log
tprofiler.log.2017-05-12
…snip…
tsample.log
tsample.log.2017-05-12

Check and Analyze the Logging
Based on sampler.log, generate method.log and thread.log
>java -cp lib/tprofiler-1.0.1.jar com.taobao.profile.analysis.SamplerLogAnalysis /opt/tprofiler/jvm-logging/tsampler.log.2017-05-13 /opt/tprofiler/jvm-logging/method.log /opt/tprofiler/jvm-logging/thread.log

method.log details
method information ————— frequencies of the method
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:534) 2697
com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:64)   899
com.taobao.profile.thread.InnerSocketThread.run(InnerSocketThread.java:44)      899
com.taobao.profile.thread.TimeControlThread.await(TimeControlThread.java:84)    899
com.amazonaws.http.IdleConnectionReaper.run(IdleConnectionReaper.java:181)      899
com.taobao.profile.thread.TimeControlThread.run(TimeControlThread.java:116)     899
com.taobao.profile.thread.SamplerThread.run(SamplerThread.java:57)      899
com.taobao.profile.thread.DataDumpThread.run(DataDumpThread.java:69)    810
com.taobao.profile.thread.DataDumpThread.run(DataDumpThread.java:62)    82

thread.log details
thread information ———— frequencies of the thread
16      feeds2g-scheduler-1     TIMED_WAITING   899
5       TProfiler-TimeControl   TIMED_WAITING   899
27      com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2 TIMED_WAITING   899
28      Abandoned connection cleanup thread     TIMED_WAITING   899
4       Signal Dispatcher       RUNNABLE        899
26      com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1 TIMED_WAITING   899
2       Reference Handler       WAITING 899
8       TProfiler-Sampler       RUNNABLE        899
20      java-sdk-http-connection-reaper TIMED_WAITING   899
24      Timer-0 TIMED_WAITING   899
7       TProfiler-DataDump      TIMED_WAITING   899
22      DestroyJavaVM   RUNNABLE        899
25      com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0 TIMED_WAITING   899
6       TProfiler-InnerSocket   RUNNABLE        899
3       Finalizer       WAITING 898
80990   feeds2g-akka.actor.default-dispatcher-149       WAITING 566
94353   feeds2g-akka.actor.default-dispatcher-178       WAITING 269
67734   feeds2g-akka.actor.default-dispatcher-140       WAITING 94
80989   feeds2g-akka.actor.default-dispatcher-148       WAITING 27
94341   feeds2g-akka.actor.default-dispatcher-167       WAITING 26
81000   feeds2g-akka.actor.default-dispatcher-159       WAITING 25
94348   feeds2g-akka.actor.default-dispatcher-174       WAITING 23

Generate the Top Method and Top Object
>java -cp lib/tprofiler-1.0.1.jar com.taobao.profile.analysis.ProfilerLogAnalysis /opt/tprofiler/jvm-logging/tprofiler.log.2017-05-13 /opt/tprofiler/jvm-logging/tmethod.log.2017-05-13 /opt/tprofiler/jvm-logging/topmethod.log /opt/tprofiler/jvm-logging/topobject.log

Top Method Information
method Name ————————— exe times —————     average execute time ———— total execute time
com/j2c/feeds2g/services/actors/JobIDExportActor:onReceive:79   383     580     222015
com/j2c/feeds2g/services/RedisJRedisServiceImpl:fetchJobByID:102        62640   2       152078
com/j2c/feeds2g/services/CSVMappingServiceOpenCSVImpl:toCSV:73  653     135     87993
com/j2c/feeds2g/services/S3ServiceImpl:upload:87        14      604     8457
com/j2c/feeds2g/commons/utils/CompressUtil:gzuncompress:76      648     8       5276
com/j2c/feeds2g/services/JSONMappingServiceJacksonImpl:toJava:30        384     5       1842
com/j2c/feeds2g/models/Job:toString:855 238     5       1295
com/j2c/feeds2g/services/MessageServiceImpl:sendJobBucket:26    15      39      583
com/j2c/feeds2g/services/actors/JobContentExportActor:onReceive:97      689     0       163
com/j2c/feeds2g/commons/utils/DateTimeUtil:getDateTimeString:53 16      9       139
com/j2c/feeds2g/commons/utils/ArrayHelper:convertArray2String:17        9       8       73

Top Object Information
method Name ————————— exe times —————     average execute time ———— total execute time
com/j2c/feeds2g/models/messages/JobContentMessage:<init>:15     2       2       4
com/j2c/feeds2g/models/Job:<init>:248   1       4       4


References:
https://github.com/alibaba/TProfiler/wiki/TProfiler%E9%85%8D%E7%BD%AE%E9%83%A8%E7%BD%B2
https://github.com/alibaba/TProfiler/wiki/TProfiler%E6%97%A5%E5%BF%97%E5%88%86%E6%9E%90
http://www.importnew.com/22438.html
https://github.com/alibaba/TProfiler/issues/33