Summary of dbcp configuration and jdbc timeout settings

The following sql on the eve of 618 in 2014:

<!--Add sync data-->
<insert id="insert" parameterClass="order">
  INSERT INTO aa(ID,ORDERID,CREATEDATE)
  VALUES
  (seq.Nextval,#orderId#,#createDate#)
  <selectKey resultClass="java.lang.Long">
    SELECT seq.CURRVAL FROM DUAL
  </selectKey>
</insert>

Will throw more than 800 errors as follows

Caused by: java.sql.SQLException: ORA-01013: user requested to cancel the current operation
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:331)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:288)
at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:745)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:219)
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:970)

The reason is that the sql execution time is too long, and the jdbc driver actively cancels the operation.

suggestion:

1. Look at the average execution time of the sql and set a timeout for the sql. (Too long execution time will occupy the connection, causing others to not get the connection)

2. Ask the DBA if there is a way to optimize the SQL, such as whether it can be inserted in parallel. Or can do partition.

1. Data source configuration

If you use apache dbcp and may encounter a bottleneck in the number of connections, you can adjust the following configuration:

<!—It is recommended that the following values be the same as possible, and there is no need to frequently expire idle connections (unless, for example, connection pool resources are scarce, which can be considered) -->
<property name="maxIdle" value="80" />
<property name="minIdle " value="80" />
<property name="initialSize" value="80"/>
<property name="maxActive" value="80" />

<!—This is the time to wait to obtain the connection pool connection, and it should not be too large. For example, set it at 500 milliseconds-->
<property name="maxWait" value="500" />

If it is a mysql library, there may be a problem of 8 hours. You can consider opening the expiration timer (numTestsPerEvictionRun=1), and periodically expire the connection. The timeBetweenEvictionRunsMillis time can be set to about 8 hours.

In addition, the socket connection/read timeout can be configured by the following configuration:

<property name="connectionProperties"

value="oracle.net.CONNECT_TIMEOUT=2000;oracle.jdbc.ReadTimeout=2000"></property>

(The connection and read timeout time here, please consider the size according to your own business)

For specific configuration, please refer to: http://www.importnew.com/2466.html

2, ibatis configuration

**The project is using ibatis-sqlmap-2.3.4.726.jar version, and since 2.3.1:

o Removed maxTransactions, maxRequests, maxSessions from configuration, all are now controlled by the resource providers。（即已经移除了maxTransactions, maxRequests, maxSessions配置）

So we only need the following configuration:

<settings cacheModelsEnabled="false" enhancementEnabled="true"

lazyLoadingEnabled="false" errorTracingEnabled="true" maxRequests="32"

defaultStatementTimeout="2"/>

The defaultStatementTimeout unit is seconds; according to business configuration.

If you want to only set the timeout of a certain Statement, you can consider: <insert ......timeout="2">

The following error was reported online before, because the statement execution timed out.

Cause: java.sql.SQLException: ORA-01013: user requested to cancel the current operation

3, spring transaction manager configuration

Provide a global transaction-level timeout:

</bean>

Summarize:

The main timeout settings are as follows:

1. Connection timeout

2. Timeout of reading data

3. Statement timeout

4. Transaction-level timeout = N* Statement timeout + GC pause time

Some problems with transaction timeouts have been summarized before. If you are interested, you can refer to the following:

http://jinnianshilongnian.iteye.com/blog/1986023

http://www.importnew.com/2466.html

Another point to note about database connection pools:

If destroy-method is not added, and the number of restarts is too frequent, the connection of the old database connection pool of restarting tomcat will not be released, so there will be many database connections that will not be released for a period of time; so it is better to add destroy-method.

////////////////////////////////////////////// ///////////////////////////////////////////// /////////////////////////////

When the maximum number of connections is exceeded, the connection will be deleted.

if (config != null && config.getRemoveAbandoned() && (getNumIdle() < 2) && (getNumActive() > getMaxActive() - 3) ) {

removeAbandoned();

}

The function of this code is to invalidate the orphan connection, that is, someone gets the connection but does not close it.

1. Cascading effects when the network is blocked/unstable (for example, the ssdb-client I write now will set a time when the network fails (the network is unavailable), and all requests within this time will be tiemout)

Inside the connection pool, according to the current network status (such as too many timeouts), for all timeouts within a certain period of time (such as 100ms), wait (maxWait) is not performed at all.

Another is the number of people currently waiting for the connection pool, such as waiting for 1000 now, then the next waiting is meaningless, which will also cause a snowball (ssdb-client adopts this strategy).

2. The wait timeout should be as small as possible (unless it is necessary), even if an error page is returned, it is better than waiting.

The problem with dbcp is that the set timeout time is too long, resulting in a large number of TIMED_WAIT, thread blocking, and snowballing. Once a problem occurs, it is difficult to reply immediately, and this can be solved by [1].

Most database clients will have a function to cancel the execution of the statement (that is, if we set QueryTimeout=2 seconds, if no information is returned within 2 seconds, then a task will actively send a canceled sql to cancel the execution of the current statement)

1. MySQL creates a Timer for each connection (each Timer creates a Thread)

2. Every time a Statement is created, a TimerTask will be submitted (each Task will create a Thread when it is executed)

That is to say, assuming we have a pool of 500 connections and each connection executes 1 statement, the worst case will create:

500*1+500*1=1000 threads.

Assuming that there are three mysql libraries in an application, the worst case is:

1000*3=3000 threads are created.

If our database adopts sub-database sub-table or read-write separation, it is conceivable. during times of stress.

Assuming that the os is not very fast for thread release, the canceled thread may not be available immediately (I am not sure, familiar students can explain).

And oracle uses a different strategy:

1. One watchdog thread per ClassLoader (similar to mysql's timer);

2. Each Statement has a Task, and the thread is triggered when the watchdog needs to cancel, that is, when the watchdog finds that the Statement needs to be canceled, it calls one of its methods, which quickly creates a thread and runs it;

That is to say, assuming we have a pool of 500 connections and each connection executes 1 statement, the worst case will create:

1+500*1=501 threads.

Assuming that there are three mysql libraries in an application, the worst case is:

1 + 500*3=1501 threads created.

solution:

1. The best solution is to change the mysql implementation.

2. Modify the number of threads supported by the underlying system.

////////////////////////////////////////////// ///////////////////////////////////////////// /////////////////////////////

In addition, dbcp 1.x uses commons-pool 1.x, and the performance under high concurrency is not very good; consider upgrading to 2.x or consider using druid or proxool for new projects, and migrate old projects carefully (I migrated before No problem, but be careful).