1，问题描述
同事说crm登录hang住了，tomcat后台应用报错如下：
2016-06-28 10:30:11,214 [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0] WARN [com.mchange.v2.resourcepool.BasicResourcePool] - com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@3930f3cc -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30). Last acquisition attempt exception: java.sql.SQLException: Io exception: Message file 'oracle.net.mesg.Message' is missing. at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112) at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:146) at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:255) at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:387) at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:414) at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:165) at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:35) at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:801) at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:134) at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:182) at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:171) at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:137) at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1014) at com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:32) at com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask.run(BasicResourcePool.java:1810) at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547) 2016-06-28 10:30:56,213 [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0] WARN [com.mchange.v2.resourcepool.BasicResourcePool] - com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@7127f4c0 -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30). Last acquisition attempt exception: java.sql.SQLException: Io exception: Message file 'oracle.net.mesg.Message' is missing. at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112) at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:146) at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:255) at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:387) at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:414) at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:165) at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:35) at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:801) at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:134) at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:182) at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:171) at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:137) at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1014) at com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:32) at com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask.run(BasicResourcePool.java:1810) at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)
Jun 28, 2016 10:33:59 AM org.apache.tomcat.util.net.JIoEndpoint$Acceptor run SEVERE: Socket accept failed java.net.SocketException: Too many open files at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:60) at org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:222) at java.lang.Thread.run(Thread.java:745) |
2，查看数据库能否正常操作
看到有信息java.sql.SQLException: Ioexception: Message file 'oracle.net.mesg.Message' is missing. 先去查看数据库是否正常：通过plsql连接oracle数据库，显示数据库正常：
SQL> create table zz_bak (id number);Table createdSQL> insert into zz_bak(id) values(1);1 row insertedSQL> commit;Commit completeSQL> select * from zz_bak; ID---------- 1SQL> drop table zz_bak;Table dropped SQL>3，查看是否有数据库锁
-- 用dba权限的用户查看数据库都有哪些锁，没有查询到记录 selectt2.username,t2.sid,t2.serial#,t2.logon_time from v$locked_object t1,v$sessiont2 where t1.session_id=t2.sid order by t2.logon_time; -- 查看session等待，也没用记录 SELECT/*+ rule */ lpad(' ', decode(l.xidusn, 0, 3, 0)) || l.oracle_usernameUser_name,o.owner,o.object_name,o.object_type,s.sid,s.serial#,p.spid FROMv$locked_object l, dba_objects o, v$session s, v$process p WHEREl.object_id = o.object_id AND l.session_id = s.sid and s.paddr =p.addr ORDERBY o.object_id, xidusn DESC
4，检查应用程序所在的linux服务器
看到有“java.net.SocketException:Too many open files”信息，查看打开文件数：
[tomcat@crmtestlogs]$ ulimit -acore file size (blocks, -c) 0data seg size (kbytes, -d) unlimitedscheduling priority (-e) 0file size (blocks, -f) unlimitedpending signals (-i) 112257max locked memory (kbytes, -l) 64max memory size (kbytes, -m) unlimitedopen files (-n) 65535pipe size (512 bytes, -p) 8POSIX message queues (bytes, -q) 819200real-time priority (-r) 0stack size (kbytes, -s) 10240cpu time (seconds, -t) unlimitedmax user processes (-u) 65535virtual memory (kbytes, -v) unlimitedfile locks (-x) unlimited[tomcat@crmtestlogs]$查看系统句柄文件数，当前系统文件句柄的最大数目，只用于查看，不能设置修改
# cat /proc/sys/fs/file-max[tomcat@crmtestlogs]$ cat/proc/sys/fs/file-max1427648[tomcat@crmtestlogs]$如果需要查看所有进程的文件打开数，如下图命令lsof |wc –l：
[tomcat@crmtestlogs]$ lsof |wc -l1508[tomcat@crmtestlogs]$分析得出：文件句柄数量有些多，但是打开文件数还好不算太多
文件句柄数有些多，所以采用解决办法是：重启tomcat应用服务器，释放打开的文件句柄，之后观察5分钟，再也不会报错了。
再分享一下我老师大神的人工智能教程吧。零基础！通俗易懂！风趣幽默！还带黄段子！希望你也加入到我们人工智能的队伍中来！https://blog.csdn.net/jiangjunshow