First of all, please don't close this as duplicate of What is NullPointerException and how to fix it. I know what is NullPointerException, and I know how to solve it in my own code, but not when it is thrown by mysql-connector-java-5.1.36-bin.jar, which I have no control over.
We encountered the exception when running a common DB query, on a mySQL DB, that works most of the time. We started seeing this exception after deploying a new version, but the query in which it occurs didn't change in a long time.
Here's how the query looks like (with some necessary simplifications). I surrounded it with some of the logic performed before and after. The actual code is not all in a single method, but I put it in a single block to make it easier to understand.
Connection conn = ... // the connection is open
...
for (String someID : someIDs) {
SomeClass sc = null;
PreparedStatement
stmt = conn.prepareStatement ("SELECT A, B, C, D, E, F, G, H FROM T WHERE A = ?");
stmt.setString (1, "someID");
ResultSet res = stmt.executeQuery ();
if (res.next ()) {
sc = new SomeClass ();
sc.setA (res.getString (1));
sc.setB (res.getString (2));
sc.setC (res.getString (3));
sc.setD (res.getString (4));
sc.setE (res.getString (5));
sc.setF (res.getInt (6));
sc.setG (res.getString (7));
sc.setH (res.getByte (8)); // the exception is thrown here
}
stmt.close ();
conn.commit ();
if (sc != null) {
// do some processing that involves loading other records from the
// DB using the same connection
}
}
conn.close();
res.getByte(8) causes a NullPointerException with the following call stack :
com.mysql.jdbc.ResultSetImpl.checkColumnBounds(ResultSetImpl.java:763) com.mysql.jdbc.ResultSetImpl.getStringInternal(ResultSetImpl.java:5251) com.mysql.jdbc.ResultSetImpl.getString(ResultSetImpl.java:5173) com.mysql.jdbc.ResultSetImpl.getByte(ResultSetImpl.java:1650) org.apache.tomcat.dbcp.dbcp2.DelegatingResultSet.getByte(DelegatingResultSet.java:206) org.apache.tomcat.dbcp.dbcp2.DelegatingResultSet.getByte(DelegatingResultSet.java:206)
I searched for the source code of the relevant mysql-connector version and found this (taken from here):
756 protected final void checkColumnBounds(int columnIndex) throws SQLException {
757 synchronized (checkClosed().getConnectionMutex()) {
758 if ((columnIndex < 1)) {
759 throw SQLError.createSQLException(
760 Messages.getString("ResultSet.Column_Index_out_of_range_low",
761 new Object[] { Integer.valueOf(columnIndex), Integer.valueOf(this.fields.length) }), SQLError.SQL_STATE_ILLEGAL_ARGUMENT,
762 getExceptionInterceptor());
763 } else if ((columnIndex > this.fields.length)) {
764 throw SQLError.createSQLException(
765 Messages.getString("ResultSet.Column_Index_out_of_range_high",
766 new Object[] { Integer.valueOf(columnIndex), Integer.valueOf(this.fields.length) }), SQLError.SQL_STATE_ILLEGAL_ARGUMENT,
767 getExceptionInterceptor());
768 }
769
770 if (this.profileSql || this.useUsageAdvisor) {
771 this.columnUsed[columnIndex - 1] = true;
772 }
773 }
774 }
As you can see, the exception occurs on this line :
} else if ((columnIndex > this.fields.length)) {
which means this.fields somehow became null.
The closest thing I could find was this question, which has no answer.
I suspect the issue is not in the query I posted. Perhaps something went wrong with the Connection instance due to some other statements we are running on the same connection. All I can say is that we close every statement right after we execute it and read the data from its ResultSet.
EDIT (1/19/2017):
I couldn't recreate the error in my development environment. I thought it might be some mysql-connector bug triggered when the same connection is used for a long time. I limited the above loop to load at most 6 elements at a time. In addition, we upgraded the mysql-connector version to 5.1.40.
We still see NullPointerExceptions in ResultSetImpl, but this time at a different location.
The stack trace is :
com.mysql.jdbc.ResultSetImpl.getStringInternal(ResultSetImpl.java:5294) com.mysql.jdbc.ResultSetImpl.getString(ResultSetImpl.java:5151) org.apache.tomcat.dbcp.dbcp2.DelegatingResultSet.getString(DelegatingResultSet.java:198) org.apache.tomcat.dbcp.dbcp2.DelegatingResultSet.getString(DelegatingResultSet.java:198)
which implies this time the exception was thrown as a result of one of our res.getString() calls (I don't know which one).
I found the source for mysql-connector-java-5.1.40-bin.jar here and the relevant code is:
5292 // Handles timezone conversion and zero-date behavior
5293
5294 if (checkDateTypes && !this.connection.getNoDatetimeStringSync()) {
5295 switch (metadata.getSQLType()) {
which implies this.connection is null. this.connection is an instance variable of type MySQLConnection, which gets initialized in the constructors of ResultSetImpl and is only set to null when public void realClose(boolean calledExplicitly) is called (it is called by public void close(), which, according to the documentation within the source, is called when ResultSet.close() is called). We are definitely not closing the ResultSet prior to reading all the data from it.
Any ideas how to proceed?
It has been a long time since I posted this question, and I want to post an answer that describes the exact scenario that led to this tricky NullPointerException.
I think this may help future readers who encounter such a confusing exception think outside the box, since I had almost every reason to suspect this was a mysql connector bug, even though it wasn't after all.
While investigating this exception I was sure that my application can't possibly be closing the DB connection while trying to read data from it, since my DB connections are not shared across threads, and if the same thread closed the connection and then tried to access it, a different exception should have been thrown (some SQLException). That was the main reason I suspected a mysql connector bug.
It turned out that there were two threads accessing the same connection after all. The reason this was hard to figure out was that one of these threads was a garbage collector thread.
Going back to the code I posted:
Connection conn = ... // the connection is open
...
for (String someID : someIDs) {
SomeClass sc = null;
PreparedStatement
stmt = conn.prepareStatement ("SELECT A, B, C, D, E, F, G, H FROM T WHERE A = ?");
stmt.setString (1, "someID");
ResultSet res = stmt.executeQuery ();
if (res.next ()) {
sc = new SomeClass ();
sc.setA (res.getString (1));
sc.setB (res.getString (2));
sc.setC (res.getString (3));
sc.setD (res.getString (4));
sc.setE (res.getString (5));
sc.setF (res.getInt (6));
sc.setG (res.getString (7));
sc.setH (res.getByte (8)); // the exception is thrown here
}
stmt.close ();
conn.commit ();
if (sc != null) {
// do some processing that involves loading other records from the
// DB using the same connection
}
}
conn.close();
The problem lies in the "do some processing that involves loading other records from the DB using the same connection" section, which, unfortunately, I didn't include in my original question, since I didn't think the problem was there.
Zooming into that section, we have:
if (sc != null) {
...
someMethod (conn);
...
}
And someMethod looks like this:
public void someMethod (Connection conn)
{
...
SomeOtherClass instance = new SomeOtherClass (conn);
...
}
SomeOtherClass looks like this (of course I'm simplifying here):
public class SomeOtherClass
{
Connection conn;
public SomeOtherClass (Connection conn)
{
this.conn = conn;
}
protected void finalize() throws Throwable
{
if (this.conn != null)
conn.close();
}
}
SomeOtherClass may create its own DB connection in some scenarios, but can accept an existing connection in other scenarios, like the one we have here.
As you can see, That section contains a call to someMethod that accepts the open connection as argument. someMethod passes the connection to a local instance of SomeOtherClass. SomeOtherClass had a finalize method which closes the connection.
Now, after someMethod returns, instance becomes eligible for garbage collection. When it is garbage collected, its finalize method is called by the garbage collector thread, which closes the connection.
Now we get back to the for loop, which continues to execute SELECT statements using the same connection which may be closed any time by the garbage collector thread.
If the garbage collector thread happens to close the connection while the application thread is in the middle of some mysql connector method that relies on the connection to be open, a NullPointerException may occur.
Removing the finalize method solved the issue.
We don't often override the finalize method in our classes, which made it very difficult to locate the bug.