I. Introduction
Previous article has been used for quartz has made a preliminary presentation of [actual] Quartz quartz-2.2.3 source code analysis and [actual] Quartz and Spring Quartz integration herein, this analytical Quartz distributed clusters from source implementation point of view.
Two, the Quartz cluster
Quartz cluster-based database lock is achieved, a Quartz cluster each node is an independent Quartz application, it manages other nodes. This means that you must stop or start separately for each node. Quartz cluster node does not separate Quartz management node with another node or its communication, but rather perceives another application by the same Quartz database table. Cluster architecture:
Database Table
Table describes the information
qrtz_blob_triggers: Blob type is stored in flip-flop.
Quartz Calendar information is stored qrtz_calendars
qrtz_cron_triggers storage CronTrigger, including Cron expressions, and time zone information
qrtz_fired_triggers store state information associated with the Trigger has been triggered, and the implementation of information associated Job
Qrtz_job_details storing details of each of the configured Job
Pessimistic locking qrtz_locks stored program information
Qrtz_paused_trigger_grps information storage group paused Trigger
Qrtz_scheduler_state store small amounts of state information about the Scheduler, Scheduler and other examples
qrtz_simple_triggers stores the Trigger simple, comprising a number of repetitions, spacing, and number of times has contacted the
qrtz_simprop_triggers storage CalendarIntervalTrigger two types of triggers and DailyTimeIntervalTrigger
Information qrtz_triggers configured storage of Trigger
qrtz_locks is Quartz cluster synchronization mechanisms row lock table, including the following lock: CALENDAR_ACCESS, JOB_ACCESS, MISFIRE_ACCESS, STATE_ACCESS, TRIGGER_ACCESS
Third, source code analysis Quartz cluster synchronization mechanism (database row lock table)
Source Based Quartz2.3.0. From scheduling tasks QuartzSchedulerThread start to see its run method triggers the process of acquiring
try {
triggers = qsRsrcs.getJobStore().acquireNextTriggers(
now + idleWaitTime, Math.min(availThreadCount, qsRsrcs.getMaxBatchSize()), qsRsrcs.getBatchTimeWindow());
acquiresFailed = 0;
if (log.isDebugEnabled())
log.debug("batch acquisition of " + (triggers == null ? 0 : triggers.size()) + " triggers");
} catch (JobPersistenceException jpe) {
if (acquiresFailed == 0) {
qs.notifySchedulerListenersError(
"An error occurred while scanning for the next triggers to fire.",
jpe);
}
if (acquiresFailed < Integer.MAX_VALUE)
acquiresFailed++;
continue;
} catch (RuntimeException e) {
if (acquiresFailed == 0) {
getLog().error("quartzSchedulerThreadLoop: RuntimeException "
+e.getMessage(), e);
}
if (acquiresFailed < Integer.MAX_VALUE)
acquiresFailed++;
continue;
}JobStoreSupport.java
public List<OperableTrigger> acquireNextTriggers(final long noLaterThan, final int maxCount, final long timeWindow)
throws JobPersistenceException {
String lockName;
if(isAcquireTriggersWithinLock() || maxCount > 1) {
lockName = LOCK_TRIGGER_ACCESS;
} else {
lockName = null;
}
return executeInNonManagedTXLock(lockName,
new TransactionCallback<List<OperableTrigger>>() {
public List<OperableTrigger> execute(Connection conn) throws JobPersistenceException {
return acquireNextTrigger(conn, noLaterThan, maxCount, timeWindow);
}
},
new TransactionValidator<List<OperableTrigger>>() {
public Boolean validate(Connection conn, List<OperableTrigger> result) throws JobPersistenceException {
try {
List<FiredTriggerRecord> acquired = getDelegate().selectInstancesFiredTriggerRecords(conn, getInstanceId());
Set<String> fireInstanceIds = new HashSet<String>();
for (FiredTriggerRecord ft : acquired) {
fireInstanceIds.add(ft.getFireInstanceId());
}
for (OperableTrigger tr : result) {
if (fireInstanceIds.contains(tr.getFireInstanceId())) {
return true;
}
}
return false;
} catch (SQLException e) {
throw new JobPersistenceException("error validating trigger acquisition", e);
}
}
});
}Lock appeared, lockName
protected static final String LOCK_TRIGGER_ACCESS = "TRIGGER_ACCESS";
protected static final String LOCK_STATE_ACCESS = "STATE_ACCESS";Then look at the process to obtain the lock
/**
* Execute the given callback having optionally acquired the given lock.
* This uses the non-managed transaction connection.
*
* @param lockName The name of the lock to acquire, for example
* "TRIGGER_ACCESS". If null, then no lock is acquired, but the
* lockCallback is still executed in a non-managed transaction.
*/
protected <T> T executeInNonManagedTXLock(
String lockName,
TransactionCallback<T> txCallback, final TransactionValidator<T> txValidator) throws JobPersistenceException {
boolean transOwner = false;
Connection conn = null;
try {
if (lockName != null) {
// If we aren't using db locks, then delay getting DB connection
// until after acquiring the lock since it isn't needed.
if (getLockHandler().requiresConnection()) {
conn = getNonManagedTXConnection();
}
//获取锁
transOwner = getLockHandler().obtainLock(conn, lockName);
}
if (conn == null) {
conn = getNonManagedTXConnection();
}
final T result = txCallback.execute(conn);
try {
commitConnection(conn);
} catch (JobPersistenceException e) {
rollbackConnection(conn);
if (txValidator == null || !retryExecuteInNonManagedTXLock(lockName, new TransactionCallback<Boolean>() {
@Override
public Boolean execute(Connection conn) throws JobPersistenceException {
return txValidator.validate(conn, result);
}
})) {
throw e;
}
}
Long sigTime = clearAndGetSignalSchedulingChangeOnTxCompletion();
if(sigTime != null && sigTime >= 0) {
signalSchedulingChangeImmediately(sigTime);
}
return result;
} catch (JobPersistenceException e) {
rollbackConnection(conn);
throw e;
} catch (RuntimeException e) {
rollbackConnection(conn);
throw new JobPersistenceException("Unexpected runtime exception: "
+ e.getMessage(), e);
} finally {
try {
releaseLock(lockName, transOwner);
} finally {
cleanupConnection(conn);
}
}
}/**
* Grants a lock on the identified resource to the calling thread (blocking
* until it is available).
*
* @return true if the lock was obtained.
*/
public boolean obtainLock(Connection conn, String lockName)
throws LockException {
if(log.isDebugEnabled()) {
log.debug(
"Lock '" + lockName + "' is desired by: "
+ Thread.currentThread().getName());
}
if (!isLockOwner(lockName)) {
//执行sql，获取锁操作
executeSQL(conn, lockName, expandedSQL, expandedInsertSQL);
if(log.isDebugEnabled()) {
log.debug(
"Lock '" + lockName + "' given to: "
+ Thread.currentThread().getName());
}
getThreadLocks().add(lockName);
//getThreadLocksObtainer().put(lockName, new
// Exception("Obtainer..."));
} else if(log.isDebugEnabled()) {
log.debug(
"Lock '" + lockName + "' Is already owned by: "
+ Thread.currentThread().getName());
}
return true;
}org.quartz.impl.jdbcjobstore.DBSemaphore
Look at these two values
private String expandedSQL;
private String expandedInsertSQL;
private void setExpandedSQL() {
if (getTablePrefix() != null && getSchedName() != null && sql != null && insertSql != null) {
expandedSQL = Util.rtp(this.sql, getTablePrefix(), getSchedulerNameLiteral());
expandedInsertSQL = Util.rtp(this.insertSql, getTablePrefix(), getSchedulerNameLiteral());
}
}Look at lockHandler acquisition
private Semaphore lockHandler = null; // set in initialize() method...The initialize method
/**
* <p>
* Called by the QuartzScheduler before the <code>JobStore</code> is
* used, in order to give it a chance to initialize.
* </p>
*/
public void initialize(ClassLoadHelper loadHelper,
SchedulerSignaler signaler) throws SchedulerConfigException {
if (dsName == null) {
throw new SchedulerConfigException("DataSource name not set.");
}
classLoadHelper = loadHelper;
if(isThreadsInheritInitializersClassLoadContext()) {
log.info("JDBCJobStore threads will inherit ContextClassLoader of thread: " + Thread.currentThread().getName());
initializersLoader = Thread.currentThread().getContextClassLoader();
}
this.schedSignaler = signaler;
// If the user hasn't specified an explicit lock handler, then
// choose one based on CMT/Clustered/UseDBLocks.
if (getLockHandler() == null) {
// If the user hasn't specified an explicit lock handler,
// then we *must* use DB locks with clustering
if (isClustered()) {
setUseDBLocks(true);
}
if (getUseDBLocks()) {
if(getDriverDelegateClass() != null && getDriverDelegateClass().equals(MSSQLDelegate.class.getName())) {
if(getSelectWithLockSQL() == null) {
String msSqlDflt = "SELECT * FROM {0}LOCKS WITH (UPDLOCK,ROWLOCK) WHERE " + COL_SCHEDULER_NAME + " = {1} AND LOCK_NAME = ?";
getLog().info("Detected usage of MSSQLDelegate class - defaulting 'selectWithLockSQL' to '" + msSqlDflt + "'.");
setSelectWithLockSQL(msSqlDflt);
}
}
getLog().info("Using db table-based data access locking (synchronization).");
setLockHandler(new StdRowLockSemaphore(getTablePrefix(), getInstanceName(), getSelectWithLockSQL())); //handler赋值
} else {
getLog().info(
"Using thread monitor-based data access locking (synchronization).");
setLockHandler(new SimpleSemaphore());
}
}
}I seem to see the assignment of sql and lockHandler
new StdRowLockSemaphore(getTablePrefix(), getInstanceName(), getSelectWithLockSQL())
// MISC CONSTANTS
String DEFAULT_TABLE_PREFIX = "QRTZ_";
getInstanceName() //配置文件中的实例名
getSelectWithLockSQL() //这里为null，由于采用的不是mssql;
public StdRowLockSemaphore(String tablePrefix, String schedName, String selectWithLockSQL) {
super(tablePrefix, schedName, selectWithLockSQL != null ? selectWithLockSQL : SELECT_FOR_LOCK, INSERT_LOCK);
}
public static final String SELECT_FOR_LOCK = "SELECT * FROM "
+ TABLE_PREFIX_SUBST + TABLE_LOCKS + " WHERE " + COL_SCHEDULER_NAME + " = " + SCHED_NAME_SUBST
+ " AND " + COL_LOCK_NAME + " = ? FOR UPDATE";
public static final String INSERT_LOCK = "INSERT INTO "
+ TABLE_PREFIX_SUBST + TABLE_LOCKS + "(" + COL_SCHEDULER_NAME + ", " + COL_LOCK_NAME + ") VALUES ("
+ SCHED_NAME_SUBST + ", ?)";
public DBSemaphore(String tablePrefix, String schedName, String defaultSQL, String defaultInsertSQL) {
this.tablePrefix = tablePrefix;
this.schedName = schedName;
setSQL(defaultSQL);
setInsertSQL(defaultInsertSQL);
}There are two sql, two lockname
It can be seen using a Quartz cluster uses a pessimistic locking manner triggers table row lock, in order to ensure the correctness of the task synchronization. When the thread performing operations on the data in the table, the row line lock database using the SQL described above; the same time another thread performs operations before the data line needs to acquire a lock, but this time is occupied, then this thread can only wait until the row lock is released.
Back to
private String expandedSQL;
//SELECT * FROM QRTZ_LOCKS WHERE SCHED_NAME = 'schedulerFactory' AND LOCK_NAME = ? FOR UPDATE
private String expandedInsertSQL;
//INSERT INTO QRTZ_LOCKS (SCHED_NAME, LOCK_NAME) VALUES ('schedulerFactory', ?);
private void setExpandedSQL() {
if (getTablePrefix() != null && getSchedName() != null && sql != null && insertSql != null) {
expandedSQL = Util.rtp(this.sql, getTablePrefix(), getSchedulerNameLiteral());
expandedInsertSQL = Util.rtp(this.insertSql, getTablePrefix(), getSchedulerNameLiteral());
}
}The two values can be determined, and look at the process of acquiring the lock, where lockName as "TRIGGER_ACCESS"
org.quartz.impl.jdbcjobstore.StdRowLockSemaphore#executeSQL
/**
* Execute the SQL select for update that will lock the proper database row.
*/
@Override
protected void executeSQL(Connection conn, final String lockName, final String expandedSQL, final String expandedInsertSQL) throws LockException {
PreparedStatement ps = null;
ResultSet rs = null;
SQLException initCause = null;
// attempt lock two times (to work-around possible race conditions in inserting the lock row the first time running)
int count = 0;
do {
count++;
try {
ps = conn.prepareStatement(expandedSQL);
ps.setString(1, lockName);
if (getLog().isDebugEnabled()) {
getLog().debug(
"Lock '" + lockName + "' is being obtained: " +
Thread.currentThread().getName());
}
rs = ps.executeQuery();
if (!rs.next()) {
getLog().debug(
"Inserting new lock row for lock: '" + lockName + "' being obtained by thread: " +
Thread.currentThread().getName());
rs.close();
rs = null;
ps.close();
ps = null;
ps = conn.prepareStatement(expandedInsertSQL);
ps.setString(1, lockName);
int res = ps.executeUpdate();
if(res != 1) {
if(count < 3) {
// pause a bit to give another thread some time to commit the insert of the new lock row
try {
Thread.sleep(1000L);
} catch (InterruptedException ignore) {
Thread.currentThread().interrupt();
}
// try again ...
continue;
}
throw new SQLException(Util.rtp(
"No row exists, and one could not be inserted in table " + TABLE_PREFIX_SUBST + TABLE_LOCKS +
" for lock named: " + lockName, getTablePrefix(), getSchedulerNameLiteral()));
}
}
return; // obtained lock, go
} catch (SQLException sqle) {
//Exception src =
// (Exception)getThreadLocksObtainer().get(lockName);
//if(src != null)
// src.printStackTrace();
//else
// System.err.println("--- ***************** NO OBTAINER!");
if(initCause == null)
initCause = sqle;
if (getLog().isDebugEnabled()) {
getLog().debug(
"Lock '" + lockName + "' was not obtained by: " +
Thread.currentThread().getName() + (count < 3 ? " - will try again." : ""));
}
if(count < 3) {
// pause a bit to give another thread some time to commit the insert of the new lock row
try {
Thread.sleep(1000L);
} catch (InterruptedException ignore) {
Thread.currentThread().interrupt();
}
// try again ...
continue;
}
throw new LockException("Failure obtaining db row lock: "
+ sqle.getMessage(), sqle);
} finally {
if (rs != null) {
try {
rs.close();
} catch (Exception ignore) {
}
}
if (ps != null) {
try {
ps.close();
} catch (Exception ignore) {
}
}
}
} while(count < 4);
throw new LockException("Failure obtaining db row lock, reached maximum number of attempts. Initial exception (if any) attached as root cause.", initCause);
}The process of acquiring the lock also worth a careful look. Acquire a row lock mode, insert a guarantee to acquire the lock because the select for update failure may not have this data.
If you already have lockName representatives of the line, direct lock, if not inserted. But it may fail when locking or insertion failure retry, retry if more than a certain number will be directly thrown.
0 scheduler threads run ()
1 Get cocked trigger
1.1 database table TRIGGER_ACCESS row lock LOCKS
1.2 read information JobDetail
1.3 reads the trigger information in the trigger table and marked as "access"
1. 4commit transaction, the lock is released
2 . Trigger trigger
2.1 database table STATE_ACCESS row lock LOCKS
2.2 confirm the status of the trigger
2.3 reads the trigger information JobDetail
2.4 reads the trigger information Calendar
2.3 update trigger information
2. 3commit transaction, the lock is released
3 instantiate and execute Job
3.1 get the run method of the thread from the thread pool to perform JobRunShell
IV Summary
A scheduler instance before performing operations related to the distributed database issues, first obtain QUARTZ2_LOCKS scheduler table corresponding to the current row-level locks, the database can perform other operations after the acquisition of the table lock, with the operation of the transaction submitted , row level locking is released, for instance obtain other schedulers.
