Originally published in Daxie's blog: https://ziyang.moe/article/mydb1.html
1. Start with TM
TM maintains the state of the transaction by maintaining the XID file, and provides an interface for other modules to query the state of a certain transaction.
XID file
In MYDB, each transaction has an XID, which uniquely identifies the transaction.
The XID of the transaction starts from 1 and increments by itself, and cannot be repeated. Special XID 0 is a super transaction. When some operations want to be performed without applying for a transaction, you can set the XID of the operation to 0. The status of a transaction with XID 0 is always committed.
TransactionManager maintains a file in XID format to record the status of each transaction. In MYDB, each transaction has the following three states:
1. active, running, not finished yet
2. committed, submitted
3. aborted, revoked (rolled back)
The XID file allocates one byte of space to each transaction to store its state. At the same time, an 8-byte number is stored at the head of the XID file, recording the number of transactions managed by the XID file. Therefore, the status of the transaction xid in the file is stored at (xid-1)+8 bytes, and xid-1 is because the status of xid 0 (Super XID) does not need to be recorded.
The content of the XID file is as follows:
Define the TransactionManager interface for other modules to call to create transactions and query transaction status.
public interface TransactionManager {
long begin(); // 开启一个新事务
void commit(long xid); // 提交一个事务
void abort(long xid); // 取消一个事务
boolean isActive(long xid); // 查询一个事务的状态是否是正在进行的状态
boolean isCommitted(long xid); // 查询一个事务的状态是否是已提交
boolean isAborted(long xid); // 查询一个事务的状态是否是已取消
void close(); // 关闭TM
}
accomplish
First define some necessary constants and member variables:
File reading and writing all use the NIO FileChannel.
public class TransactionManagerImpl implements TransactionManager{
// XID文件头长度
static final int LEN_XID_HEADER_LENGTH = 8;
// 每个事务的占用长度
private static final int XID_FIELD_SIZE = 1;
// 事务的三种状态
private static final byte FIELD_TRAN_ACTIVE = 0;
private static final byte FIELD_TRAN_COMMITTED = 1;
private static final byte FIELD_TRAN_ABORTED = 2;
// 超级事务，永远为commited状态
public static final long SUPER_XID = 0;
// XID 文件后缀
static final String XID_SUFFIX = ".xid";
private RandomAccessFile file;
private FileChannel fc;
private long xidCounter;
private Lock counterLock;
TransactionManagerImpl(RandomAccessFile raf, FileChannel fc) {
this.file = raf;
this.fc = fc;
counterLock = new ReentrantLock();
checkXIDCounter();
}
}
Check whether the XID file is legal, read the xidcounter in XID_FILE_HEADER, calculate the theoretical length of the file according to it, and compare the actual length
/**
* 检查XID文件是否合法
* 读取XID_FILE_HEADER中的xidcounter，根据它计算文件的理论长度，对比实际长度
*/
private void checkXIDCounter() {
long fileLen = 0;
try {
fileLen = file.length(); //返回文件的字节数
} catch (IOException e1) {
Panic.panic(Error.BadXIDFileException);
}
if(fileLen < LEN_XID_HEADER_LENGTH) {
Panic.panic(Error.BadXIDFileException);
}
ByteBuffer buf = ByteBuffer.allocate(LEN_XID_HEADER_LENGTH);
try {
fc.position(0);
fc.read(buf);
} catch (IOException e) {
Panic.panic(e);
}
this.xidCounter = Parser.parseLong(buf.array());
long end = getXidPosition(this.xidCounter + 1); //计算理论的字节长度
if(end != fileLen) {
Panic.panic(Error.BadXIDFileException);
}
}
Calculate the theoretical byte length (where the current xid state is located)
// 根据事务xid取得其在xid文件中对应的位置
private long getXidPosition(long xid) {
return LEN_XID_HEADER_LENGTH + (xid-1)*XID_FIELD_SIZE;
}
Obtain the position of the current xid state in the file through the ==getXidPosition(long xid) == method, and update the state of the xid tostatus
// 更新xid事务的状态为status
private void updateXID(long xid, byte status) {
long offset = getXidPosition(xid);
byte[] tmp = new byte[XID_FIELD_SIZE];
tmp[0] = status;
ByteBuffer buf = ByteBuffer.wrap(tmp);
try {
fc.position(offset);
fc.write(buf);
} catch (IOException e) {
Panic.panic(e);
}
try {
fc.force(false);
} catch (IOException e) {
Panic.panic(e);
}
}
After updating the status of the xid, you need to update the header of the xid file
// 将XID加一，并更新XID Header
private void incrXIDCounter() {
xidCounter ++;
ByteBuffer buf = ByteBuffer.wrap(Parser.long2Byte(xidCounter));
try {
fc.position(0);
fc.write(buf);
} catch (IOException e) {
Panic.panic(e);
}
try {
fc.force(false);
} catch (IOException e) {
Panic.panic(e);
}
}
Start the transaction: first use the ReentrantLock to lock, record the current xid status as 0, and add the xid information in the header, and finally release the lock.
@Override
public long begin() {
counterLock.lock();
try {
long xid = xidCounter + 1;
updateXID(xid, FIELD_TRAN_ACTIVE);
incrXIDCounter();
return xid;
} finally {
counterLock.unlock();
}
}
The submit operation is the same as the undo operation, only need to modify the status of the current xid
@Override
public void commit(long xid) {
updateXID(xid, FIELD_TRAN_COMMITTED);
}
@Override
public void abort(long xid) {
updateXID(xid, FIELD_TRAN_ABORTED);
}
To detect whether the XID transaction is in the status state, first use getXidPosition()the method to find the location where the current xid state is stored, then move FeilChannal to the state location, read the current state, and then statuscompare with
// 检测XID事务是否处于status状态
private boolean checkXID(long xid, byte status) {
long offset = getXidPosition(xid);
ByteBuffer buf = ByteBuffer.wrap(new byte[XID_FIELD_SIZE]);
try {
fc.position(offset);
fc.read(buf);
} catch (IOException e) {
Panic.panic(e);
}
return buf.array()[0] == status;
}
Check whether it is in progress, submitted, or withdrawn. The same reason, you only need to call checkXID()the method to pass in the corresponding parameters.
@Override
public boolean isActive(long xid) {
if(xid == SUPER_XID) {
return false;
}
return checkXID(xid, FIELD_TRAN_ACTIVE);
}
@Override
public boolean isCommitted(long xid) {
if(xid == SUPER_XID) {
return true;
}
return checkXID(xid, FIELD_TRAN_COMMITTED);
}
@Override
public boolean isAborted(long xid) {
if(xid == SUPER_XID) {
return false;
}
return checkXID(xid, FIELD_TRAN_ABORTED);
}
The last is to close the operation
@Override
public void close() {
try {
fc.close();
file.close();
} catch (IOException e) {
Panic.panic(e);
}
}
test code
public class TransactionManagerTest {
static Random random = new SecureRandom();
private int transCnt = 0;
private int noWorkers = 50;
private int noWorks = 3000;
private Lock lock = new ReentrantLock();
private TransactionManager tmger;
private Map<Long, Byte> transMap;
private CountDownLatch cdl;
@Test
public void testMultiThread() {
tmger = TransactionManager.create("E:\\学习\\test\\tmp\\tranmger_test");
transMap = new ConcurrentHashMap<>();
cdl = new CountDownLatch(noWorkers);
for(int i = 0; i < noWorkers; i ++) {
Runnable r = () -> worker();
new Thread(r).run();
}
try {
cdl.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
//tmger.close();
assert new File("E:\\学习\\test\\tmp\\tranmger_test.xid").delete();
}
private void worker() {
boolean inTrans = false;
long transXID = 0;
for(int i = 0; i < noWorks; i ++) {
int op = Math.abs(random.nextInt(6));
if(op == 0) {
lock.lock();
if(inTrans == false) {
long xid = tmger.begin();
transMap.put(xid, (byte)0);
transCnt ++;
transXID = xid;
inTrans = true;
} else {
int status = (random.nextInt(Integer.MAX_VALUE) % 2) + 1;
switch(status) {
case 1:
tmger.commit(transXID);
break;
case 2:
tmger.abort(transXID);
break;
}
transMap.put(transXID, (byte)status);
inTrans = false;
}
lock.unlock();
} else {
lock.lock();
if(transCnt > 0) {
long xid=(long)((random.nextInt(Integer.MAX_VALUE)%transCnt) + 1);
byte status = transMap.get(xid);
boolean ok = false;
switch (status) {
case 0:
ok = tmger.isActive(xid);
break;
case 1:
ok = tmger.isCommitted(xid);
break;
case 2:
ok = tmger.isAborted(xid);
break;
}
assert ok;
}
lock.unlock();
}
}
cdl.countDown();
}
}