The new version of Alibaba Seata finally solves the idempotence, suspension and empty rollback problems of TCC mode

author ： Zhu Jinjun

Hello everyone , I'm brother Jun .

Today, let's talk about Alibaba  Seata  The new version （1.5.1） How to solve  TCC  Idempotence in mode 、 Suspension and empty rollback problems .

TCC  review

TCC  Pattern is the most classic distributed transaction solution , It divides distributed transactions into two phases to execute ,try  Stage reserves resources for each branch transaction , If all branch transactions reserve resources successfully , entering  commit  Phase commit global transaction , If a node fails to reserve resources, enter  cancel  Phase rollback global transaction .

With traditional orders 、 stock 、 Take account services for example , stay  try  Phase tries to reserve resources , Insert order 、 Deducting the inventory 、 Deduction amount , These three services all commit local transactions , Resources can be transferred to the intermediate table . stay  commit  Stage , And then  try  The resources reserved in the phase are transferred to the final table . And in the  cancel  Stage , hold  try  Release the resources reserved in the stage , For example, return the account amount to the customer's account .

Be careful ：try  The phase must be the one to commit the local transaction , For example, deduct the order amount , The money must be deducted from the customer's account , If not , stay  commit  There is not enough money in the customer's account at this stage , There will be problems .

try-commit

try  In this phase, resources are reserved first , And then in  commit  Phase deduction resources . Here's the picture ：

try-cancel

try  In this phase, resources are reserved first , Failed to deduct inventory when reserving resources, resulting in global transaction rollback , stay  cancel  Stage release resources . Here's the picture ：

TCC  advantage

TCC  The biggest advantage of the model is high efficiency .TCC  Patterns in  try  Locking resources in a phase is not really locking , Instead, the local transaction is actually committed , Reserve resources in the intermediate state , There is no need to block waiting , Therefore, the efficiency is higher than other modes .

meanwhile  TCC  The pattern can also be optimized as follows ：

Asynchronous submission

try  After the successful stage , Do not enter immediately  confirm/cancel  Stage , Instead, it thinks that the global transaction is over , Start a scheduled task to execute asynchronously  confirm/cancel, Deduct or release resources , This will greatly improve the performance .

Same as library mode

TCC  There are three roles in the pattern ：

• TM： Manage global affairs , Including starting global transactions , Submit / Roll back global transactions ;

• RM： Manage branch transactions ;

• TC:  Manage the status of global and branch transactions .

The following figure comes from  Seata  Official website ：

TM  When opening a global transaction ,RM  You need to  TC  Send registration message ,TC  Save the state of the branch transaction .TM  When requesting commit or rollback ,TC  You need to  RM  Send commit or rollback messages . So in a distributed transaction that contains two branch transactions ,TC  and  RM  There are four times  RPC.

The optimized process is shown in the following figure ：

TC  Save the state of the global transaction .TM  When opening a global transaction ,RM  No need to talk to  TC  Send registration message , Instead, the branch transaction state is saved locally .TM  towards  TC  After sending a commit or rollback message ,RM  The asynchronous thread first finds out the uncommitted branch transactions saved locally , And then to  TC  Send a message to get （ The local branch office is located in ） Global transaction status , To decide whether to commit or rollback the local transaction .

After this optimization ,RPC  Fewer times  50%, Performance improvement .

RM  Code example

Take inventory service as an example ,RM  The inventory service interface code is as follows ：

@LocalTCC
public interface StorageService {

 /**
 *  Deducting the inventory
 * @param xid  overall situation xid
 * @param productId  product id
 * @param count  Number
 * @return
 */
 @TwoPhaseBusinessAction(name = "storageApi", commitMethod = "commit", rollbackMethod = "rollback", useTCCFence = true)
 boolean decrease(String xid, Long productId, Integer count);

 /**
 *  Commit transaction
 * @param actionContext
 * @return
 */
 boolean commit(BusinessActionContext actionContext);

 /**
 *  Roll back the transaction
 * @param actionContext
 * @return
 */
 boolean rollback(BusinessActionContext actionContext);
}


adopt  @LocalTCC  This annotation ,RM  During initialization, you will send a message to  TC  Register a branch transaction . stay  try  Stage approach （decrease Method ） There's one on  @TwoPhaseBusinessAction  annotation , Here we define the branch transaction  resourceId,commit  Methods and  cancel  Method ,useTCCFence  This attribute will be discussed in the next section .

TCC  Existing problems

TCC  The three major problems in patterns are idempotence 、 Hang and empty rollback . stay  Seata1.5.1  In the version , A transaction control table is added , Table name is  tcc_fence_log  To solve this problem . And in the last section  @TwoPhaseBusinessAction  The attributes mentioned in the comments  useTCCFence  To specify whether to enable this mechanism , The default value of this attribute is  false.

tcc_fence_log  Build the predicative sentence as follows （MySQL  grammar ）：

CREATE TABLE IF NOT EXISTS `tcc_fence_log`
(
 `xid` VARCHAR(128) NOT NULL COMMENT 'global id',
 `branch_id` BIGINT NOT NULL COMMENT 'branch id',
 `action_name` VARCHAR(64) NOT NULL COMMENT 'action name',
 `status` TINYINT NOT NULL COMMENT 'status(tried:1;committed:2;rollbacked:3;suspended:4)',
 `gmt_create` DATETIME(3) NOT NULL COMMENT 'create time',
 `gmt_modified` DATETIME(3) NOT NULL COMMENT 'update time',
 PRIMARY KEY (`xid`, `branch_id`),
 KEY `idx_gmt_modified` (`gmt_modified`),
 KEY `idx_status` (`status`)
) ENGINE = InnoDB
DEFAULT CHARSET = utf8mb4;


idempotent

stay  commit/cancel  Stage , because  TC  No response received from branch transaction , Need to try again , This requires that branch transactions support idempotence .

Let's see how the new version solves the problem . The following code is in  TCCResourceManager  class ：

@Override
public BranchStatus branchCommit(BranchType branchType, String xid, long branchId, String resourceId,
 String applicationData) throws TransactionException {
 TCCResource tccResource = (TCCResource)tccResourceCache.get(resourceId);
 // Omit judgment
 Object targetTCCBean = tccResource.getTargetBean();
 Method commitMethod = tccResource.getCommitMethod();
 // Omit judgment
 try {
 //BusinessActionContext
 BusinessActionContext businessActionContext = getBusinessActionContext(xid, branchId, resourceId,
 applicationData);
 Object[] args = this.getTwoPhaseCommitArgs(tccResource, businessActionContext);
 Object ret;
 boolean result;
 // annotation  useTCCFence  Whether the property is set to  true
 if (Boolean.TRUE.equals(businessActionContext.getActionContext(Constants.USE_TCC_FENCE))) {
 try {
 result = TCCFenceHandler.commitFence(commitMethod, targetTCCBean, xid, branchId, args);
 } catch (SkipCallbackWrapperException | UndeclaredThrowableException e) {
 throw e.getCause();
 }
 } else {
 // Ellipsis logic
 }
 LOGGER.info("TCC resource commit result : {}, xid: {}, branchId: {}, resourceId: {}", result, xid, branchId, resourceId);
 return result ? BranchStatus.PhaseTwo_Committed : BranchStatus.PhaseTwo_CommitFailed_Retryable;
 } catch (Throwable t) {
 // Omit
 return BranchStatus.PhaseTwo_CommitFailed_Retryable;
 }
}


The above code can see , When executing the branch transaction commit method , First judgement  useTCCFence  Whether the property is  true, If  true, Then go  TCCFenceHandler  Class  commitFence  Logic , Otherwise, follow the normal submission logic .

TCCFenceHandler  Class  commitFence  Method is called  TCCFenceHandler  Class  commitFence  Method , The code is as follows ：

public static boolean commitFence(Method commitMethod, Object targetTCCBean,
 String xid, Long branchId, Object[] args) {
 return transactionTemplate.execute(status -> {
 try {
 Connection conn = DataSourceUtils.getConnection(dataSource);
 TCCFenceDO tccFenceDO = TCC_FENCE_DAO.queryTCCFenceDO(conn, xid, branchId);
 if (tccFenceDO == null) {
 throw new TCCFenceException(String.format("TCC fence record not exists, commit fence method failed. xid= %s, branchId= %s", xid, branchId),
 FrameworkErrorCode.RecordAlreadyExists);
 }
 if (TCCFenceConstant.STATUS_COMMITTED == tccFenceDO.getStatus()) {
 LOGGER.info("Branch transaction has already committed before. idempotency rejected. xid: {}, branchId: {}, status: {}", xid, branchId, tccFenceDO.getStatus());
 return true;
 }
 if (TCCFenceConstant.STATUS_ROLLBACKED == tccFenceDO.getStatus() || TCCFenceConstant.STATUS_SUSPENDED == tccFenceDO.getStatus()) {
 if (LOGGER.isWarnEnabled()) {
 LOGGER.warn("Branch transaction status is unexpected. xid: {}, branchId: {}, status: {}", xid, branchId, tccFenceDO.getStatus());
 }
 return false;
 }
 return updateStatusAndInvokeTargetMethod(conn, commitMethod, targetTCCBean, xid, branchId, TCCFenceConstant.STATUS_COMMITTED, status, args);
 } catch (Throwable t) {
 status.setRollbackOnly();
 throw new SkipCallbackWrapperException(t);
 }
 });
}


You can see that in the code , When committing a transaction, you will first judge  tcc_fence_log  Whether there are records in the table , If there's a record , Judge the transaction execution status and return . In this way, if it is judged that the transaction status is already  STATUS_COMMITTED, Will not submit again , It guarantees idempotency . If  tcc_fence_log  There are no records in the table , Then insert a record , For later retry .

Rollback  The logic of  commit  similar , Logical in class  TCCFenceHandler  Of  rollbackFence  Method .

Empty rollback

Here's the picture , The account service is a cluster of two nodes , stay  try  Stage account services  1  This node has failed ,try  Stage without considering retry , The global transaction must go to the end state , In this way, you need to execute on the account service once  cancel  operation , This idles a rollback operation .

Seata  Our solution is to  try  Stage   Go to  tcc_fence_log   Table inserts a record ,status  The field value is  STATUS_TRIED, stay  Rollback  Stage judge whether the record exists , If it doesn't exist , No rollback operation is performed . The code is as follows ：

//TCCFenceHandler  class
public static Object prepareFence(String xid, Long branchId, String actionName, Callback<Object> targetCallback) {
 return transactionTemplate.execute(status -> {
 try {
 Connection conn = DataSourceUtils.getConnection(dataSource);
 boolean result = insertTCCFenceLog(conn, xid, branchId, actionName, TCCFenceConstant.STATUS_TRIED);
 LOGGER.info(&quot;TCC fence prepare result: {}. xid: {}, branchId: {}&quot;, result, xid, branchId);
 if (result) {
 return targetCallback.execute();
 } else {
 throw new TCCFenceException(String.format(&quot;Insert tcc fence record error, prepare fence failed. xid= %s, branchId= %s&quot;, xid, branchId),
 FrameworkErrorCode.InsertRecordError);
 }
 } catch (TCCFenceException e) {
 // Omit
 } catch (Throwable t) {
 // Omit
 }
 });
}


stay  Rollback  The processing logic of the phase is as follows :

//TCCFenceHandler  class
public static boolean rollbackFence(Method rollbackMethod, Object targetTCCBean,
 String xid, Long branchId, Object[] args, String actionName) {
 return transactionTemplate.execute(status -> {
 try {
 Connection conn = DataSourceUtils.getConnection(dataSource);
 TCCFenceDO tccFenceDO = TCC_FENCE_DAO.queryTCCFenceDO(conn, xid, branchId);
 // non_rollback
 if (tccFenceDO == null) {
 // Do not execute rollback logic
 return true;
 } else {
 if (TCCFenceConstant.STATUS_ROLLBACKED == tccFenceDO.getStatus() || TCCFenceConstant.STATUS_SUSPENDED == tccFenceDO.getStatus()) {
 LOGGER.info("Branch transaction had already rollbacked before, idempotency rejected. xid: {}, branchId: {}, status: {}", xid, branchId, tccFenceDO.getStatus());
 return true;
 }
 if (TCCFenceConstant.STATUS_COMMITTED == tccFenceDO.getStatus()) {
 if (LOGGER.isWarnEnabled()) {
 LOGGER.warn("Branch transaction status is unexpected. xid: {}, branchId: {}, status: {}", xid, branchId, tccFenceDO.getStatus());
 }
 return false;
 }
 }
 return updateStatusAndInvokeTargetMethod(conn, rollbackMethod, targetTCCBean, xid, branchId, TCCFenceConstant.STATUS_ROLLBACKED, status, args);
 } catch (Throwable t) {
 status.setRollbackOnly();
 throw new SkipCallbackWrapperException(t);
 }
 });
}


updateStatusAndInvokeTargetMethod  Method execution  sql  as follows ：

update tcc_fence_log set status = ?, gmt_modified = ?
 where xid = ? and branch_id = ? and status = ? ;


It can be seen that  tcc_fence_log  It's recorded in the table   status   The field value is from  STATUS_TRIED  Change it to  STATUS_ROLLBACKED, If the update is successful , Execute rollback logic .

Hang

Hang up is because of network problems ,RM  I didn't receive it at first  try  Instructions , But it was carried out  Rollback  after  RM  I got it again  try  Command and reserve resources successfully , At this time, the global transaction has ended , Finally, the reserved resources cannot be released . Here's the picture ：

Seata  The solution to this problem is to execute  Rollback  Method first judge  tcc_fence_log  Whether there is a current  xid  The record of , If not, to  tcc_fence_log  Table inserts a record , Status is  STATUS_SUSPENDED, And no more rollback operations . The code is as follows ：

public static boolean rollbackFence(Method rollbackMethod, Object targetTCCBean,
 String xid, Long branchId, Object[] args, String actionName) {
 return transactionTemplate.execute(status -> {
 try {
 Connection conn = DataSourceUtils.getConnection(dataSource);
 TCCFenceDO tccFenceDO = TCC_FENCE_DAO.queryTCCFenceDO(conn, xid, branchId);
 // non_rollback
 if (tccFenceDO == null) {
 // Insert anti hang record
 boolean result = insertTCCFenceLog(conn, xid, branchId, actionName, TCCFenceConstant.STATUS_SUSPENDED);
 // Ellipsis logic
 return true;
 } else {
 // Ellipsis logic
 }
 return updateStatusAndInvokeTargetMethod(conn, rollbackMethod, targetTCCBean, xid, branchId, TCCFenceConstant.STATUS_ROLLBACKED, status, args);
 } catch (Throwable t) {
 // Ellipsis logic
 }
 });
}


And then execute  try  In the phase method, the first step is to  tcc_fence_log  Table inserts a current  xid  The record of , This causes primary key conflicts . The code is as follows ：

//TCCFenceHandler  class
public static Object prepareFence(String xid, Long branchId, String actionName, Callback<Object> targetCallback) {
 return transactionTemplate.execute(status -> {
 try {
 Connection conn = DataSourceUtils.getConnection(dataSource);
 boolean result = insertTCCFenceLog(conn, xid, branchId, actionName, TCCFenceConstant.STATUS_TRIED);
 // Ellipsis logic
 } catch (TCCFenceException e) {
 if (e.getErrcode() == FrameworkErrorCode.DuplicateKeyException) {
 LOGGER.error(&quot;Branch transaction has already rollbacked before,prepare fence failed. xid= {},branchId = {}&quot;, xid, branchId);
 addToLogCleanQueue(xid, branchId);
 }
 status.setRollbackOnly();
 throw new SkipCallbackWrapperException(e);
 } catch (Throwable t) {
 // Omit
 }
 });
}


Be careful ：queryTCCFenceDO  Method  sql  Used in  for update, So you don't have to worry  Rollback  Method cannot get  tcc_fence_log  Table records but cannot be judged  try  The execution result of the local transaction in the phase .

summary

TCC  Patterns are the most used patterns for distributed transactions , But idempotent 、 Hang and empty rollback have always been  TCC  Patterns need to be considered ,Seata  In the framework of  1.5.1  The version perfectly solves these problems .

Yes  tcc_fence_log  The operation of tables also needs to consider the control of transactions ,Seata  Proxy data sources are used , send  tcc_fence_log  Table operation and  RM  Business operations are performed in the same local transaction , In this way, local operation and  tcc_fence_log  The operation of both succeeded or failed .