Microservice system design -- distributed transaction service design

Abstract

Some in the system involve multiple database write operations during multi service invocation , It may involve data reading and writing and multi service data sharing ……, Usually spring Pass through @Transactional Annotation for transaction control , Data integrity has not been guaranteed in the service , Data integrity cannot be protected after cross service . This kind of problem can be solved by using distributed transactions , This blog post uses Seata Component to ensure data integrity in cross service scenarios .

One 、 Problem scenario

Take a key scene to pave the way for the theme . After the vehicle leaves the site after paying the fee , The main business logic is as follows ：

• The billing service is self-directed , Write departure information
• Invoke financial services , Write charging information
• Call message service , Write message record

It involves collaboration between three services , Data is written to each of the three repositories , Is a typical distributed transaction data consistency problem . Let's take a look at the code logic of the normal scenario ：

@Service
@Slf4j
public class ExistsServiceImpl implements ExistsService {

    @Autowired
    ExistsMapper ExistsMapper;

    @Autowired
    EntranceMapper entranceMapper;

    @Autowired
    RedisService redisService;

    @Autowired
    BillFeignClient billFeignClient;

    @Autowired
    MessageClient messageClient;

    @Autowired
    Source source;

    @Override
    @Transactional(rollbackFor = Exception.class)
    public int createExsits(String json) throws BusinessException {
        log.info("Exists data = " + json);
        Exists exists = JSONObject.parseObject(json, Exists.class);
        int rtn = ExistsMapper.insertSelective(exists);
        log.info("insert into park_charge.Exists data suc !");

        // Calculate parking fees 
        EntranceExample entranceExample = new EntranceExample();
        entranceExample.setOrderByClause("create_date desc limit 0,1");
        entranceExample.createCriteria().andPlateNoEqualTo(exists.getPlateNo());
        List<Entrance> entrances = entranceMapper.selectByExample(entranceExample);
        Entrance lastEntrance = null;
        if (CollectionUtils.isNotEmpty(entrances)) {
            lastEntrance = entrances.get(0);
        }
        if (null == lastEntrance) {
            throw new BusinessException(" Abnormal vehicle , Admission data not found !");
        }
        Instant entryTime = lastEntrance.getCreateDate().toInstant();
        Duration duration = Duration.between(LocalDateTime.ofInstant(entryTime, ZoneId.systemDefault()),
                LocalDateTime.now());
        long mintues = duration.toMinutes();
        float fee = caluateFee(mintues);
        log.info("calu parking fee = " + fee);

        // call   Third party payment services , Pay for parking , Omit here . Write payment records directly 
        Billing billing = new Billing();
        billing.setFee(fee);
        billing.setDuration(Float.valueOf(mintues));
        billing.setPlateNo(exists.getPlateNo());
        CommonResult<Integer> createRtn = billFeignClient.create(JSONObject.toJSONString(billing));
        if (createRtn.getRespCode() > 0) {
            log.info("insert into billing suc!");
        }else {
            throw new BusinessException("invoke finance service fallback...");
        }

        // Update offsite screen , Refresh the number of available parking spaces 
        redisService.increase(ParkingConstant.cache.currentAviableStallAmt);
        log.info("update parkingLot aviable stall amt = " +redisService.getkey(ParkingConstant.cache.currentAviableStallAmt));
        // Send a payment message 
        Message message = new Message();
        message.setMcontent("this is simple pay message.");
        message.setMtype("pay");
        source.output().send(MessageBuilder.withPayload(JSONObject.toJSONString(message)).build());
        log.info("produce msg to apache rocketmq , parking-messge to consume the msg as a consumer...");

        // Write payment message record 
        CommonResult<Integer> msgRtn = messageClient.sendNotice(JSONObject.toJSONString(message));
        if (msgRtn.getRespCode() > 0) {
            log.info("insert into park_message.message data suc!");
        }else {
            throw new BusinessException("invoke message service fallback ...");
        }

        return rtn;
    }

    /**
     * @param stayMintues
     * @return
     */
    private float caluateFee(long stayMintues) {
        String ruleStr = (String) redisService.getkey(ParkingConstant.cache.chargingRule);
        JSONArray array = JSONObject.parseArray(ruleStr);
        List<ChargingRule> rules = JSONObject.parseArray(array.toJSONString(), ChargingRule.class);
        float fee = 0;
        for (ChargingRule chargingRule : rules) {
            if (chargingRule.getStart() <= stayMintues && chargingRule.getEnd() > stayMintues) {
                fee = chargingRule.getFee();
                break;
            }
        }
        return fee;
    }

}

Under normal circumstances , No problem , Once the sub service has write exception logic , There will be data inconsistencies . For example, the vehicle departure record is written successfully , But the payment record fails to be written , The code cannot rollback the error data in time , Cause incomplete business data , Hindsight is difficult .

Two 、 Distributed transaction problems

What is business , A transaction is a reliable, independent unit of work consisting of a set of operations , All or nothing , All or nothing , Partial success and partial failure are not allowed . Under the single architecture , More local transactions , For example, using Spring Frame words , It's basically from Spring To manage affairs , Ensure the normal logic of transactions . But local transactions are limited to the current application , The transactions of other applications are out of reach .

What is distributed transaction , A large business operation involves many small operations , Various small operations are scattered in different applications , Ensure the integrity and reliability of business data . It's also either totally successful , Or it's a total failure . The scope of transaction management has evolved from a single application to a distributed system .

There is a lot of discussion about distributed transactions in the network , Mature solutions also exist , There will not be too much discussion here , Interested partners can supplement this knowledge first , Let's go back to this article .

When data consistency is not high , The industry generally advocates the adoption of final consistency , To ensure the data integrity involved in distributed transactions . This article will focus on Seata Schemes fall into this category .

3、 ... and 、Seata Solution

Seata Is an open source distributed transaction solution , We are committed to providing high-performance and easy-to-use distributed transaction services under the microservice architecture . Support Dubbo、Spring Cloud、grpc etc. RPC frame , This introduction is also related to Spring Cloud The reason why the system integration is better . For more details, please refer to its official website ：

Seata There are three important concepts in ：

• TC—— A business coordinator ： Maintain the state of global and branch transactions , Drive global transaction commit or rollback , Independent of each application .
• TM—— Transaction manager ： Define the scope of the global transaction ： Start global transaction 、 Commit or roll back global transactions , That is, the initiator of the transaction .
• RM—— Explorer ： Manage resources for branch transactions , And TC Talk to register branch transactions and report the status of branch transactions , And drive branch transaction commit or rollback ,RM It should be maintained in various microservices .

3.1 Seata Server install

This case is based on AT Module deployment , Need to combine MySQL、Nacos Joint completion .

 When the download is complete , Get into  seata  Catalog ：

drwxr-xr-x   3 apple  staff    96 10 16 15:38 META-INF/
-rw-r--r--   1 apple  staff  1439 10 16 15:38 db_store.sql
[email protected]  1 apple  staff   829 12 18 11:40 db_undo_log.sql
-rw-r--r--   1 apple  staff  3484 12 19 09:41 file.conf
-rw-r--r--   1 apple  staff  2144 10 16 15:38 logback.xml
-rw-r--r--   1 apple  staff   892 10 16 15:38 nacos-config.py
-rw-r--r--   1 apple  staff   678 10 16 15:38 nacos-config.sh
-rw-r--r--   1 apple  staff  2275 10 16 15:38 nacos-config.txt
-rw-r--r--   1 apple  staff  1359 12 19 09:41 registry.conf

Transaction registration support file、nacos、eureka、redis、zk、consul、etcd3、sofa Multiple modes , Configuration also supports file、nacos、apollo、zk、consul、etcd3 And so on , This use nacos Pattern , modify registry.conf After the following .

appledeMacBook-Air:conf apple$ cat registry.conf 
registry {
  # file、nacos、eureka、redis、zk、consul、etcd3、sofa
  type = "nacos"

  nacos {
    serverAddr = "localhost:8848"
    namespace = ""
    cluster = "default"
  }
}

config {
  # file、nacos、apollo、zk、consul、etcd3
  type = "nacos"

    nacos {
    serverAddr = "localhost"
    namespace = ""
    cluster = "default"
  }
}

Transaction registration is selected nacos after , It needs to be used nacos-config.txt The configuration file , Open the file to modify key configuration items —— Transaction group and storage configuration items ：

service.vgroup_mapping.${your-service-gruop}=default

In the middle of the ${your-service-gruop} The name of the service group defined for yourself , In service application.properties Configure the service group name in the file . How many sub services involve global transaction control , You need to configure how many .

service.vgroup_mapping.message-service-group=default
service.vgroup_mapping.finance-service-group=default
service.vgroup_mapping.charging-service-group=default

...
store.mode=db
store.db.url=jdbc:mysql://127.0.0.1:3306/seata-server?useUnicode=true
store.db.user=root
store.db.password=root

initialization seata-server database , Three tables are involved ：branch_table、global_table and lock_table, Used to store global transactions 、 Branch transaction and lock table related data , The script is located in the conf Under the table of contents db_store.sql In file .

SET NAMES utf8mb4;
SET FOREIGN_KEY_CHECKS = 0;

-- ----------------------------
-- Table structure for branch_table
-- ----------------------------
DROP TABLE IF EXISTS `branch_table`;
CREATE TABLE `branch_table` (
  `branch_id` bigint(20) NOT NULL,
  `xid` varchar(128) NOT NULL,
  `transaction_id` bigint(20) DEFAULT NULL,
  `resource_group_id` varchar(32) DEFAULT NULL,
  `resource_id` varchar(256) DEFAULT NULL,
  `lock_key` varchar(128) DEFAULT NULL,
  `branch_type` varchar(8) DEFAULT NULL,
  `status` tinyint(4) DEFAULT NULL,
  `client_id` varchar(64) DEFAULT NULL,
  `application_data` varchar(2000) DEFAULT NULL,
  `gmt_create` datetime DEFAULT NULL,
  `gmt_modified` datetime DEFAULT NULL,
  PRIMARY KEY (`branch_id`),
  KEY `idx_xid` (`xid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

-- ----------------------------
-- Table structure for global_table
-- ----------------------------
DROP TABLE IF EXISTS `global_table`;
CREATE TABLE `global_table` (
  `xid` varchar(128) NOT NULL,
  `transaction_id` bigint(20) DEFAULT NULL,
  `status` tinyint(4) NOT NULL,
  `application_id` varchar(32) DEFAULT NULL,
  `transaction_service_group` varchar(32) DEFAULT NULL,
  `transaction_name` varchar(128) DEFAULT NULL,
  `timeout` int(11) DEFAULT NULL,
  `begin_time` bigint(20) DEFAULT NULL,
  `application_data` varchar(2000) DEFAULT NULL,
  `gmt_create` datetime DEFAULT NULL,
  `gmt_modified` datetime DEFAULT NULL,
  PRIMARY KEY (`xid`),
  KEY `idx_gmt_modified_status` (`gmt_modified`,`status`),
  KEY `idx_transaction_id` (`transaction_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

-- ----------------------------
-- Table structure for lock_table
-- ----------------------------
DROP TABLE IF EXISTS `lock_table`;
CREATE TABLE `lock_table` (
  `row_key` varchar(128) NOT NULL,
  `xid` varchar(96) DEFAULT NULL,
  `transaction_id` mediumtext,
  `branch_id` mediumtext,
  `resource_id` varchar(256) DEFAULT NULL,
  `table_name` varchar(32) DEFAULT NULL,
  `pk` varchar(36) DEFAULT NULL,
  `gmt_create` datetime DEFAULT NULL,
  `gmt_modified` datetime DEFAULT NULL,
  PRIMARY KEY (`row_key`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

SET FOREIGN_KEY_CHECKS = 1;

  After the table structure initialization is completed , You can start it seata-server：

#192.168.31.101  Local area network  ip

#  initialization  seata  Of  nacos  To configure 
cd seata/conf
sh nacos-config.sh 192.168.31.101

#  start-up  seata-server, Port conflict is required , Here it is adjusted to  8091
cd seata/bin
nohup sh seata-server.sh -h 192.168.31.101 -p 8091 -m db &

3.2 In service Seata To configure

Each independent business library , Need to be undo_log Data table support , In order to roll back when an exception occurs . In the member library , The following scripts are executed in the financial library and the message library respectively , write in un_log surface .

CREATE TABLE `undo_log`
(
  `id`            BIGINT(20)   NOT NULL AUTO_INCREMENT,
  `branch_id`     BIGINT(20)   NOT NULL,
  `xid`           VARCHAR(100) NOT NULL,
  `context`       VARCHAR(128) NOT NULL,
  `rollback_info` LONGBLOB     NOT NULL,
  `log_status`    INT(11)      NOT NULL,
  `log_created`   DATETIME     NOT NULL,
  `log_modified`  DATETIME     NOT NULL,
  `ext`           VARCHAR(100) DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `ux_undo_log` (`xid`, `branch_id`)
) ENGINE = InnoDB
  AUTO_INCREMENT = 1
  DEFAULT CHARSET = utf8;

In the service of each module pom.xml Add to file seata relevant jar Support ：

<!-- seata-->
<dependency>
     <groupId>com.alibaba.cloud</groupId>
     <artifactId>spring-cloud-alibaba-seata</artifactId>
</dependency>
<dependency>
     <groupId>io.seata</groupId>
     <artifactId>seata-all</artifactId>
</dependency>

jar After the package is introduced , Corresponding to the module service application.properties add seata Related configuration items ：

#  To communicate with the server  nacos-config.txt  In profile  service.vgroup_mapping  The suffix of corresponds to 
spring.cloud.alibaba.seata.tx-service-group=message-service-group
#spring.cloud.alibaba.seata.tx-service-group=finance-service-group
#spring.cloud.alibaba.seata.tx-service-group=charging-service-group
logging.level.io.seata = debug

#macbook pro  Low configuration ,server  The time configuration shall be appropriately reduced 
#hystrix  After a specified time , It's automatic  fallback  Handle 
hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds=20000
feign.client.config.defalut.connectTimeout=5000
#feign  It's through  ribbon  Complete client load balancing , We need to configure  ribbon  Connection timeout for , If timeout occurs, it will automatically  fallback
ribbon.ConnectTimeout=6000

application.properties Under the same category , basis Spring Boot The principle of agreement over configuration , increase registry.conf file , This file will be loaded by default when the application starts , The default file name has been written in the code , Here's the picture ：

3.3 Data source agent configuration

To validate a global transaction , For the corresponding repository of each microservice , Must be Seata Data source proxy , For unified management , The configuration code is as follows , Write the following code files to all relevant microservice modules , Automatic configuration at service startup .

@Configuration
public class DataSourceProxyConfig {

    @Value("${mybatis.mapper-locations}")
    private String mapperLocations;

    @Bean
    @ConfigurationProperties(prefix = "spring.datasource")
    public DataSource hikariDataSource(){
      //spring boot  The default integration is  Hikari  data source , If you want to change to  driud  The way , Can be in  spring.datasource.type  It is specified in 
        return new HikariDataSource();
    }

    @Bean
    public DataSourceProxy dataSourceProxy(DataSource dataSource) {
        return new DataSourceProxy(dataSource);
    }

    @Bean
    public SqlSessionFactory sqlSessionFactoryBean(DataSourceProxy dataSourceProxy) throws Exception {
        SqlSessionFactoryBean sqlSessionFactoryBean = new SqlSessionFactoryBean();
        sqlSessionFactoryBean.setDataSource(dataSourceProxy);
        sqlSessionFactoryBean.setMapperLocations(new PathMatchingResourcePatternResolver()
                .getResources(mapperLocations));
        sqlSessionFactoryBean.setTransactionFactory(new SpringManagedTransactionFactory());
        return sqlSessionFactoryBean.getObject();
    }

}

Remember the main business logic codes at the beginning of this chapter ？ Method except local transaction annotation @Transactional Outside , There needs to be more Seata Global transaction configuration annotation ：

@Transactional(rollbackFor = Exception.class)
@GlobalTransactional
public int createExsits(String json) throws BusinessException { }

 thus ,Seata Server、 Global transaction configuration 、 Transaction rollback configuration 、 Data source agent 、 Code support has been completed , Let's start the application , See what's different ：

2020-01-09 09:22:19.179  INFO 16457 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2020-01-09 09:22:19.970  INFO 16457 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2020-01-09 09:22:20.053  INFO 16457 --- [           main] io.seata.core.rpc.netty.RmRpcClient      : register to RM resourceId:jdbc:mysql://localhost:3306/park_charge
2020-01-09 09:22:20.053  INFO 16457 --- [           main] io.seata.core.rpc.netty.RmRpcClient      : register resource, resourceId:jdbc:mysql://localhost:3306/park_charge
2020-01-09 09:22:20.060 DEBUG 16457 --- [lector_RMROLE_1] i.s.core.rpc.netty.AbstractRpcRemoting   : [email protected] msgId:2, future :[email protected], body:version=0.9.0,extraData=null,identified=true,resultCode=null,msg=null

From 3 Line log start , You can see that the corresponding application has submitted itself to global transaction control . Is this distributed transaction really useful ？ Let's do a test together , See if the integrity of the data can be guaranteed .

3.4 Distributed transaction testing

Abnormal condition test ： Start only parking-charge Billing services , The other two services （ Financial sub service and message sub service ） Do not start , When calling finance-service when , Service not available ,hystrix Will fail directly and quickly , Throw an exception , At this point, the global transaction fails , The exit record successfully written just now is rolled back and cleared , Take a look at the following screenshot of the Key log output ：

Normal test ：

Start all three microservice instances , Can be in nacos The console sees three normal service instances , adopt swagger-ui or PostMan Make a request call , Pay special attention to the console output of the next three sub services ：

parkging-charging Billing service instance , As the business initiator , Open global transaction , The global transaction number displayed is 192.168.31.101:8091:2032205087, The callee transaction number should be the same .

 parking-finance Financial sub service console log output , You can see that the service is running normally , The global transaction number is 192.168.31.101:8091:2032205087.

parking-message Console log output of the message sub service , The global transaction number is consistent with the previous two services . After the above two positive and negative tests , You can see that the distributed transaction configuration is running normally . Careful friends found ,seata No data exists in the supported tables of , What's going on here ？ When will there be data , Let's think about it , It's the next question for everyone to think about .

Blog reference