Heterogeneous transaction scenario interaction process and consistency assurance

Questions as follows ：

A System online synchronous call B System （A and B Not the same company system , Distributed transactions cannot be used ）, How to ensure the quasi real-time consistency of data between systems （ Just talk about the design idea ）？

remind ： Call timeout needs to be considered 、 Concurrent 、 idempotent 、 Counter trade first come, etc .

How to handle various abnormal scenarios should be considered more perfectly , Such as transaction isolation 、 Concurrent 、 Anti transaction comes first when the caller and the service provider agree （ Front end customers can't wait all the time ）

This is a question of talking about ideas , They often ask a lot , Or rather bluff , In fact, this kind of problem , What we have to do is abstract . Abstract the scene , Abstract out the core of the problem . We can extract the main points ：

1. Not the same company system （ Cross network , isomerism ）;
2. Anti trading comes first （ We can basically confirm that the idea the questioner wants to know is related to the transaction ）;
3. Service provider reservation ;
4. Front end customers cannot wait （ The transaction process is often a long transaction process , You cannot simply rely on a single interface call , It's basically an asynchronous process ）.

With these premises , We can basically abstract the background and model of the discussion .

One 、 Model and background extraction

Because the premise is A、B The system belongs to different companies , in other words A B The system is two heterogeneous systems that interact through the public network , The technology stacks that are most likely to be implemented vary , Therefore, they can only interact with each other through exposed interfaces , We think it is through http Interface .

Because it is A system call B System , Therefore, we can abstract it into a point-to-point message communication scenario , among A It is the active puller ,B Is the pulled party .

The question mentioned that , We say that distributed transactions cannot be used , In fact, it means that strong consistency cannot be used / class 2PC The transaction implementation of , Such as 2PC、3PC、SEATA etc. , However, a flexible method such as best effort notification can be used for data synchronization .

With a plan , We then abstract out Transaction list This model , And specify a state machine for it ：

1. When a transaction order is created , Status as   initialization ,
2. A System to B When the system sends a transaction order, it changes to ,  In processing
3. If B The system failed to respond to the receipt synchronously , be A The system modification status is   Failure
4. Sync B The system synchronously responds to the receipt successfully , be A The system modification status is   The bill of lading has been
5. When B The system processes transaction completion , notice A The system transaction is completed , be A The system modifies the transaction order status to   success ,（ Success here is real success , Because there has been a capital deduction / Consumption of subject matter such as integral ）
6. When B The system failed to process the transaction , Notice A System transaction failed , be A The system modifies the transaction order status to   Failure  （ The failure at this point is assumed to be B The system failed before deducting money ）

Two 、 Timeout processing method

Let's discuss how to deal with the overtime of the bill of lading request .

A The exit gateway of the system goes to B The entry gateway of the system initiates the bill of lading request , This is a synchronous communication , For the failure of synchronization request （ Such as ： Signature failed , Parameter exception failure, etc ）,A The system can initiate a retry , here This request is a real failure , Because there is no transaction at all , So in principle, the data are consistent , For funds, there is no deduction .

3、 ... and 、 Drop order query  

If A System submission bill of lading request timeout , Failed to receive at this time B Synchronization of system replies Receipt succeeded Response , At this time, the data is inconsistent . This scene is called Drop the list , be A The system needs to check the missing data （ The status is processing ） Initiate a drop order query , The idea is to initiate a query on a regular basis , obtain B Processing of transaction orders by the system .

generally speaking B The system will notify A The suggested time for the system to initiate the drop order query , Such as initiating a transaction order 10 The final result of processing will be available in minutes , that A The system can check the submitted 10 Minutes or more , Status as In processing Initiate a drop order query . in other words ,10 Minutes later , This kind of intermediate state data ,AB An agreement can be reached between systems .

Special , If a lost order query fails to find a definite result , You can set the next query , It is recommended to use       Time decay strategy The query , This is a common idea for timing synchronization of unknown data commonly used in transaction scenarios and even middleware .

Four 、 Result notification  

For trading scenarios , Considering the real-time performance of data , We often hope that the downstream system can timely inform us of the results after processing the data .

In this scenario is A The system needs docking B Systematic Real time transaction result notification interface , When a trade order is B After the system processing ,B The system will send a notice to the transaction order processing result , Call back in time A Results of system processing . here ,A The system becomes the callee ,B The system is the caller , Compared with the drop order query , The result notification is almost quasi real time , from B The system sends a notification to A The notification received by the system is usually at the level of 100 milliseconds （ Alipay payment result notification can reach tens of milliseconds ）.

In a brief summary , For timeout processing , That's what we did Drop order query and Real time notification The way , By actively polling the processing results and passively accepting the notification of the results , adopt Push pull combination In such a way as to jointly guarantee AB Data between systems are finally agreed .

5、 ... and 、 Concurrent bill of lading  

For concurrent bills of lading , In fact, it is a common topic . Its essence lies in Anti replay processing idea for requests in distributed scenarios .

The core is request serialization , We tend to pass CAS、 Lock and other methods .

Specific to specific implementation details ,CAS The mode has database status identification （ State machine ）、 The locking method is actually a distributed lock , Simply put, it is handled through distributed locking , By adding a distributed lock to a transaction , The request can only be initiated after the distributed lock is successfully obtained , Write idempotent records after the request is initiated , Release the lock after completing the request , Prevent concurrent submissions .

6、 ... and 、 Idempotent train of thought

For idempotent , We usually need to use idempotent check to , such as ：

High concurrency scenarios , Write idempotent ID to Redis cache , Used to idempotent write requests perhaps If the request is small , Then idempotent processing is performed through the unique constraint of the database , Ensure that only one transaction order is deposited （ Such as unique constraint The order number ）.

The only constraint is the last line of defense , Used to idempotent write requests In low concurrency scenarios , By first querying , Insert after （ to update ） Idempotent check can also be performed in the way of , But there will be repeated updates in high concurrency scenarios / New risks , Therefore, it is often necessary to cooperate with distributed locks , Serialize concurrent requests In terms of idempotence alone , Natural idempotent query , Update can be idempotent guaranteed in the above way .

7、 ... and 、 Anti trading comes first

First, define what is “ Anti trading ”, Anti trading , seeing the name of a thing one thinks of its function , Reverse trading , Let's give an example to understand .

for instance , Anti transaction of deduction , Is to correct （ for instance , Transfer operation , deduction A The money , to B Adding money failed . be A The deducted money needs to be made up , This process is called correction . The actual reversal covers a wider range , We just need to consider it as the reverse operation of deduction , But it should be different from cash withdrawal and recharge .）

for instance ,A The system requests payment for the transaction order 100 element ,B After the system deducts money successfully, it will send it to A The system returns the notification message of successful payment ; here B System subsequent operation failure , As a result, the transaction cannot continue , be B System pair A Deducted by the system 100 After the yuan is reversed , notice A The system transaction has reversed the chargeback .

The so-called anti trading comes first , That is to say, the network is congested , The message that causes the reversal of the chargeback , The message of successful payment comes first .

our A There is a state machine in the transaction order of the system , The state machine is the first to arrive tool for dealing with anti transaction .

We require that transactions be processed serially , How to serial , In fact, to put it simply , It can be well realized through the state machine .

Of course, the point to be made is , For the actual situation , Need specific business specific analysis , For the scenario we are currently discussing , We can solve the problem through the state machine , The specific process is as follows ,

We assume that ,A The state machine of the transaction order of the system can only follow In processing -> Successful payment -> Chargeback This process flows , When chargeback arrives successfully before payment , We need to insert the pipeline in a transaction at the same time , Update of transaction order status . For update operations , our sql The expectations are as follows ：

update order set order_status= Chargeback  where order_status= Successful payment  and order_id=xxxxx

Because the state machine only allows fixed order state migration , We update the status with the old status , In fact, the current order_status = In processing , therefore update Failure . The final processing is notification processing failure ,A The system informs B The system resends the notification . that B The system can only honestly restart the notification , This is what a qualified trading system must have . Otherwise, your system does not support repeated notification , The user experience is too bad .

At this time, due to the state machine , Transaction order status is still in process , When the notification of the success of the congested payment arrives , The transaction order status was successfully updated to Successful payment At this time update Statement for ：

update order set order_status= Successful payment  where order_status= In processing  and order_id=xxxxx

Subsequent retries Chargeback notice （ The so-called anti trading ） After arrival , The transaction order can be successfully circulated according to the state machine , Concrete update The statement is as follows ：

update order set order_status= Chargeback  where order_status= Successful payment  and order_id=xxxxx

From our analysis we can see that , As long as there is a state machine , in any case , The status of a transaction order can only be determined by In processing -> Successful payment -> Chargeback This method flows without state transition .

So we say , State machine , It is the first to arrive tool for anti transaction processing between systems , The state machine is also a powerful weapon commonly used in transaction systems .

8、 ... and 、 Front end users can't wait for processing ideas

Another point is that front-end customers cannot wait all the time , In fact, we have already solved this problem in the above process .

The overall scheme we adopted for this analysis is based on Best effort notification The idea of , The core step is Synchronous bill of lading , Drop order query , Result notification . By combining these steps , We can avoid the front-end waiting , Because the transaction belongs to a long transaction business , The upstream / The front end only needs to submit successfully to do other things , The remaining complex operations are handled slowly by the downstream system , This actually reflects asynchronous thinking .

Nine 、 Ensure data consistency through reconciliation

Last but not least , For trading systems , Reconciliation mechanism is the best solution to ensure data consistency , About reconciliation , I will also write a separate article for detailed discussion in the near future （ another flag）.

Trading systems usually have t+1 Reconciliation of , Simply put , Generate the statement of the previous day every day , In our scenario ,A The system reports to B The system requests the transaction statement of the previous day , Download to local , adopt A The system's own channel serial number / Transaction number , And B The transaction orders provided by the system are reconciled item by item , This process can often be automatically executed through timed tasks , Balance the inconsistent trading orders . So that the data between the two systems can be finally agreed , for instance ,A The system did not receive B System notification , The transaction orders that can not be found by the drop order query , Often the final status of data can be obtained through reconciliation .