How comfortable it is to use Taijiquan to talk about distributed theory!

It's very comfortable to watch and listen , Life is just a few autumn ...

In the story of relying on heaven to kill dragons Zhao Min The princess took a group of experts to besiege Wudang , Wudang sect leader Zhang Sanfeng Be plotted against , Passed a set of martial arts to zhang wuji To deal with Zhao Min's men . This set of martial arts is Taiji boxing .

*

Zhang Sanfeng ： No taboo , How much I taught you to remember ？

zhang wuji ： I totally forgot ！

Zhang Sanfeng ： very good , You just need to remember to beat the two old men down .

Part 1 use War Within Three Kingdoms Talking about Byzantine Generals in distribution , It's interesting , This time we use The Heaven Sword and the Dragon Sabre Medium Taiji boxing Let's talk about the rest Three theories ：

• CAP theory
• ACID theory
• BASE theory

*

The essence of Taijiquan ： killing with kindness , Hard and soft go hand in hand , Four or two thousand catties , Nothing beats nothing .

I put CAP The theory is called Tai Chi ,ACID The theory is called Yang or Just. ,BASE The theory is called Yin or The soft .ACID Theory pursues consistency ,BASE The theory is called flexible transaction , It's about usability . Why did Zhang Wuji forget all about it and beat xuanming Er Lao ？ Because the essence of Taijiquan is the meaning of boxing , Nothing beats nothing .

1、 Two sides of Taiji

CAP The theory is a high abstraction of the characteristics of distributed system , It's three indicators ：

• Uniformity （Consistency）
• Usability （Availability）
• Partition tolerance （Partition Tolerance）

Consistency in distribution , We can understand it as every time the client Read operations , No matter which point is visited , Or you can read the same newly written data , Either the read failed . It's very rigid , It can't be said that Just. Not good. , In many scenarios , And there really needs to be a high degree of consistency .

To help you understand consistency , Let me give you an example The Heaven Sword and the Dragon Sabre The story of ： The six factions besieged Guangming summit . Emei Abbess extinction as the leader , Lead the six major factions in the world to attack Bright Summit , The initial attack strategy is from north attack . Nun annihilation found it bad to attack from the North , So the pigeon sent a message to Wudang sect and Shaolin school from south Orders to attack , however Shaolin school Wei Yixiao, the green winged bat king with the highest lightness skill of the Ming Dynasty to intercept 了 , The end result is Shaolin school from north attack ,‍‍‍ Wudang sect from south attack , Isn't that a mess ？ As shown in the figure below ：

around Attack the light The top

1.1 Understand... In distributed computing CAP

CAP How to understand it in a distributed system ？ Here's an example to help you understand .

• Initial environment ： The client queries or updates the node 1 and node 2, The value of two nodes A = 1.

Initial environment

• The client updates the node 1 in A Value , Set up A = 5.

The client updates the node 1

• node 1 take A The value of is updated to 5 after , The update is successfully returned to the client .

  

  node 1 Return to update successful

• The client accessed the node 2 , The request for A Value , The result returned to A = 1. It's about nodes 1 Stored in the A The values of are inconsistent .

The client accesses the node 2

• So how to ensure that the values in both nodes are A = 5 Well ？ The client will send the node 1 After the update , node 2 It also needs to be updated , To tell the client that the update was successful .

node 2 It also needs to be updated

• After both nodes are successfully updated , When a client accesses any of these nodes, it gets A = 5. This is called consistency .

After both nodes are updated

Uniformity The emphasis is that the data is correct , Every time the data in the read node is the latest written data . This is what I call Just. .

But in our cluster environment, if partition failure occurs （ Node lost connection , Node failed to respond , Node cannot write data ）, When the client queries the node , We can't return the error message to the client . For example, some key systems in a business cluster , Such as registration center , It can't be because a node is disconnected , Don't respond to the latest data . Then the relevant business can not get the correct registration information, resulting in system paralysis .

Usability That comes in handy , Sacrifice data accuracy , Each node uses local data to respond to client requests . In addition, when the node is not available , You can use the quick fail strategy , At least don't let the service fail to respond for a long time . Usability emphasizes the availability of services , There is no guarantee that the data is correct . This is what I call The soft .

As shown in the figure below ： node 1 And nodes 2 The values returned to the client are A = 5 and A = 1, That is, nodes 1 and node 2 There's no guarantee of data consistency , It's about the availability of the nodes .

Data inconsistency

Partition tolerance When any number of messages are lost or high latency occurs between nodes , The system is still working . The distributed system tells the client , No matter what kind of data synchronization problems appear inside me , I'll keep running . The emphasis is on the fault tolerance of cluster heap partition failure .

1.2 CAP Triangle

So what's the relationship between these three indicators ？ This is what we often hear CAP theory .C For consistency （Consistency）,A For usability （Availability）、P Represents partition fault tolerance （Partition Tolerance）.

For distributed systems ,CAP Only two of the three indicators can be selected .

• CA： Ensure consistency and availability . When the distributed system is running normally （ Most of the time in a state of ）, It's not necessary at this time P, that C and A Can guarantee at the same time . Only in case of partition failure , That's what we need P, At this time, it can only be C and A Make a choice between . Typical applications ： Stand alone deployment MySQL.

• CP： Ensure data consistency and partition fault tolerance , Like configuration information , We must make sure that every node is up-to-date , Correct data . such as Raft The strong consistency system of , Can cause read and write operations to fail . Typical applications ：Etcd、Consul、Hbase.

• AP： Ensure the availability and fault tolerance of distributed system . User access system , You can get the data , There are no response errors , But it's possible to read old data . Typical applications ：Cassandra and DynamoDB.

2、 The rigidity of Taiji

2.1 ACID I just

At first I knew ACID It's research SQL When it comes to databases , Atomicity （Atomicity）、 Uniformity （Consistency）、 Isolation, （Isolation）、 persistence （Durability）.

These four attributes are aimed at Business In terms of the , A transaction is a series of operations performed for a single unit of work . Such as query 、 Modifying data 、 Modify the data definition .

Transactions are not just on databases , It can also be used in business systems , For example, after issuing the voucher, the inventory will be deducted , This kind of business scenario can be defined as a transaction . In stand-alone scenario, we can lock 、 Time series and other mechanisms to ensure that a single node ACID characteristic , But it can't guarantee the operation between nodes ACID characteristic .

So how to solve the transaction problem in distributed system ？ This is also a common question in an interview . You must have heard of distributed transaction protocol , such as Two stage submission agreement and TCC agreement , Next I'll use The six factions besieged Guangming summit The story is about the two-stage agreement .

2.2 Besiege Guangming Summit

Emei Want to gather Shaolin school 、 Wudang sect 、 Kunlun sect Attack together tomorrow Bright Summit . If one side doesn't agree to attack , Or the timing of the attack is inconsistent , You need to cancel the whole action plan . Shaolin school 、 Wudang sect 、 The Kunlun attack on Guangmingding can be regarded as A distributed transaction , Or all 、 Or none at all . As shown in the figure below ：

How to help abbess extinction solve this collaborative problem ？ We can use the two-phase commit protocol to illustrate .

2.3 Two stage submission agreement

In the two-stage submission agreement , Abbess exterminate first sent the message of attack to Shaolin , Shaolin as a coordinator , Shaolin school should contact Wudang school and Kunlun school to attack or retreat .

The second stage means that there are two stages ,1. Submit request phase （ Voting stage ）,2. Submission execution phase （ Completion stage ）.

Stage 1 ： Submit request phase ：

• First step ： As a coordinator, Shaolin sent messages to Wudang and Kunlun respectively ： “ Attack Guangming top tomorrow , feasible ？”
• The second step ： Shaolin school 、 Wudang sect 、 The Kunlun faction evaluated whether they could attack Guangming top tomorrow , If you can , Just set aside time and lock in , No more offense .
• The third step ： Shaolin got all the answers , Including Shaolin's own assessment results . The result of the last three parties is feasible .

As shown in the figure below ：

Stage 1

Stage two ： Submission execution phase ：

• First step ： Shaolin people count themselves 、 News from Kunlun and Wudang schools , All are Can attack , So you can execute distributed transactions , Attack Guangming top .
• The second step ： Shaolin tells Kunlun and Wudang to attack Guangming top .
• The third step ： Shaolin school 、 Kunlun sect 、 The Wudang sect gathered its disciples , Attack Guangming top （ Perform transactions ）.
• Step four ： Kunlun sect 、 Wudang will Whether an attack has been launched Tell Shaolin .
• Step five ： The Shaolin group gathered themselves 、 Kunlun sect 、 The result of Wudang's attack is to annihilate abbess . What the Abbess sees in this way is a unified operational plan .

Stage two

Be careful ：

• You can use abbess extinction as a client . Shaolin school 、 Wudang sect 、 Kunlun Paidang is the three nodes of the distributed system . Shaolin as a coordinator .
• The evaluation of whether it can attack the light top and the reserved time can be understood as the object to be operated and the state of the object , Are you ready for , Can I submit a new operation .
• Send a message 、 Flying pigeons can be understood as network messages .
• In the first stage , Or to submit each participant to vote , Once a vote has been submitted for business , Then it is not allowed to abandon the transaction .
• In the second stage , Each participant carries out the final unified decision , Commit a transaction or abandon a transaction . This is ACID The atomicity of .
• In the first stage , Resources need to be reserved , The reservation period , No one else can manipulate this resource .

2.4 The problems of the two-stage agreement

ACID Characteristic is CAP Consistency in The border , It can be called the strongest consistency , If consistency is implemented in a distributed system , It is bound to affect Usability . If a node fails , The execution of this distributed transaction fails .

In most scenes , The requirement for consistency is not that high , There is no need to guarantee strong consistency , Temporary inconsistencies can also be accepted , Finally, to ensure that the data is correct, just OK. That is to say, we can use Final consistency Scheme to ensure the consistency of data .

Another thing to mention is TCC agreement （ Three stage submission agreement ）, He's aiming at the second phase submission ： Coordinator failure , Participants lock in resources for a long time Pain points And out of the agreement . Query phase and timeout mechanism are introduced , Reduce resource lock-in for a long time . But more information is needed to negotiate , Increased system load and response latency , So the three-phase commit protocol is rarely used .

3、 The softness of Taiji

3.1 BASE The softness of

I talked about the rigidity of Taiji , Let's talk about the softness of Taiji . When it comes to distributed transactions , It must be mentioned BASE theory , Be commonly called Flexible business .BASE The theory is CAP In theory AP An extension of . Most Internet distributed systems emphasize availability , Will consider introducing BASE Support . This theory is very, very important , What I want to tell you is , Master this theory , It will be much easier to design a distributed architecture suitable for your own business , It's not that you can't feel it .

BASE At the heart of ： Basic available BA（Basically Available）、 Soft state S（Soft state）、 Final consistency E（Eventually consistent）.

Then why do you call it flexible affairs ？ In fact, it and ACID Is relative , No need to guarantee strong consistency , For example, a rubber band is bent , When you let go of the rubber band , It will recover itself , This is the flexible side of the rubber band .

3.2 BASE What does it have to do with Taijiquan

Every move of Taijiquan is not straight , Every move is about smooth 、 Draw arc , It looks soft , In fact, it's soft with hard . The last move of every move is a very rigid shake （ I can't describe the effect in words , Let's go watch TV ）. This last moment can be seen as the hard side , That's final consistency .

3.3 Basic available

How to understand basic usability ？ The point is in this basic , This theory doesn't tell us how to define basic , It's a vague concept . It's about The soft To what extent .

In distributed systems , We can understand basic availability as ensuring the availability of core functions , Allow loss of availability of some features . Basic available It can be done in four ways .

• Traffic peak clipping ： For example, multiple second kill sessions , Some East 8 Kill in seconds ,12 It's a little bit of a second kill .
• Delay response ： Such as double 11 Orders created by a mall during the period , The customer will be prompted that the order is being created , It may take more than ten seconds .
• Experience degradation ： For example, a competition activity , A large number of users go to the active page to view pictures , This is the time , A lot of pictures can't be displayed because of network timeout , At this time, you can consider replacing the original image , The returned image is not as clear as the small one .
• Overload protection ： For example, our usual message queue is full , Consider dropping subsequent requests , Or clear some requests in the queue , Protect the system from overload , But they all need to be designed in combination with their own business scenarios .

3.4 Final consistency

Final consistency ： After a period of synchronization, all the data copies in the system , Eventually a consistent state can be reached . In the end, it can be understood as a short delay .

Finally, consistency is adopted in many Internet services . But dealing with money or the financial system uses strong consistency or transactions .

As mentioned earlier ACID Of Strong consistency , and Final consistency What is the relationship with it ？

Strong consistency is also a kind of final consistency . How to understand the final consistency ？ Strong consistency can be regarded as consistency without delay . If you can't tolerate latency, use strong consistency , Otherwise, use final consistency .

3.5 What's the relationship between ultimate consistency and Taijiquan

One of the most amazing things about Taijiquan is that Take off the power , When they attack you with all their strength , Use Taiji moves to unload the strength of the opponent , Make the attack invalid . The unloading force can correspond to the flow peak clipping we mentioned before . In addition, after unloading the force , It's when we attack .

4、 Nothing beats nothing

Go back to the beginning of the article , Zhang Sanfeng taught Zhang Wuji Taijiquan , Zhang Wuji forgot all about it , How can you defeat the elder xuanming ？

*

Because Taijiquan focuses on the meaning of boxing , Not the moves . So Zhang Wuji understood the meaning of boxing , Nothing beats nothing .

When we design distributed systems , Don't memorize the three theories by rote , To really understand the principle , Then we can iterate out the most suitable distributed architecture for the current business system bit by bit .

5、 summary

• Taijiquan is divided into yin and Yang , as CAP Medium C and A.
• CAP Theory is the basic theory in distributed computing , There are three important indicators ： Uniformity 、 Usability 、 Partition tolerance .
• ACID It's the design concept of traditional database , The pursuit of strong consistency . Four indicators ： Atomicity 、 Uniformity 、 Isolation, 、 persistence . yes CAP in CP Extension of .
• BASE The theory is CAP The result of the trade-off between consistency and availability . yes CAP Medium AP Extension of . Focus on availability and performance first , According to the characteristics of business scenarios , Realize the basic availability of elasticity , Then achieve the final consistency of the data .
• BASE The theory is, to a large extent , It solves the problem of transactional system in performance 、 Fault tolerance 、 Common problems in usability, etc .
• BASE The theory is NoSQL Widely used in , yes NoSQL The actual theoretical support of system design .

The article also explains the core principles of the two-stage submission through the case of the six major factions besieging the Guangming summit , I believe you can understand .

This article conceives 2 Zhou , It's finally coming out , Wukong shamelessly asks for a second look and forward ~

- END -

Talk about distributed algorithms in Three Kingdoms , Comfortable, right ？

2020-12-11

It's been a terrible three years , Expose ten pits

2020-09-24

Set up a learning exchange group , You can scan the code and add me , remarks [ Study ]  Step up together ！

I'm Wukong , Try to be strong , Become a super Saiya ！

This article is from WeChat official account. - Wukong chat structure （PassJava666）.
If there is any infringement , Please contact the [email protected] Delete .
Participation of this paper “OSC Source creation plan ”, You are welcome to join us , share .