Nirvana rebirth! Byte Daniel recommends a large distributed manual, and the Phoenix architecture makes you become a God in fire

Preface

From mainframe to single architecture , From micro service architecture to no service architecture , Every evolution of architecture pattern is a Nirvana . Every software system is an ecosystem composed of a large number of services , Of individual services “ Death ” and “ be reborn ” It is the key factor for the continuous and reliable operation of the whole system . Notes from 5 This paper comprehensively analyzes how to build a reliable distributed system , At the same time Spring Boot、Spring Cloud、Kubernetes、Istio、AWS Lambda Sample projects of five architectural styles .

from 5 It comprehensively explores how to build reliable large-scale distributed systems ：

• Evolution from architecture

• Architecture design thinking

• Distributed cornerstone

• Indispensable infrastructure

• Technical methodology

Due to space limitation , Too many documents , Only part of the content can be displayed , Interested friends , It can be obtained by private letter

The first part Architecture in evolution

Architecture was not invented , But the result of continuous evolution . In this chapter, let's put aside code and Technology , In the name of discussing history , To sort out the terms that have appeared in the development of software architecture , From a global perspective , From the origin of these concepts to analyze what they are , What do they replace , Why can they succeed in the competition , Why become indispensable , And why they failed , Be eliminated in the struggle , Gradually annihilated in the dust of history .

The second part is the perspective of architects

Remote services extend the working range of computer programs from stand-alone to network , Extend from local to remote , Is the primary foundation for building distributed systems . And remote services are not only for distributed systems , In the age of Internet , browser 、 Mobile devices 、 Desktop applications and server-side programs , There is a general need to interact with other devices , So it's hard to find programmers who haven't developed and used remote services today , But there are many programmers who do not understand remote services correctly .

Transaction processing is involved in almost every information system , The significance of its existence is to ensure that all data in the system meet the expectations , And there will be no contradiction between the interrelated data , That is, the consistency of data state （Consistency）. According to the classical theory of database , To achieve this goal , It needs the joint efforts of three parties to ensure .

Modern enterprise or Internet Systems ,“ shunt ” Is a design that must be considered , The number of means used for diversion 、 Involving a wide range of scenes , Maybe even its developers may not be fully aware . This doesn't sound reasonable , But I think this is just a reflection of excellent architecture design ,“ It's widely distributed ” From the “ multistage ”,“ Not aware ” The supposed “ transparent ”, That is, the topic we will discuss in this chapter “ Transparent multistage shunt system [1]”（Transparent Multilevel Diversion System） The cause of

Even if only limited to “ Software architecture design ” In this context , System security is still a big topic . We're talking about computer system security , Not just “ The defense system is attacked by hackers ” Such narrow security , It should also include at least （ Not limited to, ） The following specific solutions to these problems .

The third part is the cornerstone of cloth

Before formally discussing various technical problems and solutions faced in distributed environment , Let's turn our attention from industry to academia , Learn several representative distributed consensus algorithms , Lay a theoretical foundation for the subsequent operation of shared data in a distributed environment .

An important design principle of microservice architecture is “ An organization that realizes independence and autonomy through services ”（Componentization via Service）, Emphasize that “ service ”（Service） instead of “ Class library ”（Library） To build componentized programs , The difference between the two is that the class library is statically linked to the program by the compiler , Use the functions by calling the methods of this , And services are out of process components , Use the functions by calling remote methods .

Fault tolerant design （Design for Failure） Is another core principle of microservices , It is also the change of development concept repeatedly emphasized in the author's book . however , Even if you have some psychological preparation , Most developers who first introduced microservice architecture into real-life production systems , In service discovery 、 With the support of gateway routing , After taking the first step of service , It is likely that you will still experience a period of labor , As more and more services are split , Then we will face the following two problems .

Microservices advocate decentralized governance （Decentralized Governance）, Do not pursue a unified technology platform , We advocate that the team members have the right to choose freely , Not subject to the language and technical framework . When building services in the development phase , Decentralized governance breaks the constraints brought about by the technology stack , The benefits are self-evident . But when deploying services in the O & M phase , Especially when considering safety issues , from Java、Go、Python、Node.js And other languages and frameworks , The probability of security vulnerabilities is certainly higher than using only one of these languages 、 The monomer system built by a certain framework is higher . In order to avoid the fear that once the vulnerability of the service node is broken by the attacker , Thus, the whole system and intranet are invaded , We must break some traditional safety concepts , To build a more reliable inter service communication mechanism .

As distributed architecture becomes mainstream , Observability （Observability） The word is also mentioned more and more frequently . first , It is related to controllability （Controllability） Together , By Hungarian mathematicians Rudolf E.Kálmán A set of dual attributes for linear dynamic control systems is proposed , The original meaning is “ The degree of its internal state can be inferred from its external output ”.

The fourth part Immutable infrastructure

Container is cloud computing 、 Microservices are the common cornerstone of many core technologies in the software industry . The primary goal of the container is to make the software distribution and deployment process from the traditional release and installation package 、 Change from manual deployment to direct release of deployed 、 Virtualized image containing the entire operating environment . Before container technology matures , The mainstream software deployment process is that the system administrator compiles or downloads the binary installation package , Prepare the correct operating system according to the software deployment instructions 、 Third party Library 、 The configuration file 、 After various pre dependencies such as resource permissions , To run the program correctly .Chad Fowler In the proposed “ Immutable infrastructure ” Article on this concept “Trash Your Servers and Burn Your Code”[1] I make complaints about the opening chapters. ： I don't know how many upgrade patches I have made 、 I don't know how many administrators have migrated their systems to other machines , There is no doubt that it will be a disaster .

In this chapter, we will discuss the topic of virtualization network , If there are no restrictions ,“ Virtual network ” Is a very rich content , Research on computer technology with a long history , It is an independent branch of computer science , Not attached to the virtualization container at all . Network operators often mention “ Virtualization of network functions ”（Network FunctionVirtualization,NFV）, Network equipment manufacturers and network management software providers often mention “ Software Defined Network ”（Software Defined Networking,SDN） And so on belong to the category of virtualized network . For ordinary software developers , Fully understand and master the virtual network , We need to reserve a lot of professional knowledge that is not commonly used in development , Consume a lot of time cost , Generally, it is unnecessary .

A container is a runtime instance of a mirror , In order to ensure that the image can repeatedly produce consistent runtime instances , It is necessary to ensure that the image itself is persistent and stable , This determines that all data changes in the container cannot be really written to the image , Otherwise, the stability of the image will be destroyed . So , Most of the data modification operations in the container are based on copy on write （Copy-on-Write） Strategy to achieve , The container will take advantage of the overlay file system （OverlayFS） Characteristics of , When the user intends to modify the image , Automatically write the changed content to a separate area , And then superimposed with the original data , Make it look like “ Cover ” The original content . Such changes are usually temporary , Once the container terminates running the line , These change information stored in separate areas will also be removed , No longer exists . thus it can be seen , If there is no additional treatment , By default, containers do not have the ability of persistent storage .

Scheduling is one of the core functions of container scheduling system ,“ layout ” The word itself contains “ Dispatch ” The meaning of . Scheduling refers to creating a schedule for a new Pod Find the most appropriate host node to run it , Whether this process is successful or not 、 The result is appropriate or not , The key depends on how the container orchestration system manages and allocates the resources of the cluster nodes . It can be considered that scheduling is necessary, and the resource control of container scheduling system is the premise , Let's start with Kubernetes Resource model .

The most fine-grained management of container choreography system can only reach the container level , Technical details below this granularity , Still can only rely on programmers to manage , The choreography system is difficult to provide effective support .2016 year , primary Twitter Infrastructure Engineer William Morgan and Oliver Gould stay GitHub Launched the first generation of service grid products Linkerd, And in a very short time around Linkered Set up the Buoyant company . And then , In the position of CEO Of William Morgan Published articles “What’s A Service Mesh?And Why Do I Need One?” For the first time “ Service Grid ”（Service Mesh） The word" , Since then , As a new communication concept, service grid began to spread rapidly , It appears more and more frequently in the vision of companies and technology communities . The reason why service grid can get the attention of enterprises and communities , It is because it makes up for the lack of fine-grained control ability of container orchestration system for distributed applications .

The fifth part Technical methodology

appendix A Technical demonstration engineering practice

appendix B Deploy Kubernetes colony

---

If you have questions in the reading process , Please leave a message to discuss , Finally, I wish you all success in getting your favorite offer.