Openstack learning notes (I)

be based on OpenStack Design and implement book content notes ：

Learn about the original initialization virtualization technology ：

Time to allow multiple users to remotely share the same high-performance computing device

Understand the original intention of virtualization technology ：

Make full use of expensive large resources as much as possible

Three models of Cloud Computing ： Public cloud 、 Private cloud and hybrid cloud

Expand understanding ：

1、 Public cloud ： A service that provides resources to the public ; Users access the use of these resources through the Internet ;

advantage ： The cost is low , Scalability is very good

shortcoming ： Lack of control over cloud resources 、 Security of confidential data 、 Network performance and matching issues

2、 Private cloud ：“ private ” This kind of platform belongs to non shared resources , Rather than its security advantages . The private cloud is built for a single customer to use , So the data 、 Security and service quality are better guaranteed than public cloud .

Private clouds fall into two categories ：

Internal private cloud ： Built by organizations in their own data centers , This form has limitations in scale and resource scalability , But it's good for standardizing cloud service management process and security . This method is suitable for those who need to apply 、 Platform configuration and security mechanisms are completely controlled by the organization .

External private cloud ： This kind of private cloud is deployed outside the organization , Managed by a third party . A third party provides the organization with a dedicated cloud environment , And guarantee privacy and confidentiality . The cost of this solution is lower than that of internal private cloud , And it's easier to scale up .

3、 A hybrid cloud ： In hybrid cloud mode , Cloud platform consists of two different modes （ Private or public ） Cloud platform . These platforms are still separate entities , But using standardization or proprietary technology to achieve binding , Data and applications can be transplanted to each other

​ Apply the hybrid cloud model , An organization can deploy secondary applications and data to the public cloud , Make full use of the scalability and cost advantages of the public cloud . At the same time, put mission critical applications and data in the private cloud , safer .

4、 Public cloud users need to go online quickly , Economic material benefit . Private cloud users are more likely to require a smooth transition , Keep the existing experience , No impact on existing business . When selecting public cloud or private cloud , The final requirement comes from the user himself , Through our comparison , It can be seen that public cloud and private cloud have their own preferences , There are also different comments in the eyes of different users .

Expand understanding Vanderpool Virtualization technology ： Can be on the same machine PC The technology of running multiple operating systems simultaneously on

The prototype concept of cloud computing ： The network is the computer

initial OpenStack: from RackSpace and NASA Cooperate in R & D and initiate , With Apache Licensed open source code projects

OpenStack Understanding of the foundation ： Simply put, it is a non-profit organization , Contributions from various companies , To co manage OpenStack project , Help promote OpenStack Development of 、 Release and application

There are three types of services in cloud computing ：

IaaS（ Infrastructure ）： Cloud service providers put IT The infrastructure layer of the system is rented out as a service , It's up to the consumer to install the operating system 、 middleware 、 Databases and Applications .

PaaS（ Platform as a service ）： Cloud service providers put IT The platform software layer in the system is rented out as a service , Consumers develop or install programs themselves , And run the program .

SaaS( Software as a service )： Cloud service providers put IT The application software layer in the system is rented out as a service , Consumers don't have to install their own apps , It can be used directly , This further reduces the technical threshold for cloud service consumers

PaaS and SaaS It does not necessarily need the support of the underlying virtualization technology , but IaaS It is generally based on virtualization technology

IaaS System problems to be solved ： How to automatically manage virtual machines on these physical hosts

IaaS The purpose of ： On a large-scale network or in a data center , There will be thousands of physical hosts , It is unrealistic to rely solely on the operation and maintenance personnel to complete these management tasks , At this time, the software system is required to automatically assist the operation and maintenance personnel to manage and maintain the operation of the system , Provide users with virtual machine services

Expand understanding ：

1、Hypervisor： An intermediate software layer running between the underlying physical server and the operating system , Allows multiple operating systems and applications to share hardware .

2、Hypervisor Coordinate hardware resource access , Protection between virtual machines . After the server starts , It will load the operating systems of all virtual machine clients , Allocate memory for the virtual machine 、 Disk and network, etc .

3、Hypervisor Is the core of all virtualization technologies , The main function is non disruptive support for multi workload migration .

Video based notes ：

Cloud computing : Represents computing resources like cloud water , Distribute on demand , Recycle . For example, a dry place needs rain 100 drop , The clouds will fall 100 Drips of rain （ resources ） To this arid land , When not needed ,100 A drop of rain evaporates and answers the cloud

Cloud application forms ：

One 、 Private cloud ： Build the infrastructure and hardware and software resources in the firewall , be based on iaas Build a private cloud platform for internal use

Two 、 Cloud storage ： Cloud storage system is a cloud computing system with data storage and management as the core

3、 ... and 、 Cloud games ： The game runs on the cloud platform server , The cloud platform decompresses the game screen and transmits it to the user , The client does not need a highly configured processor and graphics card , Only basic video decompression capability is required .

Four 、 Cloud and things ： Based on the cloud platform, we can realize the Internet connecting things .

5、 ... and 、 Cloud security ： Detect software exceptions in the network through a large number of mesh clients , Get the Trojan horse , The latest information about malware , Push to the cloud platform server for automatic analysis and processing , Then send the solution to each client

6、 ... and 、 Public cloud ： Cloud platform is open to the outside world , Mainly with Iaas and Paas Mainly , The more mature ones are Iaas, Such as ali cloud , Tencent cloud , Qingyun et al

7、 ... and 、 A hybrid cloud ： Combination of public cloud and private cloud , That is to the inside and outside of the enterprise , for example AWS

Traditional applications and cloud applications ：

One 、 Traditional applications

     Traditional applications are like keeping pets , Pets should be taken care of when they are sick

     Each application is unique 、 Special

     Dedicated servers 、 Hardware and software guarantee reliability

     Insufficient resources , increase cpu、 Memory 、 disk

     Specialized technical support

Two 、 Cloud aware applications

     Cloud aware applications are like cattle raising , The cow is sick , You need a new cow

     Applications run in one or more virtual machines

     Insufficient resources , Add new virtual machines

     App pending , Restart or create a new virtual machine

openstack Introduction to relevant components and communication process ：

Five components

When you need to use a computer ： Send an order to buy a computer , Then connect to the Internet , Install the operating system , Additional hard disk , The backup data

horizon-> Provide web Interface console , Click the mouse to initiate a request ->kevstone-> Authentication global , All components must pass through it ->nova-> Call the virtualization driver , Create a virtual machine （ Equivalent to buying a computer ）->neutron-> Connect to the Internet （ Network virtualization ）->glance-> Provide images （ Equivalent to installing the operating system , The image is also backed up in swift in ）->cinder-> Additional hard disk ( Block storage )->swifit-> Backup （ Object storage ）（ It is used for storage and backup of massive data ）

horizon,nova,neutron The three components basically implement one openstack The functions of the private cloud , So it's the core component

Three core projects

1、 Console ：

Project name ：Horizon

function ：web How to manage cloud platforms , Build a virtual machine , Distribution network , Equipped with safety group , Cloud disk

2、 Calculation ：

Project name ：Nova

function ： Responsible for responding to virtual machine creation requests 、 Dispatch 、 Destroy the virtual machine

3、 The Internet ：

Project name ：Neutron

function ： Provide a complete set of API, Users can base on this API Define your own network , Different manufacturers can be based on this API Provide your own product realization

Storage project 2 individual

1、 Object storage

Project name ：Swift

function ：REST Style interface and flat data organization structure .RESTFUL HTTP API To save and access any unstructured data ,ring Ring can realize automatic data replication and highly scalable architecture , Ensure high fault tolerance and reliability of data

2、 Block storage

Project name ：Cinder

function ： Provide persistent block storage , That is, provide an additional cloud disk for the virtual machine .

Shared service items 3 individual

1、 Certification services

Project name ：Keystone

function ： For the visit openstack Each component provides authentication and authorization functions , After certification , Provide a list of services （ Store the services you have access to ）, Each component can be accessed through this list .

2、 Image services

Project name ：Glance

function ： Provide different image options for the host to install the operating system

3、 Billing services

Project name ：Ceilometer

function ： Collect cloud platform resource usage data , Used for billing or performance monitoring

High level service items 1 individual

1、 Choreography Services

Project name ：Heat

function ： Automated deployment of applications , Automate the entire lifecycle of management applications . It is mainly used for Paas

Communication between components is based on rest api

keystone function ：

1、 authentication

2、 distribution rest api

keystone Decided to user The user permissions are role,endpoint Only three types of access are provided , Users can access any path , Does not change the user's permissions , Just for the sake of unification

Endpoint Divided into three categories ：

admin url –> to admin The user to use ,Port：35357

internal url –> OpenStack Internal services are used to communicate with other services ,Port：5000

public url –> Addresses that Internet users can access ,Port：5000

keystone Detailed explanation ：

User:user user ; Use Openstack The client of the component can be a person 、 service 、 System , Any client to access openstack Components , You need a user name .

Credetials: The credentials used to confirm the user's identity

Authentication: The process of authenticating a user .Keystone The service checks the user's Credential To determine the user's identity

Token: Is a numeric string , When accessing resources, you need " flashed " Your token .

Role: to user User partition permissions

Policy:Policy It's just one. JSON file , The default is /etc/keystone/policy.json. By configuring this file ,Keystone Realized with User be based on Role Authority management .

Project(Tenant): It's a person 、 Or a collection of resources owned by a service . Different Project Resources are isolated from each other . In a Project(Tenant) Can contain more than one User, every last User Will be used according to the division of permissions Project(Tenant) The resource

Service: namely Openstack Various component services running in .

Endpoint Divided into three categories ：

admin url –> to admin The user to use ,Port：35357

internal url –> OpenStack Internal services are used to communicate with other services ,Port：5000

public url –> Addresses that Internet users can access ,Port：5000

Catalog:catalog One openstack A collection of related services deployed , Each service has one or more endpoint（ That is, accessible url Address ）, namely catalog=services+endpoint

glance Components :

v1 edition

It is divided into glance-api,glance-registry

glance-api:1、 Accept api request , issue glance-registry.2、 Pull the image from the back-end storage device

glance-registry:1、 Query the database , Get the metadata of the image （ The image information exists mysql In the database ）

v2 edition

Only glance-api To complete the above operation

glance The communication within the component does not go rpc Mechanism （MQ）, Direct communication

Simply understand three storage methods

1、 From the user's point of view , What you see is a folder , File storage

2、 From the user's point of view , What you see is a bare disk , Block storage

3、 From the user's point of view , Storing the file calls rest api The way to save , Even if the object stores

File storage can be modified based on the original file , Object storage does not work （ Can only cover ）

cinder Main components ：

1、cinder-api： Provide rest Interface , Responsible for handling client request , And will RPC Request sent to cinder-scheduler Components

2、cinder-scheduler： be responsible for cinder Request scheduling , be responsible for cinder-volume Specific scheduling processing , send out cinder RPC Request to selected cinder-volume.（ In fact, it calls all... From the database cinder-volume Information , Choose by your own algorithm ）

3、cinder-volume： Be responsible for specific volume Request processing , Provided by different back-end storage volume Storage space

openstack Communication within components , A component and a service are based on rpc Realized by mechanism .

rpc The mechanism is based on AMQP( Advanced message queue protocol ) Realized .AMQP Is a message oriented middleware protocol for asynchronous message communication , So as to meet the loose coupling within the component .rabbitmq Is based on AMQP Message queue of , It can be understood as rpc The mechanism is based on rabbitmq Realized

AMQP The model has four important roles :

1、Exchange： according to Routing key Forward the message to the corresponding Message Queue in

2、Routing key： be used for Exchange Determine which messages need to be sent Message Queue

3、Publisher： Sender of message （ The publisher of the message ）, Send a message Exchange And specify Routing Key, In order to Message Queue You can receive the message correctly

4、Consumer: Receiver of message （ Subscribers to the message ）, from Message Queue Get message

Take express delivery as an example ：

publisher： Express content

exchange： Jingdong Express

routing key： Address

message queue： Express box

consumer： The recipient

Express release (publisher) Choose a courier -> JD express or SF express (exchange)-> Send to the corresponding address （routing key）-> Put in inbox (message queue)<- The recipient （consumer） Pick up items from your inbox

Publisher Can be divided into 4 class ：

1、Direct Publisher Send point-to-point messages ;

2、Topic Publisher use “ Release —— subscribe ” Mode to send messages ;

3、Fanout Publisher Sending broadcast messages ;

4、Notify Publisher Same as Topic Publisher, send out Notification Related messages .

Exchange Can be divided into 3 class ：

 1.Direct Exchange according to Routing Key Make a precise match , Only the corresponding Message Queue Will receive messages ;

 2.Topic Exchange according to Routing Key Pattern matching , As long as it matches the pattern Message Queue We'll all get messages ;

 3.Fanout Exchange Forward the message to all bound Message Queue.

OpenStack RPC The module provides rpc.call,rpc.cast, rpc.fanout_cast Three RPC Calling method , Send and receive RPC request .

rpc.call: send out RPC Request and return the request processing result （ There must be a return value ）

rpc.cast： send out RPC Request no return （ You don't need to return a value ）

rpc._cast: Used for sending RPC No result returned from broadcast

Communication between components is based on rest api（ There will be one in each component api Components exist ）, Intra component communication is based on rpc Mechanism

nova And cinder It works in a similar way

nova Main components :

nova-api： And cinder-api Works in a similar way

nova-scheduler： And cinder-scheduler Works in a similar way

nova-compute: And cinder-volume Works in a similar way

nova-conductor： help nova-computer view the database , Then pass the message through message queue Pass to nova-computer

Why should I use... When viewing a database nova-conductor instead of nova-computer Look it up directly ？

1、 Based on safety considerations ： Once the virtual machine is broken , Would pass nova-computer Get information about virtual machines

2、 Based on database pressure ：nova-computer There are many , Many virtual machines can be built at the same time , Multiple nova-computer Accessing the database at the same time will put pressure on the database

MQ（message queue） Is the message distribution mechanism , It is used to decouple and . Is an asynchronous mechanism

The database is used to record the whole openstack State of , For example, the status of each component , Details of setting up the virtual machine , Image details , User's token, etc

openstack All the components of can be arranged by a single machine , It's completely distributed

neutron Contains the components

1、neutron-server: It can be understood as a special receiver neutron rest api Called server , And then responsible for putting different rest api Distribute to different neutron-plugin On

2、neutron-plugin: It can be understood as the entrance to the realization of different network functions , Each manufacturer can develop its own plugin. Received the distributed rest api after , image neutron database Complete the registration of some information , Then, the specific business operations and parameters to be executed are notified to their corresponding neutron agent

3、neutron-agent: It can be understood intuitively as neutron-plugin Agent on device , Transform the notified business operations and parameters into specific device level operations , To guide the action of the equipment . When there is a problem with the local device ,neutron-agent Will inform neutron-plugin.

neutron database:neutron The database of , Some business-related parameters exist .

neutron-plugin In two parts ：core-plugin,service-plugin

core-plugin:neutron Middle is ML2( Layer 2 plug-ins , Data link layer , be based on mac Address )

ML2 Include network、subnet( subnet )、port( A virtual switch port on a virtual switch )

service-plugin: except core-plugin Other than plugin, Include route、firewall、loadbalancer（ Load Balancer ）、VPN etc.

Deployment is usually neutron-server and neutron-plugin Deployed on one machine ,neutron-agent( Specific work ) Deployed on a dedicated machine （ This machine is called a network node ）

Three nodes ： The control node （ Of each component api Put it on the top to send and receive requests ）、 Computing node （ Responsible for building virtual machines ）、 Network nodes （ Responsible for various network functions ）

vlan Characteristics of network mode :

1、 Must be bound to the physical network

2、 The second floor ： Broadcast communications

3、 You don't need network nodes

4、 Small scale deployment , Efficient

shortcoming ：

1、4096 vlan The upper limit problem

2、mac Address and arp Watch is too big

3、 The broadcast storm problem

4、 be based on ip The subnet division of the address limits the network size

flow chart ：

OpenStack Communication flow chart of some components

Rpc Mechanism implementation diagram

 

Cinder Component internal communication flow chart