当前位置：网站首页>Understanding openstack network
Understanding openstack network
2022-06-24 19:08:00 【51CTO】
Abstract ： If you can understand OpenStack Network of , So for other cloud platform networks , It should also be understood and mastered through analysis .
This article is shared from Huawei cloud community 《 《 Learn cloud network from teacher Tang 》 - OpenStack Network implementation 》, author ： tsjsdbd .
Integral design
First ,OpenStack Is used to manage a large number of VM Of “ lord ”. His goal is to control the physical world , To manage a large number of VM. namely ： You can give VM grouping , In the same group VM, In the same network , It can communicate with each other . Different groups of VM, In different networks , Cannot communicate with each other .
As for why to group ,
1、 It is the same as the physical server , So many machines , According to the servers in different computer rooms , Connect to different networks .
2、 Yes, I can put different groups of VM, Sell to different “ user ”, such , Group 1 Of VM It belongs to Zhang San , Group 2 Of VM It belongs to Li Si , So they are separated from each other , They don't know each other . So I can become a cloud vendor , Cloud platform services are provided externally .
- Of course, as a cloud vendor , You must also allow a user , You can have 2 A network . In case the client has too much money , Just bought a bunch of VM, Play it separately , Is that so? .
Logical view
In reality ,2 Servers in computer rooms , If the network wants to be connected , It depends on the router to help . It is similar in the virtual world .
therefore , logically ,VM The network of the world is just like this .
Zhang San's VM Network of , To be with Lisi VM Network of Interworking , Or Zhang San's own 2 Two independent networks are interconnected . You have to go through a process called Router Of “ Virtual router ” To complete .
Physical view
The logical view above , In Physics , It's like this ：
As for how to connect to a network cable , Messages running multiple virtual networks at the same time . This is the message on a network cable , It means different sects . You can go back and have a look VLAN/VxLAN chapter .
in addition , Here you can see , A virtual network “Router”, In fact, it is not a specific virtual router device , And it's just a “ The Internet namespace+ Forwarding rules ” It was done. , I'll talk more about .
Simple model
Suppose now you design OpenStack Network implementation of .
That's what we learned before OVS chapter , You can know , In order to achieve the above mentioned OpenStack The purpose of network virtualization . The simplest implementation is to add one to each physical server OVS Virtual switch ; Then each VM All connected OVS On port , Each port is grouped by , Put the corresponding VLAN label . The basic requirements can be achieved .
however , This initial 1.0 Implementation of version , There is a place where there is no criticism , Just can't give VM Set security group . You have a great goal to become a cloud platform , How can a platform not have the capability of a security group （ although , stay VM Inside , You can set firewell perhaps iptables The rules , however VM Inside , It has been sold to users , You run into someone's room , It is not appropriate to set rules , It may conflict with the user's own business rules ）.
So the first 2 A version , Improvement . We will be having VM Set the security group outside the ：
therefore , We are in each VM The gate , Add one more Bridge bridge , whatever VM Of traffic , They're going through this Bridge. such , By means of Bridge above , increase iptables The rules , You can give VM The purpose of setting the security group .（ Be careful , This is the time ,VM Your message hasn't arrived yet OVS, So the message is still not typed VLAN The original message of the tag , therefore iptables Rules can also be implemented ）.
This is our OpenStack The Internet 2.0 edition .
however , In practice , You found this single seedling OVS, To set port forwarding rules, there are 2 part ：
- The first half . namely ： to VM Set up Tag label .
Every additional one VM when , Just label this port , Plug and unplug the virtual network cable and other configuration actions . The logic of this part is relatively fixed , Not much change .
- The second part of . namely ： Through the physical network cable , How to call a message “ Sects ” Mark .
This part has changed a lot , Sometimes physical networks , We have to go GRE, Sometimes I have to go VLAN, Sometimes I have to VxLAN. There is still time. , We have to use special network equipment . The platform should be based on the deployed computer room network cable , Customize different rules .
So this 2 class OVS The rules of （ All of them openflow In the forwarding table of ）, In the programmer's mind “ Sub database and sub table ”（ Or when we write code ,“ Extract function ” The logic of ） The idea of , Let's put 1 A single seedling OVS, Divided into multiple VOS. Do different things separately .
therefore , Here we are 3.0 edition , This version is more general . Basically, it can be compared with reality OpenStack Our network is closer . But in the network node section , It needs to be enhanced . It's ours. VM, In addition to mutual visits （ Traffic is still wandering between several physical servers ）, You have to access the Internet （ The flow goes outside the machine room ）. therefore , We must continue to strengthen it VM Ability to access external networks ：
In this way , It's almost there 4.0 Version of the . Later we will introduce OpenStack The Internet , It is based on this version . stay OpenStack network , For all kinds of ovs,bridge, Name of interface , There is a set of its own norms . It's not so casually named as above .
The control node
With the above for VM Set up a model for the virtual network , Then it should be automated （ namely ： For every one created VM, Set up the supporting virtual network cable connection for it ）. You have to write a master program , It is used to control the behavior of these computing nodes . as follows ：
therefore , according to OpenStack Official structure , It needs to have 3 Types of nodes ： Pipe joints , Computing node , Network nodes .
Above each node , Deployed a bunch of agent, It is used to receive the control command from the boss . The boss is Master Manage nodes .
notes ： The previous chapter also mentioned , Distributed systems , Reliable control requires a “ Agent ”, such as RabbitMQ（OpenStack I chose this ）,ETCD（Kubernetes I chose this ）,ZooKeeper（Hadoop I chose this ） such .
（1） The first is the top red line , Namely “ Master logic ” Used to control Agent Working , Management surface network for short .
（2） Then there is the green line in the middle , That's it VM We are on this cable , Send a lot of “ Own sect ” Message of , namely VM Communicate with each other , Have to go through the network , Large amount of data . Data plane for short .
To ensure that the management side and the data side are isolated , They don't influence each other （ namely ：VM Crazy contract awarding , Don't flush out the message of management command ）. On each physical server , Must have 2 Block NIC , One is used to run the management network cable , One is used to run the data network cable .
（3） Then there is the dark green line in the lower left corner , This is VM They are used to access the external network of the computer room . Only network nodes are needed , Just have an extra network card .
（4） Finally, the purple line . Your master logic , Would you like it packaged as API Interface , Access channels to external exposures ？ If you want to , You can add . If not , Log in to the master node every time , Manual command control is also OK .
Computing node
Here we open a OpenStack Of computing nodes , Look at its network structure , Remote recording of the year （2013 year ,OpenStack edition Havana） I learned OS When it comes to the Internet , See a file , It helps me a lot , Stick it directly here ：
According to the above “ Design thinking ”, You should be able to understand the network logic in the figure above . Name up , Generally internal ovs It's called br-int. The name of the tunnel is br-tun. If you have an environment , Query on the node , Use various network commands （ip,ovs-vsctl,brctl etc. ）, You can confirm that .
Network nodes
Again , Network nodes , The network is composed as follows ：
among , The upper red dotted line , It means a The Internet namespace.dnsmasq It's a DHCP The server （ Automatically assigned IP The program , To give VM Distribute IP Address ）.
This node also uses various network commands to query, view and confirm .
ps, Because there are many networks on this network node namespace, So remember to use ip netns exec Command to enter the corresponding ns Query the details of this virtual space .
floating IP（EIP）
VM In addition to having its own virtual network IP, You can also have one floating IP（ notes ： Corresponding to cloud vendors , This is usually called EIP）. Let's take a look at this “ float IP” What implementation logic is it .
The concept of logic
First , float IP, Is the physical network world , namely OpenStack Of the external network . It is a real existence IP Address （ Not with VM equally , You made it up IP）.
Pictured above , Yes floating IP, My summary is ：VM External name .
When you are from an external network , Visit this “ float IP”, It's like visiting this one VM. As for why it's called “ float ” The word , Because of the name , Will drift .
for instance ：“ Grand mage protecting the country ” The name is very loud , When you report that you are looking for “ Grand mage protecting the country ” This person is , Everyone knows who you are looking for . But this “ Grand mage protecting the country ” Name , It can be transferred from one person to another .
Directly corresponding to the cloud vendor EIP, Is it easy to understand .
Concrete realization
Our focus , Focus directly on one of the network nodes namespace Inside .（ The float of this example IP yes 192.168.101.3）. Here's the picture ：
At the network node , Inquire about ns.
[email protected]:/# ip netns
qdhcp-a7e512cf-1ca0-4ec7-be75-46a8998cf9ca
qrouter-4cdb0354-7732-4d8f-a3d0-9fbc4b93a62d
- 1.
- 2.
- 3.
Find the corresponding router that ns（ Above, at the five pointed star ）, Then query the network card information in this ：
[email protected]:/# ip netns exec qrouter-4cdb0354-7732-4d8f-a3d0-9fbc4b93a62d ip address
11: qg-1423ba35-7c: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
inet 192.168.101.2/24 brd 192.168.101.255 scope global qg-1423ba35-7c
inet 192.168.101.3/32 brd 192.168.101.3 scope global qg-1423ba35-7c
12: qr-9f1fa61e-1e: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
inet 172.17.17.1/24 brd 172.17.17.255 scope global qr-9f1fa61e-1e
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
You can see , There's a man named qg-xx Network card of , With this floating IP Address .
Then we check this ns Inside iptables The rules ：
[email protected]:/# ip netns exec qrouter-4cdb0354-7732-4d8f-a3d0-9fbc4b93a62d iptables -t nat -S
- 1.
You will find that there are so 2 Bar rule ：
The first 1 The bar is SNAT The rules , Is to put the source IP Change of address means . The specific content is ： If the source IP yes 172.17.17.2 Of （VM Of ）, So put the source IP Switch to 192.168.101.3（floatingIP Of ）.
The first 2 The bar is DNAT The rules , Is to set the goal IP Address change . The specific content is ： If the purpose IP Of 192.168.101.3（floatingIP Of ）, Just put the purpose IP Switch to 172.17.17.2 Of （VM Of ）.
thus , All the messages were sent to this VM Well .
therefore , One VM Once you have it floatingIP（ Also called EIP）, It can be accessed by the Internet , You can also access the Internet directly .
however , The real exterior IP, It could be limited , We have to save some money , Here we have the following SNAT and DNAT function .
SNAT function
If one VM, Want to access the external network , But I don't assign it floatingIP. It can be used at this time SNAT.
This is the same as the one in the previous section ns, Check this out ns The network card information inside , You can see , One more 101.2 Of IP.
[email protected]:/# ip netns exec qrouter-4cdb0354-7732-4d8f-a3d0-9fbc4b93a62d ip address
11: qg-1423ba35-7c: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
inet 192.168.101.2/24 brd 192.168.101.255 scope global qg-1423ba35-7c
inet 192.168.101.3/32 brd 192.168.101.3 scope global qg-1423ba35-7c
12: qr-9f1fa61e-1e: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
inet 172.17.17.1/24 brd 172.17.17.255 scope global qr-9f1fa61e-1e
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
Let's check this ns Inside iptables The rules ：
[email protected]:/# ip netns exec qrouter-4cdb0354-7732-4d8f-a3d0-9fbc4b93a62d iptables -t nat -S
- 1.
You will find that there are so 1 Bar rule ：
The rule is , All sources IP yes 172.17.17.0/24 The message of this network segment （ Is all of the... In the network VM）, All sources IP Change of address means .
therefore , Once a virtual network is set up SNAT function , So all the people in this network VM, You can access the Internet . It's just that everyone shares an external exit IP Address （ In essence EIP）, This means to save some money .
The disadvantage is that ： Only from the inside （ virtual network ） External access （ External network ）, The outside cannot access the inside （ After all , This IP It's shared by everyone , Not one of them VM Of ）.
DNAT function
In line with the principle of economizing （ That is, a lot VM Share an external IP）. If you want external access to internal VM, You can also use DNAT function .
On the principle of , You should have thought of , Is in the ns Add one inside , Depending on the port , Forwarding for different purposes IP Address of the DNAT The rules .
This is a way to save money , The disadvantage is that ： Only the corresponding destination port can be specified . such as , External port 80, Assigned to VM1 Occupied . that VM2 You can't use 80 了 , It can only be wronged , Use external port number 81（ Or others ） 了 .
Saving money always means losing something , otherwise , For each VM Buy one EIP It's over .
Router
OpenStack Inside Router, It's used to put “ A network ”, Connect to “ Another network ” Of . It can be 2 A virtual network , It can also be 1 A virtual network +1 An actual external network .
One Router The essence is a The Internet namespace, The previous chapter describes floating ip equally , This ns It's a virtual one “ Transfer station ”. All the Internet connections , You need to go to this transfer station first “ Take a break and dress up ”, Then go to the destination network .
Be careful , In the same user's 2 Network interconnection , and 2 Network interconnection of different users , The technology of the underlying implementation is the same . The difference is that different users , Need to control permissions , Otherwise, Zhang San can connect to Li Si's network at will .
Router Concept , Corresponding to cloud vendors , General name “VPC interconnection ”. There are a wide variety of products , Like before “vpc peering”, current “ Cloud enterprise network ”“ Enterprise routing ”“ Cloud connectivity ” etc. .
Metadata service
metadata service , Is to allow everyone VM Ask God （OpenStack platform ）：“ You create my profile above , What did it all say ？”. This is a very interesting feature .
Function is introduced
you （VM） Ask God , You have to know where God put ？ So in OpenStack On , Put God's address , Write a special IP：169.254.169.254, It's good to remember .
Ask God's way ：
You can try , If you find this IP You can visit , It shows that you can prove that your machine is a virtual machine VM, Instead of a physical machine .
for instance ,VM ask ： What is the startup script after I was born ？
That is, ask yourself “userdata Information ”
This function , It's more useful , Especially doing VM When automating （ps, You can go and look up one called cloud-init Things that are ）.
metadata The feature should be from AWS.OpenStack For compatibility AWS This “ Ask God ” The function of （ Of course , It must be recognized that this function is still useful ）. It also supports this metadata service .
Concrete realization
We know to create & management VM The components of , It's called Nova. That is to say metadata characteristic , From the virtual world （VM Inside ） To access the physical world （Nova Of API）, After the introduction above , In this case , There must be a “ Transfer station ” Of .
Let's take a look first VM Inside , visit 169.254.169.254 When , Where did the message go ：
stay VM Knock inside ：
You can see visit “ lord ” when , The message went VM Gateway to network IP（172.17.17.1） That's it .
So gateway IP Where is the ？ On the network node namespace Inside ：
There is a network card called ：
Let's see , visit 169.254 Message of , To this “ Transfer station ” after , By how “ Take it ” Of .
You can see , The destination address is 169.254 Message of , It will be transferred to the local 9697 port .
that , Who's here （ This namespace Inside ） monitor 9697 Port? ？ The answer is God's proxy , One agent Eavesdropping here .
[email protected]:/# ip netns exec qrouter-7a44de32-3ac0-4f3e-92cc-1a37d8211db8 netstat -anpt
tcp 0 0 0.0.0.0:9697 0.0.0.0:* LISTEN 11937/python
- 1.
- 2.
Look at the process number , Specific orders ：.
[email protected]:/# ps -ef | grep 11937
root 11937 1 0 08:43 ? 00:00:00 python
/usr/bin/neutron-ns-metadata-proxy -metadata_proxy_socket=/var/lib/neutron/metadata_proxy
- 1.
- 2.
- 3.
You can see , There is one proxy Process listener 9697 port , And will “ Visit God's request ”, Transferred to local unix domain socket The monitor of （ namely agent）.
Use
perhaps
It is found that you are listening to local unix domain socket The process of ID
Then look at the process ID, Is it God's agent：
logically , The whole process is as follows ：
Refer to the figure for details ：
therefore , The key is still that namespace Transfer station .
attach , This paragraph refers to links ：
http://niusmallnan.com/_build/html/_templates/openstack/metadata_server.html
http://techbackground.blogspot.com/2013/06/metadata-via-quantum-router.html
DVR（Distributed Virtual Routing）
In the above introduction, you can see , be-all VM The virtual machine , To access the Internet , Must pass through the network node . There are also some disadvantages ,1 The network traffic pressure of network nodes is very high ;2 Once the network node is abnormal , a large number of VM Will be affected . therefore , Here, can we put the network node's “ Transfer station ” function , Copy a copy to each computing node . Then on the compute node , Add judgment logic ：
The answer is DVR 了 . In order to reduce the load of network nodes , While improving scalability ,OpenStack stay Juno Version introduced DVR characteristic ,DVR Deployed on the compute node . Compute the VM Use floatingIP visit Internet, You don't have to go through network nodes , Directly from the compute node DVR You can access .
In this way, the network nodes only need to deal with the traffic that accounts for a part of the overall traffic SNAT （ nothing floating IP Of vm Correspondence with the outside ） Traffic , The load and the dependence of the whole system on network nodes are greatly reduced .
Specific calculation node if conditional , It is through openflow The rules , Controlled . This is a little too detailed , No detailed study . You can read the corresponding articles ：
https://www.cnblogs.com/sammyliu/p/4713562.html
https://docs.openstack.org/ocata/networking-guide/deploy-ovs-ha-dvr.html
So you can see , The router of the original network node , Now it is distributed to various computing nodes . I was alone （ Network node Router） The work to be done , Now it is scattered to many people （ Of each computing node Router） dry . Indeed, distributed routers .
summary
Basically ,OpenStack Network implementation of , Is the integration of all the current “ Network virtualized parts ”, Include ：ovs Switch ,bridge bridge ,veth Ethernet cable ,tap Ethernet cable ,patch Ethernet cable ,namespace Space ,iptables Rules etc. . It is also the most complex network implementation I have ever been in contact with （ So this article has been dragged to the end ）. If you can understand OpenStack Network of , So for other cloud platform networks , It should also be understood and mastered through analysis .
Last , Basic cloud network related courses , Teacher Tang can only teach this . After all, I can be an introductory tutor , It is not specially responsible for network development . So after getting started , If you want to continue to study the cloud network , Even started designing network virtualization solutions , You have to continue to practice by yourself . Sao Nian , come on. ~
notes ： At this stage, the author mainly focuses on cloud native related businesses （Kubernetes colony ）, therefore OpenStack Network information is the latest . But the relationship should not be big , Because its network design principle is inherited . also , Our course , The main purpose is to understand . To design , I have to learn more by myself .So, In order to understand OpenStack The principles of the network , This article is enough .
Click to follow , The first time to learn about Huawei's new cloud technology ~
边栏推荐
- JS clothing photo album case
- 我用sql形式的会出现cdc读取乱序吗
- Introduction and download tutorial of two types of soil data
- three.js创建的基础框架
- JS picture display and hiding cases
- Sr-gnn shift robot gnns: overlapping the limitations of localized graph training data
- System design idea of time traceability
- Volcano成Spark默認batch調度器
- ArrayList源码解析
- Use ado Net call stored procedure
猜你喜欢
Why are life science enterprises on the cloud in succession?
R语言 4.1.0软件安装包和安装教程
Starring V6 platform development take out point process
Do you have all the basic embedded knowledge points that novices often ignore?
Northwestern Polytechnic University attacked by hackers? Two factor authentication changes the situation!
Introduction, download and use of global meteorological data CRU ts from 1901 to 2020
API管理之利剑 -- Eolink
微信小程序轮播图怎么自定义光标位置
Volcano成Spark默认batch调度器
Tkde2022: Dialogue recommendation system based on knowledge enhanced sampling
随机推荐
Real time rendering: the difference between real-time, offline, cloud rendering and hybrid rendering
cdc+mysql connector从维表中join的日期时间字段会被+8:00，请问阿里云托管的
Self taught C special data type
多云模式并非“万能钥匙”
Use ado Net call stored procedure
NFT质押流动性挖矿系统开发技术
Stored procedures in sqlserver
Volcano devient l'ordonnanceur de lots par défaut Spark
Nokov motion capture system makes it possible for multi field cooperative UAV to build independently
Introduction and download tutorial of two types of soil data
Executing SQL statements with parameterized commands
Introduction, download and use of global meteorological data CRU ts from 1901 to 2020
mysql binlog 数据源配置文档麻烦分享一下
MySQL binlog data source configuration document, please share
Module V
API管理之利剑 -- Eolink
Preliminary study nuxt3
Why is nodejs so fast?
论文解读（SR-GNN）《Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training Data》
The verifiable certificate of geoscience remote sensing industry