Consumer of microservices

today , Just reviewed , Let me share with you today consul Construction and related use of ！

First , We need to know before using consul What is it? , For what ？

One 、 summary

consul yes google A use of open source go Service discovery for language development 、 Configuration Management Center Service . Built in service registration and discovery box   frame 、 Distributed consistency protocol implementation 、 health examination 、Key/Value Storage 、 Multi-data center solution , No need to rely on other tools （ such as ZooKeeper etc. ）. Service deployment is simple , There is only one runnable binary package . Each node needs to run agent, He has two modes of operation server and client. Each data center official recommendation requires 3 or 5 individual server Node to ensure data security , At the same time guarantee server-leader The election can be carried out correctly .

If you are familiar with zk My classmates should be right consul The related operations of can be started very soon .

In fact, we often use it for service discovery ：

Learn with questions , What is service discovery ？

In the framework of micro Services , Service discovery is a necessary module . I believe that children's shoes who know or are familiar with microservice should know its importance . I'm just going to mention , After all, that's not our point . Let's take a look at the picture below ：

In the figure , An interface for the client , Need to call service A-N. The client must know the network location of all services , In the past, configuration is in the configuration file , Or some configuration in the database . Here are just a few questions ：

• Need configuration N A network location for a service , Increase the complexity of configuration

• The network location of the service changes , You need to change the configuration of each caller

• In the case of clusters , Hard to load （ Except for the way of reverse proxy ）

In one sentence ： More services , Configuration is a hassle , There are many problems

Since there are these problems , So service discovery is the solution to these problems . Words , How to solve it ？ Let's look at another picture

What's different from the previous one is , Added a service discovery module . The picture is simple , Here's the text . service A-N Register your current network location to the service discovery module （ Register here means to tell ）, Service discovery is based on K-V In the same way ,K It's usually a service name ,V Namely IP:PORT. The service discovery module polling regularly to see if these services can be accessed （ This is the health check ）. The client is calling the service A-N When , Just run to the service discovery module and ask their network location , And then call their services . Is this the way to solve the above problem ？ The client does not need to record these service network locations at all , The client and server are completely decoupled ！

This process is basically like this , Of course, the service discovery module is not so simple . There are many things in it . It's just easy to understand .

The service discovery module in the figure is basically the role of service discovery in microservice architecture .

consul  brief introduction

Common frameworks for service discovery include

• zookeeper

• eureka

• etcd

• consul

There is no comparison between the better and the worse , The children's shoes they need are Google Baidu .

that consul What is it ？consul It's a tool for service discovery . And then here's a brief introduction ：

consul It's distributed 、 High availability 、 Laterally extended .consul Some of the key features provided ：

service discovery：consul adopt DNS perhaps HTTP Interface makes service registration and service discovery easy , Some external services , for example saas You can also register what you provide .
health checking： Health tests make consul It can quickly alarm the operation in the cluster . Integration with service discovery , It can prevent the service from forwarding to the failed service .
key/value storage： A system for storing dynamic configuration . Provide simple HTTP Interface , It can be operated anywhere .
multi-datacenter： No complex configuration required , Can support any number of areas .

We'll talk about service discovery here , health examination , There are also some basic KV Storage . Multiple data centers have a chance to talk about it in another article .

summary ： Just know that it's about solving the problems I raised in the last part , And the rest of it is understood

consul  A few concepts

The picture above is from consul Official documents .

We just look at the data center 1, It can be seen that consul The cluster of N individual SERVER, add M individual CLIENT Composed of . whether SERVER still CLIENT, All are consul A node of , All services can be registered on these nodes , It is through these nodes that service registration information is Shared . Except for these two , There are also some small details , A brief introduction .

• CLIENT

CLIENT Express consul Of client Pattern , That's the client mode . yes consul A pattern of nodes , In this mode , All services registered to the current node are forwarded to SERVER, It's not persistent in itself .

• SERVER

SERVER Express consul Of server Pattern , Indicates that the consul It's a server, In this mode , Function and CLIENT Are all the same , The only difference is , It will persist all the information locally , So you have a failure , Information can be retained .

• SERVER-LEADER

The middle one SERVER There is LEADER Wording , Indicates that the SERVER It's their boss , It and other SERVER The difference is , It needs to be responsible for synchronizing the registered information to others SERVER, Also responsible for the health monitoring of each node .

• Other information

Other information includes the way they communicate , And some protocol information , Algorithm . They are used to ensure data synchronization between nodes , Real time requirements and a series of cluster problems . Look at the official documents for those interested .

consul  Basic use

I have only one machine , So here's a demonstration docker The lower part is used consul. The port mapping between container and host is ignored , One for each host in the normal production environment consul, The port needs to be mapped to the host

Two 、consul Cluster building

1） install

First, go to the official website. Now it's appropriate consul package ：https://www.consul.io/downloads.html

Install direct download zip package , After decompression, there is only one executable file consul, take consul Add it to the environment variables of the system .

#unzip consul_1.2.3_linux_amd64.zip

#cp -a consul  /usr/bin

#consul

Usage: consul [--version] [--help] <command> [<args>]

Available commands are:
 agent Runs a Consul agent
 catalog Interact with the catalog
 connect Interact with Consul Connect
 event Fire a new event
 exec Executes a command on Consul nodes
 force-leave Forces a member of the cluster to enter the &quot;left&quot; state
 info Provides debugging information for operators.
 intention Interact with Connect service intentions
 join Tell Consul agent to join cluster
 keygen Generates a new encryption key
 keyring Manages gossip layer encryption keys
 kv Interact with the key-value store
 leave Gracefully leaves the Consul cluster and shuts down
 lock Execute a command holding a lock
 maint Controls node or service maintenance mode
 members Lists the members of a Consul cluster
 monitor Stream logs from a Consul agent
 operator Provides cluster-level tools for Consul operators
 reload Triggers the agent to reload configuration files
 rtt Estimates network round trip time between nodes
 snapshot Saves, restores and inspects snapshots of Consul server state
 validate Validate config files/directories
 version Prints the Consul version
 watch Watch for changes in Consul

Input consul, The above content indicates that the installation was successful .

 

2） start-up

consul Must be started agent Can be used , There are two startup modes server and client, There is also an official one ui.server Use and persist service information , Cluster official recommendations 3 or 5 Nodes .client Used only with server Interaction .ui You can view the cluster status .

server：

cn1：

#consul agent  -bootstrap-expect 2  -server   -data-dir /data/consul0 -node=cn1 -bind=192.168.1.202 -config-dir /etc/consul.d -enable-script-checks=true  -datacenter=dc1 

cn2:

#consul agent    -server  -data-dir /data/consul0 -node=cn2 -bind=192.168.1.201 -config-dir /etc/consul.d -enable-script-checks=true  -datacenter=dc1  -join 192.168.1.202

cn3:

#consul agent  -server  -data-dir /data/consul0 -node=cn3 -bind=192.168.1.200 -config-dir /etc/consul.d -enable-script-checks=true  -datacenter=dc1  -join 192.168.1.202

Parameter interpretation ：

-bootstrap-expect: The number of nodes expected by the cluster , Only when the number of nodes reaches this value can the election be made leader.

-server：  Running on the server Pattern

-data-dir： Specify the data directory , Other nodes must have read permission to this directory

-node： Specify the name of the node

-bind： Bind an address for the node

-config-dir： Specify profile , Define the , Default all one .json The end of the file will be read

-enable-script-checks=true： Set the check service to available

-datacenter:  The data center has no name ,

-join： Join an existing cluster

 

client：

#consul agent   -data-dir /data/consul0 -node=cn4 -bind=192.168.1.199 -config-dir /etc/consul.d -enable-script-checks=true  -datacenter=dc1  -join 192.168.1.202

client Nodes can have more than one , You can specify according to the service .

ui:

#consul agent  -ui  -data-dir /data/consul0 -node=cn4 -bind=192.168.1.198  -client 192.168.1.198   -config-dir /etc/consul.d -enable-script-checks=true  -datacenter=dc1  -join 192.168.1.202

 -ui: Use native ui,

-ui-dir： Appoint ui The catalog of , Use your own definition of ui

-client： Appoint web  ui、 The monitoring address of , Default 127.0.0.1 Native access only .

After the cluster is created ：

Use some common commands to check the status of the cluster ：

#consul  info

Can be in raft：stat See the status of this node is Fllower perhaps leader

#consul members
Node Address Status Type Build Protocol DC Segment
cn1 192.168.1.202:8301 alive server 1.0.2 2 dc1 <all>
cn2 192.168.1.201:8301 alive server 1.0.2 2 dc1 <all>
cn3 192.168.1.200:8301 alive client 1.0.2 2 dc1 <default>

There are several ways to add a new node ;

1、 This way, , It will not automatically join the cluster after restart

#consul join 192.168.1.202

2、# Use at startup -join Specify a cluster

#consul agent -ui -data-dir /data/consul0 -node=cn4 -bind=192.168.1.198 -config-dir /etc/consul.d -enable-script-checks=true -datacenter=dc1 -join 192.168.1.202

3、 Use -startjoin or -rejoin

#consul agent -ui -data-dir /data/consul0 -node=cn4 -bind=192.168.1.198 -config-dir /etc/consul.d -enable-script-checks=true -datacenter=dc1 -rejoin

visit ui：

http://192.168.1.198:8500/ui

port ：

8300：consul agent service relplaction、rpc（client-server）

8301：lan gossip

8302：wan gossip

8500：http api port

8600：DNS Service port

 

3） Service registration

The configuration file is used ,（ The official recommendation ） First, create a directory to store the configuration files that define the service

#mkdir /etc/consul.d/

When starting the service, use -config-dir  Parameter assignment .

Here is a service definition ：

#cat web.json

{
 "service":{
 "name":"web",
 "tags":[
 "rails"
 ],
 "port":80,
 "check":{
 "name":"ping",
 "script":"curl -s localhost:80",
 "interval":"3s"
 }
 }
}

If you start like this consul after , Will find consul There are always errors in my log , Because we didn't start 80 Service of port , Here's how go Write a program ：

package main
import (
 "io"
 "log"
 "net/http"
 "strconv"
 "fmt"
)
var iCnt int = 0;

func helloHandler(w http.ResponseWriter, r*http.request) {
 iCnt++;
 str :="Hell eorld ! friend("+ strconv.Itoa(iCnt)+")"
 io.WriteString(w,str)
 fmt.Println(str)
}

func main(){
 ht :=http.HanderFunc(helloHandler)
 if ht != nil {
 http.Handle("/hello",ht)
 }
 err := http.ListenAndServe(":80",nil)
 if err != nil{
 log.Fatal("ListenAndserve:",err.Error())
 }
}

# Need one goalong Environment ：

#go build -o web web.go
#./web

At this point, you can run without web The service is executed on the machine DNS Inquire about ：


# dig @127.0.0.1 -p 8600 web.service.consul SRV
;; ANSWER SECTION:
web.service.consul. 0 IN SRV 1 1 80 cn2.node.dc1.consul.
web.service.consul. 0 IN SRV 1 1 80 cn3.node.dc1.consul.
;; ADDITIONAL SECTION:
cn2.node.dc1.consul. 0 IN A 192.168.1.201
cn2.node.dc1.consul. 0 IN TXT "consul-network-segment="
cn3.node.dc1.consul. 0 IN A 192.168.1.200
cn3.node.dc1.consul. 0 IN TXT "consul-network-segment="
;; Query time: 17 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN:  Four  1 month  04 14:39:32 CST 2018
;; MSG SIZE rcvd: 229

You can see that the service has been registered in the cluster . 

Use dns Inquire about , Default domain name format NAME.service.consul,NAME Namely web.json Internally defined service Of name. You can specify your own domain and port ：-domain、-dns-port 53

For ease of use consul The registration of the cluster uses , So I wrote a three node client Registration script for , Facilitate unified registration service and management . And used it nfs, Share the service files to the cluster .

#!/usr/bin/env python
#encoding: utf8
#decription: registered a service to consul
from subprocess import call
import sys

hosts = {"b1":"10.10.1.01:8500","b2":"10.10.7.1:8500","b3":"10.10.8.21:8500"}


def consul_relaod():
 if (len(sys.argv) != 2) or sys.argv[1] == '-h':
 print(("Usage: {0} [option] :{1} ,if you wang update all of them,you must use 'all'").format(sys.argv[0],hosts.keys()))
 sys.exit()

 elif(sys.argv[1] == 'all'):
 for i in hosts.keys():
 call(("consul reload -http-addr {}").format(hosts[i]),shell=True)
 else:
 call(("consul reload -http-addr {}").format(hosts[sys.argv[1]]),shell=True)

if __name__ == '__main__':
 consul_relaod()

hosts yes client The node list . You can register only one of the nodes , Input hosts Corresponding key, You can also type all, Register to all nodes ;nfs What we share is /etc/consul.d Catalog .

4） health examination

check Use to do health checks for services , You can have multiple , You can also check without supporting multiple methods .check Must be script perhaps TTL type , If it is TTL Type is ttl Variables must be provided .script yes consul Take the initiative to check the health of the service ,ttl It's the service that takes the initiative to consul Report your situation . The new version of the consul Not in use script To express , Use args, If it is https Health check of service , have access to args This script is implemented , because consul Not supported by default https Health check .

script check：
"check": {
 "args": ["/data/scripts/kubeadm-ha-0.sh",""],
 "interval": "10s"
}
 
http check:
{
 "check": {
 "id": "api",
 "name": "HTTP API 500",
 "http": "http://loclhost:500/health",
 "interval": "10s",
 "timeout": "1s"
}
}
 
tcp check:
{
 "check": {
 "id": "ssh",
 "name": "ssh TCP 26622",
 "tcp": "localhost:26622",
 "interval": "10s",
 "timeout": "1s"
 }
}
 
ttl check:
{
 "check": {
 "id": "web-app",
 "name": "Web APP status",
 "notes": "Web APP does a curl internally every 10 seconds",
 "ttl": "30s"
}
}

3、 ... and 、 to update consul Version is the latest version 1.2.3.

Version update feature ：

https://github.com/hashicorp/consul/blob/v1.2.3/CHANGELOG.md

ui This is a big change from before ：

 

 

It's a little long , Take your time