Redis fragment cluster

One 、 Fragment cluster structure

Two 、 Build a cluster

A fragmented cluster requires a large number of nodes , Here we set up a minimal fragment cluster , contain 3 individual master node , Every master Contains a slave node , The structure is as follows ：

  Start in the same virtual machine 6 individual redis example , Simulate fragment cluster , The information is as follows ：

1、 preparation

Delete the previous 7001、7002、7003 These directories , Recreate 7001、7002、7003、8001、8002、8003 Catalog ：

#  Get into /tmp Catalog 
cd /tmp
#  Delete the old , Avoid configuration interference 
rm -rf 7001 7002 7003
#  Create directory 
mkdir 7001 7002 7003 8001 8002 8003

stay /tmp Next, prepare a new redis.conf file , The contents are as follows ：

port 6379
#  Turn on the cluster function 
cluster-enabled yes
#  The name of the configuration file for the cluster , We don't need to create , from redis Self maintenance 
cluster-config-file /tmp/6379/nodes.conf
#  Timeout for node heartbeat failure 
cluster-node-timeout 5000
#  Persistent file storage directory 
dir /tmp/6379
#  Binding address 
bind 0.0.0.0
#  Give Way redis Background operation 
daemonize yes
#  Registered instance ip
replica-announce-ip 192.168.1.103
#  Protected mode 
protected-mode no
#  Number of databases 
databases 1
#  journal 
logfile /tmp/6379/run.log

Copy this file to each directory ：

#  Get into /tmp Catalog 
cd /tmp
#  Execute Copy 
echo 7001 7002 7003 8001 8002 8003 | xargs -t -n 1 cp redis.conf

Modify... Under each directory redis.conf, Will be one of the 6379 Change to be consistent with the directory ：  

#  Get into /tmp Catalog 
cd /tmp
#  Modify the configuration file 
printf '%s\n' 7001 7002 7003 8001 8002 8003 | xargs -I{} -t sed -i 's/6379/{}/g' {}/redis.conf

2、 start-up Redis

Because the background startup mode has been configured , So start the service directly ：

#  Get into /tmp Catalog 
cd /tmp
#  Start all services with one click 
printf '%s\n' 7001 7002 7003 8001 8002 8003 | xargs -I{} -t redis-server {}/redis.conf

adopt ps Command view status ：

ps -ef | grep redis

Service started ：

To close all processes , Executable command ：

printf '%s\n' 7001 7002 7003 8001 8002 8003 | xargs -I{} -t redis-cli -p {} shutdown

3、 Build clusters

Although the service started , But at present, each service is independent , No connection whatsoever , Therefore, you need to execute commands to create clusters . stay Redis5.0 It was troublesome to create clusters before ,5.0 Then the cluster management commands are integrated into redis-cli in .

Redis5.0 Before ：

Redis5.0 Before, the cluster command was made by redis Under the installation package src/redis-trib.rb To achieve ; because redis-trib.rb By ruby Language writing, so it needs to be installed ruby Environmental Science ：

#  Installation dependency 
yum -y install zlib ruby rubygems
gem install redis

Then use the command to manage the cluster ：

#  Get into redis Of src Catalog 
cd xxx
#  Create clusters 
./redis-trib.rb create --replicas 1 192.168.150.101:7001 192.168.150.101:7002 192.168.150.101:7003 192.168.150.101:8001 192.168.150.101:8002 192.168.150.101:8003

Redis5.0 after ：

Cluster management and integration into redis-cli in , The format is as follows ：

redis-cli --cluster create --cluster-replicas 1 192.168.150.101:7001 192.168.150.101:7002 192.168.150.101:7003 192.168.150.101:8001 192.168.150.101:8002 192.168.150.101:8003

notes ：

  After the execution of the command ：

  Input yes, Then the cluster starts to be established ：

View the cluster status through the command ：

redis-cli -p 7001 cluster nodes

4、 test

Try to connect 7001 node , Store a piece of data ：

#  Connect 
redis-cli -c -p 7001
#  Store the data 
set num 123
#  Reading data 
get num
#  Store it again 
set a 1

Be careful ： During cluster operation , Need to give redis-cli add -c Parameters only

3、 ... and 、 Hash slot

What is a hash slot ：

Redis I'm going to take each of them master Nodes map to 0~16383 common 16384 Slots （hash slot） On , When you view the cluster information, you can see ：

In a shard cluster , data key Not bound to a node , Instead, it is bound to the slot .redis Will be based on key Calculates the slot value for the valid part of the , There are two situations ：

for example ：key yes num, So based on num Calculation , If it is {itcast}num, According to itcast Calculation . The calculation method is to use CRC16 The algorithm gets a hash value , Then on 16384 Remainder , The result is slot value .

summary ：

Four 、 Cluster scaling

redis-cli --cluster Provides a lot of commands for cluster operation , You can view ：

For example, the command to add nodes ：

5、 ... and 、 Fail over

When there is one in the cluster master What happens to downtime ？ First, the instance loses connection with other instances , Then there is a suspected outage ：

Finally, make sure to go offline , Automatically raise one slave For the new master：

Data migration ：

meanwhile , We can also use it cluster failover The command can manually make one of the clusters master Downtime , Switch to execution cluster failover This of the command slave node , Realize data migration without perception ：

be 7002 From the beginning slave node ：

Turn into master node ：

Implementation process ：