Deep parsing and implementation of redis pub/sub publish subscribe mode message queue

Keep creating , Accelerate growth ！ This is my participation 「 Nuggets day new plan · 6 Yuegengwen challenge 」 Of the 23 God , Click to see the event details

In detail Redis Of Pub/Sub Related commands and advantages and disadvantages of , And how to implement a simple message queue .

1 Pub/Sub Overview

We can use Redis Of List The data structure implements one

A simple message queue , adopt lpush Command to write messages , adopt rpop Command pull message , You can also use BRPOP Implement the blocking pull message .

The message queue above has a drawback , That's it Message multicast is not supported Mechanism , The message multicast mechanism is that a message produced by a producer can be consumed by multiple consumers , This function is very important in distributed systems .

Redis Use alone Pub/Sub Module to support message multicast , That's the release / A subscription model (publish/subscribe), It is a message communication mode ： Publisher (pub) Send a message , subscriber (sub) receive messages .

Publishers will publish their messages to a chanel（ passageway ） Instead of sending it to the specified subscriber , Publishers also don't know which subscribers they might have .

Subscribers can subscribe to one or more channel, Only receive from subscriptions channel The news of , And I don't know what （ If there is ） Publisher , This pattern decouples the publisher from the subscriber .

Pub/Sub Independent of key space , Messages are not persisted , It has nothing to do with the database , stay db10 Publish on , Will be able to be db1 Subscribers on hear . If we need a range of ranges , Then you can only set the channel Make a distinction in the name .

2 subscribe

Client side usage SUBSCRIBE channel [channel ...] Command subscription channel , You can execute the command multiple times , You can also subscribe to multiple channels at one time , Multiple clients can subscribe to the same channel .

This command returns an array , Including three parts , In turn, is : Command name ( character string “subscribe”), The channel name of the subscription , The total number of channels subscribed at present （ contain glob passageway ）. These three parts are continuous for each subscribed channel .

After the client executes the subscription , In addition to continuing to subscribe (SUBSCRIBE perhaps PSUBSCRIBE), Unsubscribe (UNSUBSCRIBE perhaps PUNSUBSCRIBE), PING Command and end connection (QUIT) Outside , No other operations can be performed , The client will block until a message published on the subscription channel arrives .

as follows , Indicates that the client subscribes to four channels at a time ：aaa、bba、ccc、ddd：

127.0.0.1:6379> SUBSCRIBE aaa bba ccc ddd
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "aaa"
3) (integer) 1
1) "subscribe"
2) "bba"
3) (integer) 2
1) "subscribe"
2) "ccc"
3) (integer) 3
1) "subscribe"
2) "ddd"
3) (integer) 4

Please note that , If you use redis-cli Once you enter subscription mode, you will not accept any commands , Only use Ctrl-C Exit this mode .

3 Unsubscribe

Client side usage UNSUBSCRIBE [channel [channel ...]] Command to unsubscribe from the specified channel , You can specify one or more unsubscribed channel names , You can also take no parameters , At this time, all subscribed channels will be cancelled ( barring glob passageway ).

This command returns an array , Including three parts , In turn, is : Command name ( character string “unsubscribe”), The channel name of the subscription , The total number of channels subscribed at present （ contain glob passageway ）. These three parts are continuous for each unsubscribed channel . When the last parameter is zero , We no longer subscribe to any channels , The client can issue any type of Redis command , Because we are in Pub/Sub Out of state .

as follows , Indicates that the client exits ccccc Channel subscription ：

127.0.0.1:6379> UNSUBSCRIBE ccccc
1) "unsubscribe"
2) "cccc"
3) (integer) 0

4 Pattern matching

Redis Pub/Sub Implementation supports pattern matching . Clients can subscribe glob passageway , This will receive all messages sent to the channel whose channel name matches the given pattern .

Client side usage PSUBSCRIBE pattern [pattern ...] Subscribe to one or more glob passageway .

This command returns an array , Including three parts , In turn, is : Command name ( character string “psubscribe”), Subscribe to the glob Channel name , The total number of channels subscribed at present ( Include non glob passageway ). These three parts are continuous for each subscribed channel .

for example , subscribe a* and *c Pattern ：

127.0.0.1:6379> PSUBSCRIBE a* *c
Reading messages... (press Ctrl-C to quit)
1) "psubscribe"
2) "a*"
3) (integer) 1
1) "psubscribe"
2) "*c"
3) (integer) 2

Client side usage PUNSUBSCRIBE [pattern [pattern ...]] Unsubscribe from one or more glob passageway . You can also take no parameters , At this time, all subscribed channels will be cancelled ( Excluding non glob passageway ).

This command returns an array , Including three parts , In turn, is : Command name ( character string “punsubscribe”), Unsubscribed glob Channel name , The total number of channels subscribed at present （ Include non glob passageway ）. These three parts are continuous for each unsubscribed channel . When the last parameter is zero , We no longer subscribe to any channels , The client can issue any type of Redis command , Because we are in Pub/Sub Out of state .

as follows , Cancel right a* Of glob Channel subscription ：

127.0.0.1:6379> PUNSUBSCRIBE a*
1) "punsubscribe"
2) "a*"
3) (integer) 0

subscribe, unsubscribe, psubscribe and punsubscribe At the end of the command, all the data subscribed by the current client are returned glob Channels and total number of channels , If 0, The client will exit automatically Pub/Sub Pattern .

5 Release

PUBLISH channel message Command to publish messages on the specified channel . Messages can only be published on one channel , You cannot publish messages on multiple channels at the same time .

Number of recipients who will return the notification . The number of recipients here is greater than or equal to the number of clients subscribing to the channel , Because a client glob Channels and non channels glob If the channel matches the publishing channel at the same time , It is regarded as two recipients . let me put it another way , If the client subscribes to multiple patterns that match the published message , Or subscribe to the pattern and channel matching the message , The client may receive the same message multiple times .

At the receiving end , The response received consists of three parts , In turn, is ：“message” character string , Matching channel name , The content of the news . If it's because glob Pattern matching and receiving , Then return to four parts ：“pmessage” character string , Matching glob Channel name , Original channel name sent , The content of the news .

If a client's subscription a* and *c Two mode channels ：

127.0.0.1:6379> PSUBSCRIBE a* *c
Reading messages... (press Ctrl-C to quit)
1) "psubscribe"
2) "a*"
3) (integer) 1
1) "psubscribe"
2) "*c"
3) (integer) 2

If the channel sending the message is ac, So it's going to return 2：

127.0.0.1:6379> PUBLISH ac xxxxx
(integer) 2

On the client side , Will receive two messages ：

1) "pmessage"
2) "a*"
3) "ac"
4) "xxxxx"
1) "pmessage"
2) "*c"
3) "ac"
4) "xxxxx"

6 Pub/Sub principle

Every Redis The server process maintains a that identifies the state of the server redis.h/redisServer structure , Among them, there are channels with subscriptions as well as A subscription model Information about ：

struct redisServer {
    // ...
    dict *pubsub_channels;  //  Subscribed Channels 
    list *pubsub_patterns;  //  A subscription model 
    // ...
};

6.1 pubsub_channels

pubsub_channels It's a dict Dictionary structure ,key（ Array elements ） by channel,value It's a certain one. client. When the client subscribes to a channel ,Redis Would be to pubsub_channels A new entry has been added to the dictionary channel and client data , Different client You can subscribe to the same channel,client Linked in a linked list , In this way, you can save multiple client To the same channel The relationship between , Very clever .

Understand the structure ,SUBSCRIBE 、PUBLISH 、UNSUBSCRIBE The implementation of the command is also very simple .

SUBSCRIBE Will be channel and client Add to dict in , If you have not done so before channel, Then add a new channel Elements , Then add another one client List nodes , If it existed before , Add a... At the end of the linked list client node .

PUBLISH You only need to locate the specific... Through the above dictionary channel, You can find all the subscriptions channel The client of , Just send them the message again .

UNSUBSCRIBE It's also very simple. , To the corresponding channel In the following linked list client Delete it .

6.2 pubsub_patterns

pubsub_patterns Used to store all glob channel, It's a list structure , The node type is redis.h/pubsubPattern：

typedefstruct pubsubPattern {
    redisClient *client;  //  Subscription mode client 
    robj *pattern;        //  Subscription mode 
} pubsubPattern;

When using PSUBSCRIBE When the command subscribes to a pattern , The program creates a pubsubPattern Add to pubsub_patterns In the list . If another client also subscribes to a schema , Add a... To the back of the linked list pubsubPattern The node can be .

therefore , actually PUBLISH Except in pubsub_channels To locate specific channel outside , The specified channel And pubsub_patterns Compare the patterns in , If designated channel If it matches a pattern , Then it will be message Send to all clients subscribed to that mode .

PUNSUBSCRIBE And it's very simple , Is deleted pubsub_patterns in ,client and pattern Nodes with consistent information comparison .

7 Pub/Sub shortcoming

The news released in Redis Cannot persist in the system , therefore , The subscription must be executed first , Wait for the news release . If the news is released first , The message has no subscribers , Messages will be discarded directly .

Just send messages , Regardless of reception , either ACK Mechanism , Message consumption cannot be guaranteed . If a consumer joins in halfway , Or hang up and restart , Then the lost messages can not be consumed again .

The above shortcomings lead to Redis Of Pub/Sub The pattern is like a little toy , There is little use in the production environment , Very embarrassed ！ So ,Redis5.0 Version added Stream data structure , Not only does it support multicast , It also supports data persistence , comparison Pub/Sub More powerful ！

Related articles ：

1. redis.io

If you need to communicate , Or the article is wrong , Please leave a message directly . In addition, I hope you will like it 、 Collection 、 Focus on , I will keep updating all kinds of Java Learning blog ！