Manually push a message platform

Design implementation

design scheme

Ready to push material

• Import the data of the message recipient
• Select conditions to pull the data of message recipients from each business line

The first point ： Importing the data of the message receiver can support excel Import , Then go to each business line to get complete users id that will do , The imported data needs to be submitted to the service or dropped in the database in pages

Second point ： Select conditions to pull message receiver data from each business line , First drop the task and its information into the database , It will be difficult to get millions of stores synchronously from the business line , It is designed to be delivered in the form of sequential messages mq Carry out self production and self marketing .

Design sequential consumption mq To pull store data by using customer service screening criteria

The system adopts rocketmq Natural support to ensure sequential delivery （messageQueueSelector), The principle can be referred to This article .

Limit the number of threads consumed to ensure the self-protection of services in the case of large data consumption

Using this method will lead to mq Performance degradation of , If there is a large amount of data and high concurrent consumption ,mq The throughput of the service will decrease , So in order to prevent mq The service has collapsed, and there are not many such data itself , This can be limited topic Number of consuming threads .

consumption mq The message starts to pull store data

adopt mq The task information delivered , Pull the conditions selected by the user to pull the data of the message receiver to be sent from each business line （ At this stage, you only need to pull stores ）. After pulling the store data, all push materials are ready , At this time, users can click start sending or make an appointment to send .

Problems in fetching store data online

Because the conditions of message push center are the conditions of user filtering , And these conditions are some enumeration values , So there is no way to get to the index when getting store data , If you check through these conditions and the amount of data that meets these conditions is very small, it will lead to full table scanning and call interface timeout , Reach... Online 8 second .

Solution

Each business line can pull data from the database through the cursor every time, and then put it into the database redis To filter the data according to the conditions , Then return , In this way, the problem of interface timeout can be solved .

Push message

Two push strategies need to be supported ,mq Push and multithreaded push

Multithreaded push

Use multithreading to push , Pull the recipient's data from the database page , Execute the task of sending messages according to the task submitted to the thread pool , After sending the message, return the number of messages sent , Use future Get the asynchronous execution result of the thread by blocking .

mq Push

Push all message recipients to mq Self marketing , Consumption to mq Push the message to the recipient