Cluster chat server: summary of project problems

1. Briefly describe your project

This project is a network server project , It is divided into the following modules .

1. The first is the network module , It's using muduo Network library to design , The advantage of the network library is that it decouples the network module code and the business module code , Enable developers to focus on business development .
2. The business layer uses some C++11 Some of the technology , such as map、 Binder, etc , It's mainly a message ID And the binding of a callback operation after this message occurs , Callback mechanism . When the network I/O When you spit out a message request for me , According to the request Json, Get the news ID, Then process the message .
3. The data storage layer uses MySQL, Some key data of the project have been landed , For example, the user's account 、 offline message , Group messages, etc , Are stored in the database . And realize the database connection pool , Save a lot of three handshakes and four waves , Improve the efficiency of accessing the database .

The above is the basic module of single server , But the concurrency ability of single server is limited , So deploy multiple servers , We need to configure Nginx TCP Load balancing （ Because it's a long connection , So it's based on TCP Of ）.

Each server has different user registrations , If users of these different servers want to communicate , The project introduced redis Release - subscribe .

2. Security of data plaintext transmission

In this project , It uses HTTP Plaintext transmission , Then the problem of ensuring security is to encrypt .

Symmetric encryption
Asymmetric encryption

Symmetric encryption ： The encryptor and the decryptor use the same key key.
Before sending, the sender encrypts the plaintext data to generate ciphertext , The decryptor uses the key to restore it to plaintext .

The advantage is high efficiency , for example AES Encryption and decryption algorithm

Asymmetric encryption ： There are public key and private key , And they are matched one by one .
Encryption is complicated , Slow efficiency , But it's safe . for example RSA encryption algorithm .

Concrete realization ：
Because asymmetric encryption is inefficient , So mixed encryption .
Generate public and private keys on the client and server when writing code , Then the client and server shake hands three times to establish a connection , Client side utilization RSA The public key encrypts symmetrically AES The key is encrypted into ciphertext , After sending to the server, the server uses RSA Private key decryption , Get this AES Private key , Then reply to the client with OK, Both client and server have AES Private key , Then in the subsequent data transmission , You can use it AES Symmetric encryption is used to encrypt plaintext .

Another problem is that so many customers need to connect to the server , Then each client must generate with the server AES secret key , This requires the server to store the key of each client in the database , Every time data is transmitted , In order to find the corresponding key easily , You can only put important parts of the data message body To encrypt , and userid No encryption , The server can userid Find the corresponding key .

3. How client messages are displayed in order

There will be this problem in the communication process , namely client1 Send to client2 The news of , Will not arrive in order , Because different routing nodes may be passed during transmission . Example ： The news is out of order

Solution ：

Add a serial number to each message seq, The receiver maintains the message sequence number of each friend seq.

Example ：
The receiver maintains the messages of each friend seq, Initialize to 0, When a message is received , View the seq, If not 0, Cache the current message , Wait until the serial number is 0 After receiving the message of , For maintaining friends seq++, Then check the cached messages .

4. server How the client perceives the state of the client

If the network is seriously blocked ,ChatServer How does the client perceive whether the client is online or offline ？

Solution ： heartbeat

Let the heartbeat business bind UDP8080 Port no. ,server Start a heartbeat timer at the end , The client sends a message to the server heartbeat Message type , Just reduce the timer of the user by one , If you time out, add one , In general , The value of the timer will be （-1, 0, 1） Change between .
If a client Your heartbeat timer has exceeded 5, It is judged that client The line has dropped , Then release this client All connections and other resources .

In general , For this kind of chat server, it is a long connection , Will design a heartbeat mechanism in the business layer .

Mention this heartbeat mechanism , There is another problem TCP Agreed keepalive Mechanism .

Example ： stay client And server establish TCP After connection , But the two sides did not transmit data , that server How to know client The state of ,client Is it a drop in line , Or is the connection normal, but no data transmission , Or is the link disconnected ？

stay Linux in , We can enable a through code socket The heartbeat of .

// on by 1 Time means open keepalive Options , The default is 0
int on = 1;
setsockopt(fd, SOL_SOCKET, SO_KEEPALIVE, &on, sizeof(on));

but keepalive The default time interval for sending heartbeat detection is 7200s, Time is too long , Not practical .
You can set three related options to change this time interval , Namely ：

1. TCP_KEEPIDLE
2. TCP_KEEPINTVL
3. TCP_KEEPCNT

Example ：

//  send out keepalive Time interval of message 
int val = 7200;
setsockopt(fd, IPPROTO_TCP, TCP_KEEPIDLE, &val, sizeof(val));

//  The time interval between two retry messages 
int interval = 75;
setsockopt(fd, IPPROTO_TCP, TCP_KEEPINTVL, &interval, sizeof(interval));

int cnt = 9;
setsockopt(fd, IPPROTO_TCP, TCP_KEEPCNT, &cnt, sizeof(cnt));

TCP_KEEPIDLE Option to send keepalive Time interval of message , When sending, if the opposite end replies ACK, Then this end TCP The protocol stack thinks that the connection is still alive , Continue to wait for 7200s Send later keepalive message ; If the peer replies RESET, It means that the peer process has been restarted , The local application should close the connection .

If there is no reply from the opposite end , Then retry at this end , If you try again 9 Time （TCP_KEEPCNT） Still not reachable , The interval between retries is 75s（TCP_KEEPINTVL）, Return... To the application TIMEOUT（ No response ） or EHOST error message .