Two ways of redis persistence -- detailed explanation of RDB and AOF

RDB

Save the data generation snapshot on disk

command

bgsave：fork Action create subprocess , Short blocking time

Trigger mode ：

1. redis The configuration file /etc/redis.conf Set automatic persistence in

save 900 1 each 900 Every second is modified automatically bgsave
3. When performing a full copy from a node , The master node automatically bgsave Generate RDB The file is sent to the slave node
4. perform debug reload Can also be triggered when save operation
5. By default , perform shutdown when , If it's not on AOF Persistence is performed automatically bgsave Into the RDB Persistence

bgsave Process description ：

1. The parent process determines whether a child process is persisting , Exit if it exists
2. fork operation , After the child process is generated, it returns ：background saving started. The parent process has been released .latest_fork_usec It's the last time fork Operation time , Company （ subtle ）
3. The child process starts to create RDB file ：lastsave You can see the last time save Time for
4. When finished, the child process sends a signal to the parent process , Parent process statistics update info persistence

save： Blocking redis, Has been abandoned

RDB file

1. Storage location ：config get dir; config get dbfilename
2. Compressed storage is recommended to be enabled ：config set rdbcompression yes
3. Check whether the file is damaged ： Tools redis-check-dump

RDB Advantages and disadvantages

Compact binary , Load fast , Applicable to disaster recovery . But using subprocesses is a heavyweight operation and is not suitable for real-time persistence , And the old version is not compatible . So there is. AOF（append only file） Persistence mode

AOF

Record each write command in an independent log , Re executing when restarting . It's solved redis The real-time problem of data persistence .

Turn on use

Turn on ：config set appendonly yes
file name ：config get dir ;config get appendfilename; The default is appendonly.aof

technological process ：

1. All write commands are appended to aof_buf Buffer zone
2. AOF The buffer makes synchronous operation to the hard disk according to the corresponding policy
3. Need to be right aof Files periodically overwrite compressed space
4. redis Load for data recovery when the server restarts

Command write AOF File format

Text protocol format

for example ：set hello world
write in AOF In the file ：*3\r\n$3\r\nset\r\n$5\r\n\hello\r\n$5\r\nworld\r\n

advantage ： Can be read , Easy to modify ; Avoid secondary treatment ; Compatibility is good.

File synchronization policy

By the parameter appendfsync control

1. always： Command write buf After calling the system fsync The operation is synchronized to AOF file , After completion, the thread returns .（ Don't suggest ）
2. everysec： write in aof_buf After the call write The operation returns... After writing to the buffer . Call every second fsync operation .（ Default , At most 2s The data of ）
3. no： write in aof_buf After the call write The operation returns... After writing to the buffer . Synchronization to the hard disk is done by the operating system , Usually 30s.（ Security cannot be guaranteed , Don't suggest ）

fsync operation ： Force synchronization to hard disk , After writing to the hard disk, the thread returns , Ensure persistence .

write operation ： When the file buffer page is written, it returns , When the buffer is full or reaches certain conditions, it is persisted . If an intermediate outage occurs , Data in buffer pages may be lost .

Rewrite mechanism

Convert the data in the process into a write command and write it again AOF file , Compress AOF file size , It can load faster .

Trigger mode ：
Manual trigger ：bgrewriteaof
Automatic triggering ： According to the parameters ：auto-aof-rewrite-min-size（ function AOF When rewriting aof Minimum file size ） and auto-aof-rewrite-percentage（ At present aof The file size aof_current_size And the volume of the last file rewritten aof_base_size The ratio of the ） The parameter determines the trigger time

When aof-current-size>auto-aof-rewrite-min-size
And (aof_current_size - aof_base_size)/aof_base_size>=auto-aof-rewrite-percentage

ps: aof_current_size and aof_base_size adopt info persistence see

Rewrite process ：

1. The parent process checks to see if the fork operation , If not, create a child process
2. The parent process returns in response to other commands , All operation records enter AOF buffer , And according to appendfsync Policy synced to hard disk ; The new data of the child process is written to AOF Rewrite buffer
3. The subprocess writes the command merge rule to the new... According to the memory snapshot AOF file , Each batch write to the hard disk , According to the parameters aof-rewrite-incremental-fsync control , Default 32M.
4. new AOF When the file is written , Parent process update info Statistics
5. The parent process put AOF The data in the file rewrite buffer is written to the new AOF In file
6. Replace old and new files , Finish rewriting .

Restart the loading mechanism

Priority load AOF file , If not, let's see RDB file

File check ：

If the wrong file cannot be loaded, it can be backed up and repaired ：redis-check-aof --fix

Power failure and other faults may cause aof Incomplete writing at the end of the file . Parameters aof-load-truncated To be compatible with this situation , Default on , If you encounter this problem, ignore and continue loading , Print warning log .

Problem optimization and positioning

fork operation

fork The operation will copy the spatial memory page table of the parent process . for example 10G Of redis The process needs to replicate about 20M Memory page table for . so fork The operation time is closely related to the total memory of the process .

fork Time consuming problem location

Under normal circumstances fork Operation per GB Time consuming 20ms about .
Check out the last fork Operating time ：info stats in latest_fork_usec( Company ： Microsecond )

How to improve fork Operating time ：

1. Using a physical machine , Avoid using Xen
2. control redis Instance maximum available memory , Suggest 10G within .
3. Reduce fork frequency , Moderate relaxation AOF Automatic trigger time , Avoid unnecessary full replication .

Subprocess overhead

cpu

1. redis yes cpu Intensive services , Child processes can consume a single core cpu Of 90%, So don't do cpu Single core binding .
2. Ensure that only one child process performs the rewrite

Memory

Child process passed fork Operation produces , The amount of memory used is equal to that of the parent process , In theory, we need 2 Times the memory to complete the persistence operation . however linux There is a copy on write mechanism （copy-on-write）,
The parent and child processes share the same physical memory pages , When the parent process processes the write request, it will create a copy of the modified memory page , The subprocess is in fork Share the whole memory snapshot of the parent process in the process .

Rewrite memory consumption ：
monitor redis journal
RDB rewrite ：copy-on-write Copy memory created by the parent process
AOF rewrite ：copy-on-write Copy memory created by the parent process +AOF Memory occupied by buffer

Memory consumption optimization :

1. Try to ensure that only one child process is working
2. Avoid doing subprocess rewriting when writing a lot , This will cause the parent process to create too many copies of memory pages , Memory consumption is large
3. linux Yes TransparentHugePage（ Default on ）, Big page 2M, When opened, the duplicate page will be copied from 4k Change to 2M, It will greatly increase the memory consumption during rewriting .
   Close the way ：sudo echo never > /sys/kernel/mm/transaparent_hugepage/enabled

Hard disk

Persist to hard disk , Hard disk write pressure , command ：sar/iostat/iotop Analyze the current hard disk load .
How to optimize ：

1. Don't deploy with other services with high hard disk load , Like storage services 、 Message queue
2. Open configuration ：no-appendfsync-on-rewrite, Indicates that it is not enabled during rewriting fsycnc operation
3. When open AOF Functional redis And when writing scenes with high traffic , The throughput of ordinary mechanical disks is 100MB/s. The bottleneck is AOF Sync on hard disk
4. If it is more redis example , Can be AOF Files are stored on different disks , Apportion write pressure

AOF Add blocking

AOF The common hard disk synchronization strategy for persistence is everysec, If the system hard disk resource is busy , Can cause redis Main thread blocking .
Main thread every 1 Second for a comparison , If the time since the last successful synchronization >2s, Blocking the main thread
therefore

1. everysec May lose 2s The data of
2. Busy hard disk resources may lead to redis The main thread is blocked

Problem location ：

1. happen AOF Blocking time ,redis Relevant contents will be output in the log
2. When blocking occurs ,info persistence The statistics aof_delayed_fsync The indicators will add up
3. iotop You can see the high load problem of the hard disk

Reference resources ：《redis Development and operations 》 Fu Lei 、 Written by zhangyijun