Experience of redis deepwater area -- Interview reference

background

I have encountered many in my work recently redis The problem of , Just sort it out redis The core basic technology of , I share , Second, you need to understand thoroughly in your work , To solve strange problems .

Redis data structure

 Infrastructure ： character string String、 Dictionaries Hash、 list List、 aggregate Set、 Ordered set SortedSet （zset）.
 If you only know the basic data structure , so to speak redis Only know the fur .

Advanced data structure ：
1. Bitmap ：
The essence can be understood as string , Identify whether each bit of the byte is 0 perhaps 1, For example, it is used to count whether users sign in every day , You can consider using the structure of bitmap .
2. hyperLogLog：
redis Use this data structure Provide an imprecise de counting scheme , The standard error is 0.8% about , It can be used for statistics of a page UV And other allowable error statistics . Compared with Set structure , It can save a lot of space , And then improve redis Overall efficiency .
3. GeoHash：
Provide an application of map coordinates , For example, calculation A And B Distance between ,A、B、C、D… Enter the coordinates after waiting , You can check A The man near the ...
Yes, it is , Circle of friends 、 The function of people nearby can be realized through this structure .

         geoadd   increase 
127.0.0.1:6379> geoadd person 118.48205 40.006794 A
(integer) 1
      geodist  distance 
     127.0.0.1:6379> geodist person A B km
     geopos  To obtain position 
     georadiusbymember  Query other elements near the specified element 
     ......

4. PubSub Publish and subscribe model
   namely Redis Message queue , Generally, a dedicated message queue is used , such as kafka、rocketmq, I won't give you too much introduction here , If you are interested, you can check .

Redis Several pits of distributed lock

People often use redis Distributed lock , Usually by setNx command （ With timeout , Prevent if the release fails , It can automatically expire ）, Only one thread can be set successfully at a time , Successfully returns true, Failure to return false.
pit 1： Master slave switch
As we all know Redis High availability is generally Master-Slave Structural , If Master Hang up , that slave It will be directly mentioned as master, There will be a problem here , If the thread 1 stay setNx Master After writing successfully ,Master All of a sudden , This is the time Slave Become the new Master, but Master The data has not been synchronized to Slave, There will be repeated locking , Business use redis Do distributed locks , It needs to be considered that the fault-tolerant processing in this case can also be manually involved , Repair impact .
But it can be introduce RedLock Algorithm libary, Its general principle is that each lock will send requests to more than half of the nodes , All success is considered success , When deleting , Also delete all locked nodes , Efficiency will be reduced a lot .

pit 2： Timeout problem
If the business logic between locking and unlocking is repeated , Execution is too long , That is beyond the lock expire time, There will be problems . Because the lock is out of date , It's going to get cleaned up , At this moment, the second thread B Hold the lock again , But then the first thread A After executing the relevant business logic , Just release the lock （ In fact, this is the thread B Lock added ）, Third thread C You will get the lock again during the logical execution of the second thread …
There are two ways to avoid this problem :

1. Redis Distributed locks should not be used for tasks with heavy business . If it does happen occasionally , The disorder of data needs to be solved manually , Fix disordered data .
2. More trouble , Need to lock at the same time , Record who added this lock , That is to say setNx When , Put in a value , such as userid, When deleting a lock , Match this value Is it equal to yourself , Release the lock only when it is equal .

Redis Expiration strategy

Redis Itself sets the expiration time of each key Put it in a separate dictionary , Given the nature of single threads , It takes two ways to expire key The harvest of , One is scheduled deletion 、 One is inert deletion .
1. Delete regularly ：
seeing the name of a thing one thinks of its function . Regularly traverse all expired key Dictionary , To delete . But he doesn't delete all every time , Mainly if it expires key quite a lot , according to reids Single thread feature , Don't a lot of overdue scans have to be delayed for a long time .redis Adopted a greedy strategy , Will take out a part first key, such as 20 individual , Look at these key Is it overdue , Clear out expired key, If expired key The ratio of , For example, over 25%, It will be acquired again 20 individual key, To deal with . In order to ensure redis A high performance , Even if it expires key The ratio of , Nor will it cycle indefinitely , Here, each scanning process is also controlled within a certain time （25ms） For details, you can refer to more detailed documents .

2. Lazy deletion
Lazy deletion refers to requesting access to this key When , redis Again Yes key Check the expiration date of , If expired
Delete it immediately . It can be considered that scheduled deletion is handled together , Lazy deletion is a fragmented process .

Understand the expiration policy , You should also pay attention here , about key The expiration date of , Try to avoid a large number of key Expire at the same time , For example, check-in , Generally, everyone likes early morning 24 Point expiration , This is the time , In fact, it can be given to everyone Add a random number after expiration , Stagger expiration .

Redis The secret of high performance

1. Redis It's single threaded
   You may feel strange here , How can single thread be the reason , In fact, the high performance of single thread is not only redis, such as node.js nginx Are representatives of single thread high performance .
   Single thread does not CPU Switch , There is no lock contention problem （ Suppose it is not single threaded redis Of hash set These structures , Just dealing with concurrency takes a lot of time ）, however redis Single thread fast , It's not just that , There are also key factors ,
   First, all data is operated in memory , Extremely fast Two is IO Non blocking IO, Talking about non blocking IO We may think of the operating system select（ Basically abandoned ）,epoll Model 、kqueue etc. , Actually redis The bottom layer depends on it , It's just redis More efficient event handling , For example, put all client instructions into the queue , First come, first serve , In detail, you can have an in-depth understanding .
2. Redis Store small object compression
   What I have to mention here is ziplist Structure , It is a compact byte array structure ,redis Many data structures of, such as zset、hash If it is small, it will pass ziplist Compression implementation , Save memory , The reading rate is also very high .

From the above two points , In fact, if we want to keep redis High performance use of , Large Value The situation of , Big Value Will increase the response delay , So big Value How to deal with the situation ：
Here are two main points ：
1. Split key, According to the business perspective, the order Key Split into multiple Key, Spread to multiple redis In the example
2. Caching complex data structures , Split into a sub item

Redis The persistence of

There are many caches in the market at present, such as memcache etc. , I have to mention that redis Persistence mechanism of .
redis There are two persistence mechanisms ：
1.SNAPSHOT snapshot
bgsave Full volume backup , Here you may have doubts ,redis In response to a request , How to deal with snapshot backup , This is actually redis Meeting fork A subprocess handles this , At the same time, use COW(copy on write) Mechanism for data backup , In short , If the data has not changed, the backup process and redis The same data used by the main process , Once written , that redis The main process will make a copy , Only write in that memory （ In fact, this involves the storage of operating system pages , Take out a page and make a copy , Does not affect the backup process ）, The data backed up by the subprocess remains unchanged at that time , Slowly the main process and Sub process data separation .
2.AOF journal
Incremental backup ,aof Logs are actually instruction texts ,redis Perform backup according to instructions . There is a problem here ,aof In fact, it is written to the memory buffer , Pass regularly fsync Sync back to disk , This regular time is , If there's a outage , Data of the time period that may be lost , such as 1 Sync once a minute , that down One clock of data may be lost in machine hour .

In view of the above characteristics , Generally we will be in redis When the instance is restarted , Use bgsave Persistent file rebuild redis Memory , Reuse aof Replay the recent operation instructions to fully recover the state before restart .

summary

The space problem , The author can't cover all the problems , The right here is to throw and turn the jade , Actually redis The implementation of the basic data structure of is also quite interesting , If you are interested, you can study it in depth .