“ Keep creating , Accelerate growth ！ This is my participation 「 Nuggets day new plan · 6 Yuegengwen challenge 」 Of the 6 God , Click to see the event details ”

Preface

This paper mainly focuses on redis The principle and implementation of distributed lock based on .

in the future , Again for redis Problems related to distributed locks are well documented .

One 、 background

Tell us why we need distributed locks ？

When multiple threads operate on the same resource at the same time , We usually use, for example synchronized To ensure that only one thread can acquire the object lock and process the resource at the same time . Under distributed conditions , Each service is deployed independently , At this point, the object of the lock service becomes the current application service , That is, other services can still execute this code , Cause service problems , So if we want to deploy multiple services independently , It can also control the independent execution of resources , Then you need to introduce distributed locks to solve this scenario .

What is distributed lock ？

Distributed lock . It is the implementation of a lock that controls different processes in a distributed system to access the same shared resource .

Distributed lock , It can be understood as “ headquarters “ The concept of .

Headquarters to control lock possession and notify other services to wait . Each independent deployment obtains the lock message from the headquarters , Thus avoiding the possibility of local governments . Distributed locking is such an idea .

We can use a third-party component （ for example redis、zookeeper、 database ） Monitor the global lock , Control the holding and release of the lock .

Two 、 Principle and application of distributed lock

setnx yes [set if not exists] Abbreviation .

take key The value of the set value, If and only if key non-existent , If a given key Already exist , be setnx Don't do anything .

Next , Let's use this command to see a simple example of the use of distributed locks , Understand its internal principle .

2.1 Distributed lock evolution ： Stage 1

Code demo as follows ：

        Boolean lock = redisTemplate.opsForValue().setIfAbsent("lock", "111");
        if(lock){
            //  Locking success 
            Map<String, List<Catelog2VO>> stringListMap = getStringListMap();
            redisTemplate.delete("lock");//  Delete lock 
            return stringListMap;
        }else{
            //  Locking failed :  retry  ( spinlocks )
            return getCatalogJsonFromRedis();
        }

problem ：

This is the most primitive lock use problem , But there is obviously a problem here , If the lock is occupied during the execution of business , Program downtime , At this point, the lock will always be redis in .

terms of settlement ：

Sets the expiration time of the lock , Even if it's not deleted , It will be deleted automatically .

2.2 Distributed lock evolution ： Stage 2

Code demo as follows ：

        Boolean lock = redisTemplate.opsForValue().setIfAbsent("lock", "111");
        if(lock){
            redisTemplate.expire("lock",30,TimeUnit.SECONDS);
            //  Locking success 
            Map<String, List<Catelog2VO>> stringListMap = getStringListMap();
            redisTemplate.delete("lock");//  Delete lock 
            return stringListMap;
        }else{
            //  Locking failed :  retry  ( spinlocks )
            return getCatalogJsonFromRedis();
        }

problem ：

setnx Set it up , Set the expiration time again , Downtime occurs in the middle , There will also be deadlocks .

solve ：

The set expiration time and placeholder must be atomic .redis Support use setnx ex command .

2.3 Distributed lock evolution ： Stage 3

        Boolean lock = redisTemplate.opsForValue().setIfAbsent("lock", "111",30,TimeUnit.SECONDS);
        if(lock){
            //  Locking success 
            Map<String, List<Catelog2VO>> stringListMap = getStringListMap();
            redisTemplate.delete("lock");//  Delete lock 
            return stringListMap;
        }else{
            //  Locking failed :  retry  ( spinlocks )
            return getCatalogJsonFromRedis();
        }

You can use a command to complete this locking operation .

problem ：

Is the lock deleted directly ？（ If the business is busy , The lock itself has expired , Let's delete , It may delete the lock that others are holding ）

solve ：

When locked , Value specified as uuid, Everyone matches their own lock before deleting it .

2.4 Distributed lock evolution ： Stage 4

        String uuid = UUID.randomUUID().toString();
        Boolean lock = redisTemplate.opsForValue().setIfAbsent("lock", uuid,30,TimeUnit.SECONDS);
        if(lock){
            //  Locking success 
            Map<String, List<Catelog2VO>> stringListMap = getStringListMap();
            String lockValue = redisTemplate.opsForValue().get("lock");
            if(uuid.equals(lockValue)){
                redisTemplate.delete("lock");//  Delete lock 
            }
            return stringListMap;
        }else{
            //  Locking failed :  retry  ( spinlocks )
            return getCatalogJsonFromRedis();
        }

There is still a problem with this . It is still possible to delete the values of other threads .

Why? ？

If we redis Get to the Value of lock , And passed equal check , Enter the logic of deleting locks , But at this time , The lock has expired , Automatically deleted , And another thread entry occupies the lock , Then there is a problem , The deleted lock is someone else's lock . The reason is the same as before ： During lock comparison and value deletion , It's not an atomic operation .

The solution is to use lua Script to delete .

2.5 Distributed lock evolution ： Stage 5

Use lua Script delete lock .

        String uuid = UUID.randomUUID().toString();
        Boolean lock = redisTemplate.opsForValue().setIfAbsent("lock", uuid,30,TimeUnit.SECONDS);
        if(lock){
            Map<String, List<Catelog2VO>> stringListMap;
            try{
                //  Locking success 
                stringListMap = getStringListMap();
            }finally {
                //  Definition lua  Script 
                String script = "if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) else return 0 end";
                //  Atomic deletion 
                Integer lock1 = redisTemplate.execute(new DefaultRedisScript<Integer>(script, Integer.class), Arrays.asList("lock", uuid));
            }
            return stringListMap;
        }else{
            //  Locking failed :  retry  ( spinlocks )
            return getCatalogJsonFromRedis();
        }

3、 ... and 、 Distributed lock Redission

3.1 Redission brief introduction

Redisson It's a Redis On the basis of implementation Java In memory data grid （In-Memory Data Grid）. It not only provides a series of distributed Java Common objects , There are also many distributed services . These include (BitSet, Set, Multimap, SortedSet, Map, List, Queue, BlockingQueue, Deque, BlockingDeque, Semaphore, Lock, AtomicLong, CountDownLatch, Publish / Subscribe, Bloom filter, Remote service, Spring cache, Executor service, Live Object service, Scheduler service) Redisson Provides the use of Redis The simplest and most convenient way .Redisson The aim is to promote the user's understanding of Redis Separation of concerns （Separation of Concern）, This allows users to focus more on processing business logic .

redission yes redis Officially recommended clients , Provides RedLock Lock of ,RedLock Inherited from juc Of Lock Interface , Provides interrupts 、 Overtime 、 Try to acquire a lock, etc , Support reentry , Mutual exclusion, etc .

3.2 Use

Import dependence

        <!--  Integrate  redission As a distributed lock and other functional framework  -->
        <dependency>
            <groupId>org.redisson</groupId>
            <artifactId>redisson</artifactId>
            <version>3.15.2</version>
        </dependency>

To configure ：

The following is the reference configuration provided on the official website ：

//  Default connection address  127.0.0.1:6379
RedissonClient redisson = Redisson.create();

//  To configure 
Config config = new Config();
config.useSingleServer().setAddress("redis://127.0.0.1:6379");
RedissonClient redisson = Redisson.create(config);

Config config = new Config();
config.useClusterServers()
    .setScanInterval(2000) //  Cluster status scan interval time , In milliseconds 
    // It can be used "rediss://" To enable SSL Connect 
    .addNodeAddress("redis://127.0.0.1:7000", "redis://127.0.0.1:7001")
    .addNodeAddress("redis://127.0.0.1:7002");

RedissonClient redisson = Redisson.create(config);

3.3 Test distributed locks

@Controller
public class TestRedissonClient {
    @Autowired
    RedissonClient redisson;

    @ResponseBody
    @GetMapping("/hello")
    public String hello(){
        // 1、 Get a lock , As long as the name of the lock is the same , It's the same lock 
        RLock lock = redisson.getLock ("my-lock");

        // 2、 Lock 
        lock.lock ();//  Block wait 

        try {
            System.out.println (" Locking success , Executive business ..."+Thread.currentThread ().getId () );
            //  Simulate long wait 
            Thread.sleep (20000);
        } catch (Exception e) {
            e.printStackTrace ( );
        }finally {
            // 3、 Unlock 
            System.out.println (" Release the lock ..."+Thread.currentThread ().getId () );
            lock.unlock ();
        }
        return "hello";
    }
}

redission Two problems have been solved ：

1、 Automatic renewal of locks , If the business is too long , Automatically lock a new one during operation 30s, Don't worry about locking for too long . The lock automatically expired and was deleted .
2、 Locking business , As long as the operation is completed , The current lock will not be renewed , Even if you don't manually unlock , By default, the lock is in 30s Delete later （ The current thread will call before it is destroyed lock Method ）

If lock The timeout time was passed , Just send it to redis Execute the script , Hold the lock , The default timeout is the time we set

If no timeout is specified , Is to use the default open dog time .（30*1000）

As long as the lock is successful , Will start a timed task 【 Reset the expiration time of the lock , The new expiration time is the watchdog's default time 】, every other 10 Second renewal , The renewal period expires .

internaLockLeaseTime【 Watchdog time 】/3 10s .

Best practice ,【 Recommend writing 】

Add a moment , Save time for renewal .

// 10s Auto unlock , The specified time must be greater than the business time （ Otherwise, it will report a mistake , Don't use it if you are not sure ）
lock.lock (10, TimeUnit.SECONDS);

3.4 Read-write lock

Only one thread at a time can occupy the read-write lock of write mode , However, multiple threads can occupy the read-write lock of read mode at the same time .

The read / write lock is suitable for the case that the number of reads to the data structure is more than the number of writes , because , Can be shared when the read mode is locked , Locking in write mode means exclusive , So read-write lock is also called sharing - An exclusive lock .

Make sure you can read the latest data , During revision , Writing lock is an exclusive lock . Read lock is a shared lock ,

If the write lock is not released, you have to wait

Write + read ： Wait for write lock

Write + read ： Wait for the write lock to be released

Write + Write ： Blocking mode

read + Write ： There's a read lock . Writing also needs to wait

As long as there is writing , Have to wait

3.5 Cache consistency issues

How to keep the data in the cache consistent with the database .

3.5.1 Double write mode

After the database changes , Change cache again .

problem ： There will be dirty data
Scheme 1 ： Lock
Option two ： If delay is allowed （ The data updated today will be displayed tomorrow , Or a few minutes and hours of delay ）, Set expiration time （ Suggest ）

3.5.2 Failure mode

The database has been modified , Then delete the cache , Wait for the next active query , Update again .

problem ： There will be reading and writing , Dirty data
Scheme 1 ： Lock
Option two ： If you write often , Read less , It is better to operate the database directly , Remove the cache layer .

3.5.3 programme （ Expiration time + Read-write lock ）

Whether it's dual write mode or failure mode , Will cause cache inconsistencies . That is, if multiple instances are updated at the same time, something will happen . What do I do ?
（1） If it is user latitude data （ Order data 、 User data ), This concurrency rate is very small , Don't think about it , Cache data plus expiration time , Trigger the active update of read at regular intervals
（2） If it's a menu , Basic data such as commodity introduction , You can also use canal subscribe binlog The way .
（3） Cache data + The expiration time is also sufficient to solve the cache requirements of most businesses .
（4） Ensure concurrent reads by locking + Write , Write + Line up in order as you write . It doesn't matter to read . So it's suitable to use read-write lock .( Business is not about heart data , Allow temporary dirty data to be ignored ）

summary :
（1） The data we can put into the cache should not be real-time 、 The requirement of consistency is very high . So when caching data, add expiration time , Make sure you get the latest data every day .
（2） We should not over design , Increase the complexity of the system
（3） When it comes to real-time 、 Data with high consistency requirements , You should check the database , Even if you slow down .

For cache data consistency , We can use Cannal To complete this scenario . Generally for big data projects