How to use rdbtools to analyze redis large keys

Big key Hazards of

be familiar with redis Our friends all know , Big key It is a risk that should be avoided in the design and practice of the application . Big key There are many dangers , for example ：

• Causing traffic to reach a bottleneck
• Cause the tilt of the partition capacity
• What is more serious is reading and writing key Cause the system to be unresponsive , Generate a large number of slow queries and so on

Generally we think that it is more than 10kb The size of string Type of key Or more than 1 Ten thousand elements set It can be called big key, A reasonable split should be made

Big key Acquisition

How to get large key For those who use cloud database , Is a relatively easy thing , Because many cloud manufacturers provide large by default key Analysis tools , For example, Tencent cloud database is in 【 Console 】【 System monitoring 】【 Monitoring Overview 】 The page provides a large key Analysis function , The principle is to analyze the static state RDB The file is then pulled out of it key Sort by size , The focus of this article is to share how to use open source tools rdbtools Carry on the big key analysis

rdbtools Installation

rdbtools There are three main functions

• Analyze static rdb File and generate csv Format memory report
• take rdb File dump becomes json Format
• utilize diff The tool compares two rdb The difference of documents Now let's start

1. The first step is to install python and pip

Because the author's operating system uses centos8.0, By default python3 and pip3 So there is no need for additional installation , Readers can install by themselves , There are many online tutorials , I won't repeat .

>>> import sys
>>> sys.version
'3.6.8 (default, May 21 2019, 23:51:36) \n[GCC 8.2.1 20180905 (Red Hat 8.2.1-3)]'
>>> 

2. obtain rdb file

build by oneself redis Students can use redis-port Toolset export rdb file . Friends who use cloud database can contact the cloud manufacturer to get the download link .

Tencent cloud redis The console gets the backup download link

Export directly to local or use rz/sz Import into your own machine

[[email protected] ~]# yum install lrzsz
Last metadata expiration check: 0:10:21 ago on Thu 18 Mar 2021 04:31:25 PM CST.
Dependencies resolved.
========================================================================================================================================================================================
 Package                                  Arch                                      Version                                             Repository                                 Size
========================================================================================================================================================================================
Installing:
 lrzsz                                    x86_64                                    0.12.20-43.el8                                      BaseOS                                     84 k

Transaction Summary
========================================================================================================================================================================================
Install  1 Package

Total download size: 84 k
Installed size: 190 k
Is this ok [y/N]: y
Downloading Packages:
lrzsz-0.12.20-43.el8.x86_64.rpm                                                                                                                         2.2 MB/s |  84 kB     00:00    
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                   2.1 MB/s |  84 kB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                1/1 
  Installing       : lrzsz-0.12.20-43.el8.x86_64                                                                                                                                    1/1 
  Running scriptlet: lrzsz-0.12.20-43.el8.x86_64                                                                                                                                    1/1 
  Verifying        : lrzsz-0.12.20-43.el8.x86_64                                                                                                                                    1/1 

Installed:
  lrzsz-0.12.20-43.el8.x86_64                                                                                                                                                           

Complete!

3. install rdbtools

[[email protected] ~]# pip install rdbtools

After installation, pass help Simply understand the command , Be careful --command or -c Is a required parameter

[[email protected] ~]# rdb --help
usage: usage: rdb [options] /path/to/dump.rdb

Example : rdb --command json -k "user.*" /var/redis/6379/dump.rdb // From dump.rdb Analyze all the data in the file `user.` At the beginning key And the output is jason Format , Print it to the screen 

positional arguments: // Positional arguments 
  dump_file             RDB Dump file to process

optional arguments:
  -h, --help            show this help message and exit// View help information for commands 
  -c CMD, --command CMD// Required parameters ,-c json Indicates stored as json Format , The most common is  -c memory To generate csv Format memory snapshot , also diff Pattern comparison 
                        Command to execute. Valid commands are json, diff,
                        justkeys, justkeyvals, memory and protocol
  -f FILE, --file FILE  // output to a file 
                        Output file
  -n DBS, --db DBS      // Specifies that only a number of db, for example db0, If you do not specify that all... Are included by default 
                        Database Number. Multiple databases can be provided.
                        If not specified, all databases will be included.
  -k KEYS, --key KEYS   // Specifies the... That analyzes and exports a prefix key, regular expression 
                        Keys to export. This can be a regular expression
  -o NOT_KEYS, --not-key NOT_KEYS // Specifies that a prefix is not parsed and exported key, regular expression 
                        Keys Not to export. This can be a regular expression
  -t TYPES, --type TYPES // Specify which data types to analyze , for example string、hash、set、list、sortedset etc. , Multiple types can be specified , If the default analysis is not specified, all 
                        Data types to include. Possible values are string,
                        hash, set, sortedset, list. Multiple typees can be
                        provided. If not specified, all data types will be
                        returned
  -b BYTES, --bytes BYTES // Specify how much the output exceeds Byte Of key, for example --bytes 10240 Indicates that only output exceeds 10k The size of key
                        Limit memory output to keys greater to or equal to
                        this value (in bytes)
  -l LARGEST, --largest LARGEST // Output TOP How many key, for example --largest 100, Means output only top 100 Of key
                        Limit memory output to only the top N keys (by size)
  -e {raw,print,utf8,base64}, --escape {raw,print,utf8,base64} // Specifies the default encoding for the output , Default RAW
                        Escape strings to encoding: raw (default), print,
                        utf8, or base64.
  -x, --no-expire       With protocol command, remove expiry from all keys //  Only command by protocol Patterns work , It means not to output all expired key, Only those that never expire key
  -a N, --amend-expire N //  Only command by protocol Patterns work , Indicates to set the expiration time key increase N Seconds past due 
                        With protocol command, add N seconds to key expiry
                        time

4. Next, let's briefly analyze

[[email protected] ~]# rdb --command json dump.rdb > ./test
WARNING: python-lzf package NOT detected. Parsing dump file will be very slow unless you install it. To install, run the following command:

pip install python-lzf

Here we are reminded that there is no python-lzf Packets cause analysis to be slow , Let's install it . You can update it first yum Source , Prevent missing packages

[[email protected] ~]# yum update

Then install again

[[email protected] ~]# pip install python-lzf
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead.
Collecting python-lzf
  Downloading http://mirrors.tencentyun.com/pypi/packages/e3/33/b8f67bbe695ccc39f868ae55378993a7bde1357a0567803a80467c8ce1a4/python-lzf-0.2.4.tar.gz
Installing collected packages: python-lzf
  Running setup.py install for python-lzf ... error
    Complete output from command /usr/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-go178sdk/python-lzf/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-6oauj90r-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_ext
    building 'lzf' extension
    creating build
    creating build/temp.linux-x86_64-3.6
    gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I. -I/usr/include/python3.6m -c lzf_module.c -o build/temp.linux-x86_64-3.6/lzf_module.o -Wall
    gcc: error: /usr/lib/rpm/redhat/redhat-hardened-cc1: No such file or directory
    error: command 'gcc' failed with exit status 1
    
    ----------------------------------------
Command "/usr/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-go178sdk/python-lzf/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-6oauj90r-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-go178sdk/python-lzf/

There is an error gcc An error was reported when compiling , On github Search for , Is not installed redhat-rpm-config, Then let's install

$ sudo dnf install redhat-rpm-config

Install well redhat-rpm-config after , Install again python-lzf Wrong again. , This time, there are few hints Python.h, Check it online , It turned out that this time there was a lack of python-devel.

[[email protected] ~]# pip install python-lzf
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead.
Collecting python-lzf
  Downloading http://mirrors.tencentyun.com/pypi/packages/e3/33/b8f67bbe695ccc39f868ae55378993a7bde1357a0567803a80467c8ce1a4/python-lzf-0.2.4.tar.gz
Installing collected packages: python-lzf
  Running setup.py install for python-lzf ... error
    Complete output from command /usr/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-dxcuwlzv/python-lzf/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-9n6nf8r_-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_ext
    building 'lzf' extension
    creating build
    creating build/temp.linux-x86_64-3.6
    gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I. -I/usr/include/python3.6m -c lzf_module.c -o build/temp.linux-x86_64-3.6/lzf_module.o -Wall
    lzf_module.c:3:10: fatal error: Python.h: No such file or directory
     #include "Python.h"
              ^~~~~~~~~~
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    
    ----------------------------------------
Command "/usr/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-dxcuwlzv/python-lzf/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-9n6nf8r_-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-dxcuwlzv/python-lzf/

We directly yum The installation could not find , For example, as shown below , Let's try to search yum search once , eureka ,python36-devel.x86_64, Then install again .

[[email protected] ~]# yum install python-devel
Last metadata expiration check: 0:00:58 ago on Thu 18 Mar 2021 05:14:53 PM CST.
No match for argument: python-devel
Error: Unable to find a match: python-devel

[[email protected] ~]# yum search python3 | grep devel
Last metadata expiration check: 0:01:29 ago on Thu 18 Mar 2021 05:14:53 PM CST.
python3-qscintilla-qt5-devel.noarch : Development files for QScintilla-qt5 python3 bindings
boost169-mpich-python3-devel.x86_64 : Shared library symbolic links for Boost.MPI Python 3 component
boost169-openmpi-python3-devel.x86_64 : Shared library symbolic links for Boost.MPI Python 3 component
boost169-python3-devel.x86_64 : Shared object symbolic links for Boost.Python 3
python3-TurboGears2.noarch : Next generation front-to-back web development megaframework
python3-cherrypy.noarch : Pythonic, object-oriented web development framework
python3-hupper.noarch : Integrated process monitor for developing servers
python3-idle.i686 : A basic graphical development environment for Python
python3-idle.x86_64 : A basic graphical development environment for Python
python3-kobo.noarch : Python modules for tools development
python3-pycxx-devel.noarch : PyCXX header and source files
python3-werkzeug.noarch : The Swiss Army knife of Python web development
python36-devel.x86_64 : Libraries and header files needed for Python development
python38-devel.x86_64 : Libraries and header files needed for Python development
python38-idle.x86_64 : A basic graphical development environment for Python

We have solved these dependencies , We will install it again smoothly

[[email protected] ~]# pip install python-lzf
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead.
Collecting python-lzf
  Downloading http://mirrors.tencentyun.com/pypi/packages/e3/33/b8f67bbe695ccc39f868ae55378993a7bde1357a0567803a80467c8ce1a4/python-lzf-0.2.4.tar.gz
Installing collected packages: python-lzf
  Running setup.py install for python-lzf ... done
Successfully installed python-lzf-0.2.4

5. Formal analysis

Here is a common command , analysis rdb The file of top100 The big key, have access to lrzsz Download to local , Use Excel Analyze .

[email protected] ~]# rdb -c memory dump-6120957-redis-server-ignore-140016731-ignore-1-ignore.rdb -l 100  -f ./dump_memory.csv

Excel Analyzing memory snapshots CSV file

CSV Field description in

database	type	key	size_in_bytes	encoding	num_elements	len_largest_element	expiry
key Belongs to db Number	key For example string list set etc.	key name	key size ( Number of bytes )	Encoding mode (hashtable,ziplist,string etc. )	key The number of elements in	key Length of the largest element in	Expiration time

You can also use LOAD DATA INFILE The statement is imported into the database , Use SQL Statement analysis , Such as total memory usage of query 、 Query total key Number 、 Query specific type Of key Number and so on . I won't repeat