How to ensure the performance and security of Redis database? It's enough to read this Redis performance test and security optimization configuration guide

Contents of this chapter
0x00 Redis performance indicator monitoring

(1) Performance index

1. Basic activity indicator: basic activity
2. Performance index:
3. Memory index: memory
4. Persistence indicator
5. Error indicator: error
6. Description of other indicators

(2) Performance test tool

1. Redis benchmark command
2.redisbench tool
3.rdb memory analysis tool

(3) Benchmarking practices

3.1 single instance redis test in k8s

0x01 Redis security optimization


Non privileged operation
file right
Interface binding
Change default service port
Authentication configuration
Disable specific commands
Prevent string escape and NoSQL injection
Protect against attacks triggered by carefully selected inputs from external clients
Firewall restricted access
It is forbidden to store sensitive plaintext data in redis
Redis security configuration summary example

2.Performance Optimization

Key optimization items
Redis performance optimization summary example

Pre knowledge learning supplement
1. Introduction and installation of redis database foundation-

2. Introduction and use of basic data types of redis database-

3. Principle introduction and master-slave configuration of redis basic operation and maintenance-

4. Sentinel and cluster installation and configuration of redis basic operation and maintenance-

5. Installation and configuration of redis basic operation and maintenance in K8S-

7. Actual combat of redis database disaster recovery and backup enterprise-

8.Redis database client operation practice and pit entry and exit-

0x00 Redis performance indicator monitoring

(1) Performance index

Common indicators and parameters of Redis server:

redis-cli -a password info > redis-Performance.txt # We can export the info related information of redis server to a file
  # 2.clients:
  # 3.memory: 
  # 4.persistence: 
  # 5.stats: General Statistics
  # 6.Replication: 
  # 7.CPU: CPU usage
  # 8.cluster: 
  # 9.Keypass: key value pair statistics quantity information> info  # (1) Interactive viewing of redis server information
# Environment parameters for Server operation
os:Linux 5.11.0-25-generic x86_64

# Clients client related information

# Memory server running memory statistics

# Persistence persistent data related information

# Stats general statistics information

# Replication master-slave related indicator information

# CPU processor mode occupancy information

# Modules module loading

# Errorstats error status information

# Cluster cluster information

# Keyspace library id and key quantity related information

1. Basic activity indicator: basic activity

connected_clientsNumber of client connections
conected_lavesNumber of slave
master_last_io_seconds_agoThe number of seconds since the last master-slave interaction
keyspaceTotal number of key values in the database
grep -En "connected_clients|conected_laves|master_last_io_seconds_ago|keyspace" redis-Performance.txt
27:connected_clients:7 # # Number of client connections
28:connected_slaves:1  # # Number of slave connections

2. Performance index:

latencyRedis response time to a request
instantaneous_ops_per_secAverage total number of requests processed per second
hi rate(calculated)Calculated hit rate
grep -En "latency|instantaneous_ops_per_sec|hi rate" redis-Performance.txt

3. Memory index: memory

used_memoryMemory used
mem_fragmentation_ratioMemory fragmentation rate
evicted_keysMaximum number of key s removed due to memory limitation
blocked_clientsClients blocked due to BLPOP,BRPOP,or BRPOPLPUSH
grep -En "used_memory|mem_fragmentation_ratio|evicted_keys|blocked_clients" redis-Performance.txt
38:used_memory_human:1.96M      # # The total amount of memory allocated by the memory allocator from the operating system
40:used_memory_rss_human:4.99M  # # The memory occupation seen by the operating system and the memory occupation seen by the top command
42:used_memory_peak_human:2.15M # # Peak of redis memory consumption
54:used_memory_lua_human:37.00K #  # Amount of memory occupied by lua script engine

4. Persistence indicator

rdb_last_save_timeTimestamp of the last persistent save disk
rdb_changes_sice_last_saveThe number of changes to the database since the last persistence
grep -En "rdb_last_save_time|rdb_changes_sice_last_save" redis-Performance.txt
89:rdb_changes_since_last_save:0   # The number of changes to the database since the last persistence

5. Error indicator: error

rejected_connectionsThe number of connections rejected due to maxclient limit reached
keyspace_missesNumber of key value lookup failures (misses)
master_link_down_since_secondsDuration of master-slave disconnection (in seconds)
grep -En "rejected_connections|master_link_down_since_seconds" redis-Performance.txt
keyspace_misses:0    # The number of times the key value search fails (misses). It may be attacked by the hacker


6. Description of other indicators

# 1. If the copy backlog buffer is set too small, the instructions in it will be overwritten and the offset will not be found, thus triggering full synchronization
repl_backlog_size: 1048576

# 2. By viewing sync_ partial_ The err variable determines whether the backlog buffer needs to be expanded. It indicates the number of master-slave semi synchronous replication failures

(2) Performance test tool

1. Redis benchmark command

Description: redis performance test is realized by executing multiple commands at the same time, which are executed under the redis directory;

Official website reference:

Syntax parameters:

Usage: redis-benchmark [-h <host>] [-p <port>] [-c <clients>] [-n <requests]> [-k <boolean>]

# Parameter Description:
 -h <hostname>      Server hostname (default
 -p <port>          Server port (default 6379)
 -s <socket>        Server socket (overrides host and port)
 -a <password>      Password for Redis Auth
 -c <clients>       Number of parallel connections (default 50)
 -n <requests>      Total number of requests (default 100000)
 -d <size>          Data size of SET/GET value in bytes (default 2)
 --dbnum <db>       SELECT the specified db number (default 0)
 -k <boolean>       1=keep alive 0=reconnect (default 1)
 -r <keyspacelen>   right SET/GET/INCR Use random keys to SADD Use random valueuse this option to extend strings within parameters_rand_int _uu),The specified range of this parameter is 0 to keyspacelen-1 12 digits between.
 -P <numreq>        Pipeline <numreq> requests. Default 1 (no pipeline).
 -q                 Quiet. Just show query/sec values
 --csv              Output in CSV format
 -l                 Loop. Run the tests forever
 -t <tests>         Only run the comma separated list of tests. The test names are the same as the ones produced as output.
 -I                 Idle mode. Just open N idle connections and wait.

Basic instance:

# (1) 10000 requests are executed at the same time to test the performance (all default tests). The - q parameter allows the result to display only the number of requests executed per second
$ ./redis-benchmark -h -p 6379 -t set,lpush -n 10000 -q
$ ./redis-benchmark -n 10000  -q 
   # PING_INLINE: 41493.78 requests per second
   # PING_BULK: 44843.05 requests per second
   # SET: 42194.09 requests per second
   # GET: 44052.86 requests per second
   # INCR: 43290.04 requests per second
   # LPUSH: 42194.09 requests per second
   # RPUSH: 42372.88 requests per second
   # LPOP: 42194.09 requests per second
   # RPOP: 42194.09 requests per second
   # SADD: 43668.12 requests per second
   # HSET: 42372.88 requests per second
   # SPOP: 44843.05 requests per second
   # LPUSH (needed to benchmark LRANGE): 42553.19 requests per second
   # LRANGE_100 (first 100 elements): 21367.52 requests per second
   # LRANGE_300 (first 300 elements): 9451.80 requests per second
   # LRANGE_500 (first 450 elements): 6807.35 requests per second
   # LRANGE_600 (first 600 elements): 5350.46 requests per second
   # MSET (10 keys): 36363.64 requests per second

# (2) Run the test of the specified project. For example, we require only the SET and LPUSH commands to be run in quiet mode
$ redis-benchmark -t set,lpush -n 100000 -q
  # SET: 74239.05 requests per second
  # LPUSH: 79239.30 requests per second

# (3) Specify the eval script command for benchmarking
$ redis-benchmark -n 100000 -q script load "'set','foo','bar')"
  # script load'set','foo','bar'): 69881.20 requests per second

# (4) Select the size of the key space. By default, the benchmark runs for a single key, and we can usually simulate a more real workload space by using large keys.
# For example, if I want to run 1 million SET operations and use a random key for each operation among 100000 possible keys,
$ redis-cli flushall
$ redis-benchmark -t set -r 100000 -n 1000000

# (5) By default, each client only sends the next command when receiving the reply from the previous command. Redis supports pipeline, so it can send multiple commands at one time, which can be imagined as parallel.
# For example: run the benchmark in MacBook Air 11 "using a pipeline of 16 commands
redis-benchmark -n 1000000 -t set,get -P 16 -q
  # SET: 403063.28 requests per second
  # GET: 508388.41 requests per second

# (6) Benchmarking using Unix domain sockets
$ numactl -C 6 ./redis-benchmark -q -n 100000 -s /tmp/redis.sock -d 256

# (7) Using TCP loopback
$ numactl -C 6 ./redis-benchmark -q -n 100000 -d 256

Benchmark comparison between Redis and Memcached in memory databases:


# BIN=./redis-benchmark

for clients in 1 5 10 20 30 40 50 60 70 80 90 100 200 300
    for dummy in 0 1 2
        S=$($BIN -n $iterations -r $keyspace -d $payload -c $clients | grep 'per second' | tail -1 | cut -f 1 -d'.')
        if [ $(($S > $SPEED)) != "0" ]
    echo "$clients $SPEED"

Finally, the following are the results in graphical form generated using gnuplot:

Factors affecting benchmarking

  • 1) Workload (number of connected clients)
  • 2) Different versions of Redis
  • 3) Physical configuration of servers providing services (disk, network, CPU, memory). On multi CPU slot servers, Redis performance depends on NUMA configuration and process location.

  • The test was completed by 50 clients executing 2 million requests at the same time.
  • Use the loopback interface to perform the test.
  • The test is performed using a key space of 1 million keys.
  • The test is executed with and without pipelines (16 command pipelines).

    Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (with pipeline) / (without pipeline)

    $ ./ Redis benchmark - R 1000000 - N 2000000 - t get, set, lpush, lpop - p 16 -q# better than no pipeline.
    SET: 552028.75 requests per second
    GET: 707463.75 requests per second
    LPUSH: 767459.75 requests per second
    LPOP: 770119.38 requests per second

    $ ./redis-benchmark -r 1000000 -n 2000000 -t get,set,lpush,lpop -q
    SET: 122556.53 requests per second
    GET: 123601.76 requests per second
    LPUSH: 136752.14 requests per second
    LPOP: 132424.03 requests per second

  • 4) Compared with using the same hardware without virtualization, Redis runs slower on the VM (Redis is recommended as the preferred physical machine)
  • 5) Depending on the platform, unix domain sockets can achieve about 50% more throughput than TCP/IP loopback (for example, on Linux).
  • 6) Compared with TCP/IP loopback, the performance advantage of Unix domain socket tends to decrease when a large number of pipelines (i.e. long pipelines) are used.
  • 7) When using Ethernet network to access Redis, when the data size remains below the Ethernet packet size (about 1500 bytes), the pipelined aggregation command is particularly effective. When processing 10 byte, 100 byte or 1000 byte queries, it will produce almost the same throughput.

2.redisbench tool

Description: the officially recommended redis benchmark has no way to specify the cluster mode during cluster benchmark testing. Redis & redis cluster benchmark tool is introduced here to facilitate the processing of cluster benchmark testing. Download address:

redisbench features

  • Developed and built with Golang
  • redis single instance can be tested
  • redis cluster can be tested
  • Multiple cores can be used
  • It supports running on multiple machines at the same time and is used to test large redis clusters (the same machine hardware is required)

Format syntax:

./redisbench -h
-a string   #Redis instance address or Cluster addresses. IP:PORT[,IP:PORT]
-c int      #Clients number for concurrence (default 1)
-cluster    #true: cluster mode, false: instance mode
-d int      #Data size in bytes (default 1000)
-ma string  #addresses for run multiple testers at the same time
-mo int     #the order current tester is in multiple testers
-n int      #Testing times at every client (default 1)

Basic example:

# Test single instance mode
./redisbench -a -c 200 -n 20000 -d 3

# Test cluster
./redisbench -cluster=true -a, -c 500 -n 2000 -d 3

# Use multiple test nodes
./redisbench -cluster=true -a, -c 500 -n 2000 -d 3 -ma,, -mo 1 &
./redisbench -cluster=true -a, -c 500 -n 2000 -d 3 -ma,, -mo 2 &
./redisbench -cluster=true -a, -c 500 -n 2000 -d 3 -ma,, -mo 3

Tips: the test results will be automatically printed out: request value, request time, TPS is not actually demonstrated here. Interested friends can download the test by themselves.


3.rdb memory analysis tool

Description: RDR is a tool for parsing redis rdbfile. Compared with redis RDB tools, RDR is implemented by golang and is faster.

  • Analyze which Key value in Redis memory occupies the most memory
  • It is analyzed that the Key at the beginning of Redis memory occupies the most, which is conducive to memory optimization
  • Redis Key value is displayed in Dashboard, which is more intuitive

Installation and download address:

matters needing attention:

  • 1. Add the executable permission Chmod + X RDR before using Linux and windows_ linux

Basic syntax:

# RDR parameter interpretation
show Web page display rdbfile Statistics for
keys from rdbfile Get all key
help help

Basic instance (take Linux as an example):

#1. Analyze and count the usage proportion of various types in multiple redis RDBS 
./rdr-linux keys *.rdb

./rdr-linux dump dump.rdb
{"CurrentInstance": "dump.rdb",
"LargestKeyPrefixes": {
   "list": [
      "Type": "list",
      "Key": "site",
      "Bytes": 144,
      "Num": 1

#2. The web page displays the analysis results
./rdr-linux show -p 8080 *.rdb
start parsing...
parse dump.rdb  done
parsing finished, please access http://{$IP}:8080

(3) Benchmarking practices

3.1 single instance redis test in k8s

Environmental description:

# Host node running the Pod
osImage: Ubuntu 20.04.1 LTS
kernelVersion: 5.4.0-42-generic
kubeProxyVersion: v1.19.6
kubeletVersion: v1.19.6
containerRuntimeVersion: docker://19.3.14

$ Server Relevant information
os:Linux 5.4.0-42-generic x86_64

$ redis Configured memory limit

$ cpu Relevant information
Model name: Intel(R) Xeon(R) CPU E3-1220 V2 @ 3.10GHz
 Physics CPU number: 1
 logic CPU number: 4
CPU Number of cores: 4



# Test 1 Execute the set command and get command 10 million times, in which the size of each read is 256 bytes, and the number of requesting clients is 50 by default.
~$ redis-benchmark -h -a 123456 -d 256 -t set,get -n 10000000 -q                            
SET: 42445.36 requests per second
GET: 49504.70 requests per second
# -Pod resource peak during test
very 1.0s: kubectl top pod -n database redis-cm-0  Tue Sep  7 20:36:50 2021
NAME         CPU(cores)   MEMORY(bytes)
redis-cm-0   848          18Mi

# Test 2 Similarly, execute the set command and get command for 10 million times, in which the size of each read is 256 bytes. The only difference is that pipelined-P 16 is used for testing (it can be seen that the number of set and get requests per second is significantly increased).
~$ redis-benchmark -h -a 123456 -d 256 -t set,get -n 10000000 -P 16 -q                            
SET: 96019.98 requests per second
GET: 316575.91 requests per second

# -Peak Pod resources during test
very 1.0s: kubectl top pod -n database redis-cm-0  Tue Sep  7 20:46:50 2021
NAME         CPU(cores)   MEMORY(bytes)
redis-cm-0   457m         337Mi


Practical test

  • 1) Insert 100000 data through shell pipe and redis pipe for comparison

    echo "start time: $(date +%s)"
    for ((i=0;i<100000;i++));do
    echo -en "helloworld-redis-${i}" | redis-cli -h --no-auth-warning -a 123456 -x set username${i} >> ok.txt
    echo "Completion time: $(date +%s)"
    echo "start time: $(date +%s)"
    python3 >> set-command.txt
    cat set-command.txt | redis-cli -h -a 123456 --no-auth-warning --pipe
    echo "Completion time: $(date +%s)"
    tee <<'EOF'
    for i in range(100000):
    print('set name'+str(i)+' helloworld-redis-'+str(i))
  • 2) The execution efficiency of script insertion is recorded by using the time command

    ## Insert data in redis pipe mode
    ~/k8s/benchmark$ time ./
    # Start time: 1631020862
    # All data transferred. Waiting for the last reply...
    # Last reply received from server.
    # errors: 0, replies: 100000
    # Completion time: 1630862
    # real    0m0.466s
    # user    0m0.126s
    # sys     0m0.035s
    # Keyspace
    ## Insert data in shell pipe mode
    ~/k8s/benchmark$ time ./
    # Start time: 1631021312
    # Completion time: 1631021921
    # real    10m9.265s # Total time spent from the beginning to the end of the program (including CPU).
    # user    3m44.411s # The program itself and the time of calling the subprocess.
    # sys     1m55.435s # The execution time of the system call called by the program itself or indirectly.
    # Keyspace

Tips: it can be seen from the above results that the total time consumption of real of the two methods is very different. The efficiency of redis pipe method is not an order of magnitude compared with that of ordinary shell pipe method, so try to use redis pipe method to submit data in the development program.

# In order to facilitate the follow-up demonstration, grip inserted 80W pieces of data into the database in about 4s.
start time: 1631022423
All data transferred. Waiting for the last reply...
Last reply received from server.
errors: 0, replies: 800000
 Completion time: 1631022427
real    0m4.023s
user    0m0.885s
sys     0m0.187s

0x01 Redis security optimization


Description: Redis provides access control, code security issues, and selecting attacks that can be triggered from outside by malicious input. Our operation and maintenance personnel need to configure them accordingly to improve security.

Non privileged operation

Description: redis can run without root permission. It is recommended to run it as a non privileged redis user only for this purpose. This way can prevent the possibility of CONFIG SET/GET directory and other similar runtime configuration instructions to the greatest extent..

#It is necessary to set up a separate redis account. redis crackit takes advantage of the root user feature to reset authorized_keys.  First create a redis account, and then start it through this account.
$ useradd redis
$ setsid sudo -u redis redis-server /etc/redis.conf
$ ps -elf|grep redis   #You can see that the redis user started
  # 4 S root      9048     1  0  80   0 - 59753 poll_s 19:43 ?        00:00:00 sudo -u redis redis-server /etc redis.conf
  # 4 S redis     9049  9048  0  80   0 - 38471 ep_pol 19:43 ?        00:00:00 redis-server 

file right

Description: because the redis password is stored in clear text in the configuration file, we need to restrict the access permission of the redis file directory. For example, set the permission of the redis home directory to 700(rwx ------), if redis The conf configuration file is independent of the redis home directory, and the permission is modified to 600(rw -------).

# file right
chmod 700 /opt/redis/redis-5.0.4/
chmod 600 /etc/redis.conf

# Owner and group
chown redis:redis /etc/redis.conf
chown redis:redis /opt/redis/redis-5.0.4/


Interface binding

Description: except for the trusted clients in the network, everyone should refuse to access the Redis port. Therefore, the server running Redis should only be directly accessed by the computer using Redis to implement the application.

If the server has two network interfaces (one area a and one area B), if only the machine access in area a is required, it will only be bound to the network interface in area A. if the server itself accesses, it will only be bound to the local loopback interface.

# Through Redis Add the following line to the conf file to bind Redis to a single interface:

Tips: note that you can bind IPV6 in addition to IPV4

Change default service port

Description: in addition to specifying the binding interface, we can also change the default redis service port to prevent hackers from scanning and detecting redis services.

# Change the default service disconnect from 6379 to 63791
port 63791


Authentication configuration

Description: it is very necessary to set an authentication password for Redis server. The following is a summary of several ways Redis configures password authentication:

Operation process:

# 1. Through redis Conf file. Redis needs to be restarted after this method is modified.
vim /etc/redis.conf
requirepass WeiyiGeek  # WeiyiGeek is the authentication password
masterauth  WeiyiGeek  # Configure the authentication password of the master node. Note that if the master is configured with a password, the slave must also configure the corresponding password parameters, otherwise it cannot be copied normally

# 2. Configure through the command line. This method has the advantage of no need to restart Redis.
redis[1]> config set requirepass my_redis  
redis[1]> config get requirepass  
1) "requirepass" 
2) "my_redis"  


Log in to Redis server with password authentication:

# Method 1: password plaintext will be recorded in the system command execution history (extremely not recommended / unsafe)
redis-cli -h -p 6379 -a WeiyiGeek

# Mode 2: interactive configuration
redis-cli -h -p 6379
redis> auth WeiyiGeek # OK

Note: the AUTH command, like all other Redis commands, is sent in an unencrypted manner, so it cannot prevent attackers who have sufficient access to the network to eavesdrop. Therefore, for highly sensitive data, it is recommended to configure TLS support (Redis optionally supports TLS on all communication channels) to encrypt data and command transmission.


Disable specific commands

Description: we can disable the commands in Redis or rename them to unpredictable names, so that ordinary clients are limited to a specified set of commands. For example, the vulnerability uses the two commands of config/save to complete the attack.

Since redis has no user permission restrictions, it is recommended to disable or rename dangerous commands using the rename configuration item, so that attackers who do not know the renaming rules cannot execute such commands FLUSHDB, FLUSHALL, KEYS, PEXPIRE, DEL, CONFIG, SHUTDOWN, BGREWRITEAOF, BGSAVE, SAVE, SPOP, SREM, RENAME, DEBUG, EVAL

For example, ordinary users may not be able to call the Redis CONFIG command to change the instance configuration, but the system providing and deleting instances should be able to do so.

# redis.conf configuration file
# Mode 1 The config / flush command is renamed to an unpredictable name
rename-command CONFIG b840fc02d524045429941cc15f59e41cb7be6c52
rename-command FLUSHALL b840fc02d524045429941cc15f59e41cb7be6c53

# Mode 2 Disable it completely (or any other command) by renaming it to an empty string
rename-command CONFIG ""
rename-command FLUSHALL  ""

Tips: note that the redis server service needs to be restarted after configuration.


Description: create an access (or Debug) log for redis (set according to requirements). When establishing redis honeypot, if there is an attack attempt, it will find and monitor redis security status in time, and can monitor cmdstat_* Indicator information alarm;

# Execute info commandstats to see the number of times the command is executed, the CPU time consumed by the command (unit: ms), and the average CPU time consumed by executing each command (unit: ms)

Logging configuration:

logfile "/usr/local/redis/redis.log" #Log file storage directory
loglevel verbose  #Record access information

Prevent string escape and NoSQL injection

Description: Redis protocol does not have the concept of string escape, so it is generally impossible to inject using ordinary client libraries, but it is possible to construct malicious scripts through Lua scripts executed by EVAL and EVALSHA commands.

> eval "return {KEYS[1],KEYS[2],ARGV[1],ARGV[2]}" 2 key1 key2 first second

Solution: applications should avoid using strings obtained from untrusted sources to form the body of Lua scripts.

Tips: evalsha evaluates cached server-side scripts through its SHA1 summary. Scripts are cached on the server side using the SCRIPT LOAD command. This command is the same as EVAL in other respects.


Protect against attacks triggered by carefully selected inputs from external clients

Description: it is possible for attackers to construct malicious data structures and insert them into Redis database, which may trigger the ill conditioned (worst-case) algorithm complexity of data structures implemented inside Redis.

For example, an attacker can provide a set of strings known to be hashed into the same bucket to the hash through a Web form, so as to turn the expected time (average time) of O(1) into the worst case of O(N), consume more CPU than expected, and eventually lead to a denial of service.

Solution: in order to prevent this specific attack, Redis uses the pseudo-random seed of each execution for the hash function.


Firewall restricted access

Description: the security configuration was made for the redis server service level earlier. Here, restrictions are made for the network level to allow access only to the specified IP address. The advantage of configuring a firewall on the host is to prevent east-west traffic in the same network segment.

Firewall setting command on Linux system:

iptables -A INPUT -s x.x.x.x -p tcp --dport 6379 -j ACCEPT  #If you need access from other machines or set the slave mode, remember to add the corresponding firewall settings (Centos6)
firewall-cmd --add-rich-rule="rule family="ipv4" source address="x.x.x.x" port protocol="tcp" port="6379" accept" --permanent  #(Centos7)

System firewall setting command on Windows:

New-NetFirewallRule -Name "redis-server-access" -DisplayName "redis-server" -Description "redis-server Client access firewall rules" -Direction Inbound -LocalPort 6379 -RemoteAddress x.x.x.x -Protocol TCP -Action Allow -Enabled True
Get-NetFirewallRule -Name "redis-server-access"  | Format-Table

It is forbidden to store sensitive plaintext data in redis

Description: Redis is designed to provide high-performance KV services. At least at present, it is relatively weak in access control and data persistence. Therefore, from the application level, it is not recommended to use Redis to store sensitive information, such as authentication passwords.

Redis security configuration summary example

Security configuration example:

# Configuration file VIM / etc / redis / redis conf
# 1. Operate the trusted intranet and try to avoid public network access (if there are other fixed IP addresses in the intranet, a firewall needs to be set)

# 2. Bind the network interface monitored by redis (multiple IPS can be bound at the same time through the redis configuration item bind), and change 6379 to another port (or use unix pipeline for data management)
port 63791  

# 3. Enable redis password authentication and set high complexity password settings. Due to the high query efficiency, auth commands can be processed more than 10w times per second (so the intensity needs to be increased)
# echo -e "weiyigeek"|sha256sum 
requirepass 097575a79efcd7ea7b1efa2bcda78a4fc7cbd0820736b2f2708e72c3d21f8b61

# 4. Log file storage directory and record redis access information.
# debug (a lot of information, useful for development/testing)
# verbose (many rarely useful info, but not a mess like the debug level)
# Notice (modestly verbose, what you want in production probability) default
# warning (only very important / critical messages are logged)
logfile "/usr/local/redis/redis.log" 
loglevel verbose  

# 5. Protection mode is enabled by default. (no) it should only be disabled if
# -Are you sure you want clients of other hosts to connect to Redis
# -Even if authentication is not configured, there is no specific set of interfaces
# -Use the "bind" instruction to explicitly list.
protected-mode yes

# 6. Rename special commands (as required)
rename-command CONFIG b840fc02d524045429941cc15f59e41cb7be6c52
rename-command FLUSHDB b840fc02d524045429941cc15f59e41cb7be6c53
rename-command FLUSHALL b840fc02d524045429941cc15f59e41cb7be6c54
rename-command EVAL b840fc02d524045429941cc15f59e41cb7be6c55
rename-command DEBUG b840fc02d524045429941cc15f59e41cb7be6c56
rename-command SHUTDOWN b840fc02d524045429941cc15f59e41cb7be6c7

2.Performance Optimization

Description: Redis development and operation and maintenance personnel pay more attention to some configuration optimization of Redis itself, such as AOF and RDB configuration optimization, data structure configuration optimization, etc., but they don't know or care about whether the operating system needs to do some configuration optimization for Redis. However, facts have proved that a good system operation configuration can escort the good operation of Redis services.

Key optimization items

  • Step 1.vm.overcommit_memory best practices
    Such logs may appear when Redis is started, and then figure out what is overcommit?
    Description: the Linux operating system replies yes to most requests for memory in order to run more programs. Because memory will not be used immediately after applying for memory, this technology is called overcommit.

    # If Redis has the above log at startup, it means ` VM overcommit_ Memory = 0 `, Redis prompts to set it to 1.
    # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the 
    command 'sysctl -w vm.overcommit_memory=1' for this to take effect.
    # Note: the available memory in this article represents the sum of physical memory and swap.
  • Redis suggests setting this value to 1 so that fork can execute successfully under low memory (set a reasonable maxmemory to ensure that the machine has 20% ~ 30% idle memory).
  • Centralized management of aof rewriting and rdb bgsave.

    Tips : In the log Background save Represents bgsave and bgrewriteaof,What should the operating system do if the currently available memory is insufficient fork. If vm.overcommit_memory=0,Means that if there is no available memory, the memory application fails, corresponding to Redis namely fork Execution failed at Redis Your log will appear:`Cannot allocate memory `


  • Step 2.vm.swapniess best practices
    Description: swap is important for the operating system. When the physical memory is insufficient, you can swap out some memory pages to solve the urgent problem. However, there is no free lunch in the world. Swap space is provided by hard disk. For applications requiring high concurrency and high throughput, disk IO will usually become a system bottleneck. In Linux, it is not necessary to wait until all physical memory is used up before using swap. The system parameter swppiness determines the tendency of the operating system to use swap. The value range of swap is 0 ~ 100. The larger the value of swap, the higher the probability that the operating system may use swap, and the lower the value of swap, indicating that the operating system prefers to use physical memory.

If Linux > 3.5, VM Swapniess = 1 (prefer swap to OOM killer), otherwise VM Swapniess = 0 (prefer OOM killer to swap) to achieve the following two goals:

1. Make Redis fast enough when there is enough physical memory.
2. Avoid the death of Redis when the physical memory is insufficient (if Redis is currently highly available, death is better than blocking).

Operation and maintenance tips: OOM(Out Of Memory) killer mechanism means that when the Linux operating system finds that there is insufficient available memory, it forces to kill some user processes (non kernel processes) to ensure that the system has enough available memory for allocation.


  • Step best practices
    Redis may see the following logs when starting:

    WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.

Tips: from the prompt, Redis suggests modifying the related configuration of transparent tiger pages (THP). The Linux kernel adds the transparent tiger pages (THP) feature in the 2.6.38 kernel, which supports the allocation of large memory pages (2MB), which is enabled by default. When enabled, the speed of fork child processes can be reduced, but after fork, each memory page changes from 4KB to 2MB, which will greatly increase the memory consumption of the parent process during rewriting. At the same time, the copy memory page unit caused by each write command is enlarged by 512 times, which will slow down the execution time of write operations and lead to a large number of slow queries of write operations.

Therefore, it is recommended to disable this feature in the Redis log. The disabling method is as follows: echo never > / sys / kernel / mm / transparent_ hugepage/enabled


  • Step 4.Transparent Huge Pages
    Redis may see the following log when starting: warning you have transparent tiger pages (THP) support enabled in your kernel This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_ hugepage/enabled' as root, and add it to your /etc/rc. local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.

From the prompt, Redis suggests modifying the relevant configuration of transparent tiger pages (THP). The Linux kernel adds the transparent tiger pages (THP) feature in the 2.6.38 kernel, which supports the allocation of large memory pages (2MB), which is enabled by default. When enabled, the speed of fork child processes can be reduced, but after fork, each memory page changes from 4KB to 2MB, which will greatly increase the memory consumption of the parent process during rewriting. At the same time, the copy memory page unit caused by each write command is enlarged by 512 times, which will slow down the execution time of write operations and lead to a large number of slow queries of write operations. For example, a simple incr command will also appear in slow queries. Therefore, it is recommended to disable this feature in Redis logs. The disabling methods are as follows:

# The THP configuration is still effective after the machine is restarted
tee -a /etc/rc.local <<'EOF'
echo never >  /sys/kernel/mm/transparent_hugepage/enabled


  • Step 5. Optimized configuration of oom killer
    The OOM killer will selectively kill the user process when the available memory is insufficient. It will set a weight for each user process. The higher the weight, the higher the probability of being "attacked". On the contrary, the lower the probability. The weight of each process is stored in / proc / {progress_id} / oom_ In score, this value is controlled by / proc/{progress_id}/oom_adj control, OOM_ The minimum value of adj is different in different Linux versions. You can refer to oom in Linux source code H (from - 15 to - 17)

When OOM_ When adj is set to the minimum value, the process will not be killed by OOM killer. The setting method is as follows:

# command
echo {value} > /proc/${process_id}/oom_adj

# script
for redis_pid in $(pgrep -f "redis-server")
  echo -17 > /proc/${redis_pid}/oom_adj


  • Step 6. Set the number of open files, handles and the maximum number of processes per user
    Description: the following parameters are mainly set to the maximum number of Linux file handles that can be used by a single process to avoid abnormal error reporting under high concurrency. Suggestions mentioned officially in Redis

    # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
    First line: Redis It is suggested that open files At least set it to 10032, so how does this 10032 come from? because maxclients The default is 10000. These are used to handle client connections. In addition, Redis Up to 32 file descriptors will be used internally, so 10032 here = 10000 + 32. 
    # Redis can't set maximum open files to 10032 because of OS error: Operation not permitted.
    Second line: Redis Cannot open files Set to 10032 because it does not have permission to set.
    # Current maximum open files is 4096. Maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit –n'.
    Line 3: current system open files It's 4096, so maxclients Set to 4096-32=4064 If you want to set higher maxclients,Please use ulimit -n To set. From the above three lines of log analysis, we can see that open files Restricted priority ratio maxclients Big.

    terms of settlement:

    # temporary
    ulimit –Sn 10032
    # permanent
  • soft nofile 10032
  • hard nofile 10032
  • soft nproc 65535
  • hard nproc 65535


  • Step 7.TCP backlog log queue optimization
    Description: the default TCP backlog of Redis is 511, which can be adjusted by modifying the configuration of TCP backlog. If the TCP backlog of Linux is less than the TCP backlog set by Redis, the following logs will be seen when Redis is started:

    # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.


    # see
    cat /proc/sys/net/core/somaxconn
    # modify
    echo 511 > /proc/sys/net/core/somaxconn


  • Step 8. Ensure the consistency of Redis server clock
    Description: we know that Redis Sentinel and Redis Cluster, which require multiple Redis instances, may involve multiple servers. Although Redis does not have strict requirements on the clocks of multiple servers, if the clocks of the servers of multiple Redis instances are inconsistent, it is very difficult to troubleshoot the logs of some abnormal situations, such as the failover of Redis Cluster. If the log times are different, it will cause great trouble for us to troubleshoot the problems (Note: it will not affect the cluster function, and the cluster nodes depend on their own clocks). Generally, NTP service will be provided in the company to provide standard time service, so as to achieve the effect of correcting the clock

For example, the NTP service is synchronized once every hour

0 * * * * /usr/sbin/ntpdate > /dev/null 2>&1

Redis performance optimization summary example

System optimization configuration

# -Set memory allocation policy
sudo sysctl -w vm.overcommit_memory=1

# -Try to use physical memory (fast) for kernel version greater than > = 3 X (prefer swap to OOM killer)
sudo sysctl -w vm.swapniess=1

# -Disable THP feature to reduce memory consumption
echo never > /sys/kernel/mm/transparent_hugepage/enabled

# -OOM killer feature optimization
for redis_pid in $(pgrep -f "redis-server")
  echo -17 > /proc/${redis_pid}/oom_adj

# -Set the number of open files, handles and the maximum number of processes per user
tee etc/security/limits.conf <<'EOF'
*  soft    nofile          10032
*  hard    nofile          10032
*  soft    nproc           65535
*  hard    nproc           65535

# -SYN queue length setting this parameter can accommodate more networks waiting to connect.
echo 511 > /proc/sys/net/core/somaxconn
sudo sysctl -w net.ipv4.tcp_max_syn_backlog=2048 

# -Synchronize time every hour
0 * * * * /usr/sbin/ntpdate > /dev/null 2>&1

Application configuration optimization

# Maximum number of client connections (it needs to be adjusted according to the actual situation, which is related to the open files of the system, and the number value is open files(10032) - 32)
maxclients 10000

# Key items of cluster configuration optimization
# Cluster timeout. If this time is set too small, the operation of reselecting the Master may be caused by network fluctuations
cluster-node-timeout 5000
# After writing, the master node must be synchronized to a slave to prevent data loss (the requirement is that its slave node must be > = 1)
min‐replicas‐to‐write 1

Optimization in application

# (1) Query execution time refers to the time spent on executing a query command, excluding IO operations such as client response (talking) and sending reply
redis> SLOWLOG LEN   # Manage the slow logs of redis and view the current number of logs 
redis> SLOWLOG RESET # Clear the slowlog, and the upper LEN becomes 0

# (2) Disconnect
# List all connected clients
addr= fd=5 age=10 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=client
# Kill the connection of the current client

Original link:

It's not easy to write this article. If you think this article is good, please give this column [point a praise, put a coin, collect a collection, close a note, and turn a hair] (the five great feelings in the world), which will affirm me. Thank you!.

Source of this article My Blog site or WeiyiGeek public account as well as My BiliBili column (please mail me for technical exchange and friend chain exchange). Thank you for your support! (๑′ᴗ‵๑) ❤
Welcome all like-minded friends to study and exchange together. If the article is wrong, please leave your valuable knowledge and suggestions through email[ ]Contact me!

Tags: security Operation & Maintenance Redis

Posted by paulytrick on Thu, 14 Apr 2022 10:54:30 +0930